I've started a Google Group for all those interested in the future of alternative languages (i.e. not Java) on the JVM. A number of you knew this was coming, and I've already invited folks that expressed an interest in my RedMonk Unconference session at JavaOne. The rest of you are also invited.
Please plan to talk shop about language implementation strategies, pain points on the JVM, and what we can do to build a common set of libraries, frameworks, and patterns to ease and improve the Java platform's support for many languages. The time for action is now, my friends, and we have a wealth of information across all our language projects to draw from. We must come together to build a stronger base for everyone to stand upon.
I welcome you to the future.
Sunday, May 27, 2007
JVM Languages: The Future
Saturday, May 26, 2007
Foo Camp 2007
I've been invited to this year's Foo Camp. Since it sounds like a great time and there's a bunch of other folks going I'd like to talk to, I'm accepting the invitation. My purpose posting this entry is to let other campers know I'll be there and try to hook up and chat a bit beforehand. So drop me a line...firstname.lastname@sun.com.
FYI, if any other campers are coming up from Oakland, I'm renting a car. My times are listed on the "Rides Offered" page.
I'm looking forward to a fun weekend!
Tuesday, May 22, 2007
The Final Bugs
We're within weeks of a final JRuby 1.0. We've pared down the bugs that we think we can or must fix for 1.0 and this email is a summary of the ones we need help with. So whatever time you can spare, please have a look and help resolve these.
In order of decreasing priority in JIRA:
JRUBY-820: Net::HTTP.get behaves differently form MRI, failing to get UTF8 properly
-and-
JRUBY-828: UTF-8 regular expressions aren't working
I believe these are both largely the same problem, and it's probably the most visible remaining bug we need to fix. Regular expressions in JRuby currently do not work well with unicode strings, both as incoming match strings and as the regular expression strings themselves. REXML uses /u regular expressions, which is why I believe these issues are closely related. Wes Nakamura has commented that he's looking into 828, but more eyes will help ensure this is fixed.
I believe these are true blocking bugs for 1.0, and I'm not comfortable doing a release if they are not resolved.
JRUBY-971: Make it clear which Java method maps to each equality method
I think we have subtle equality bugs remaining largely because we don't use a consistent naming convention for the Java implementations of Ruby methods like ==, ===, eql? and so on. We should make a quick sweep through the system and make sure all core JRuby code is using the same Java method names for all of these, and binding them in the same way. Note: This is actually the cause of some set_trace_func bugs, since the default === impl should actually invoke ==, causing trace events for both.
JRUBY-969: Add rake and rspec to standard JRuby distribution for 1.0
We have decided to ship both Rake and RSpec with JRuby 1.0, since they are both well-accepted and widely used (moreso for Rake, but RSpec has gained a lot of acceptance the past year). But the mechanism of including them is unclear. I do not want to commit installed gems to SVN; I would prefer to just install them as needed for testing and when building the JRuby distribution. Thoughts?
JRUBY-672: java.lang.Class representation of Ruby class not retrievable
This bug is basically just looking for a way to get at the actual java.lang.Class representing a Ruby-based extension of a Java class. There may have been an API added with Bill's work, or this could be simple to add.
JRUBY-914: JRuby's BigDecimal needs to round
Stu submitted a patch for this, but it unfortunately depends on behavior only present in Java 5 and higher. Since most of us are pretty unfamiliar with BigDecimal, we don't know of a good workaround to support it correctly in Java 1.4.
JRUBY-822: jruby fails to report process exit status correctly
I think Nick may have a grip on this one, but if anyone can offer suggestions as to why the exit codes are apparently shifted in this way, please let us know.
JRUBY-966: Various issues with LocalJumpError not being created early enough
-and-
JRUBY-767: next in an eval should produce a local jump error
These are likely going to be up to Tom and I since they involve pretty deep runtime/interpreter work. But we're also looking for any remaining known issues with LocalJumpError/JumpException that are still out there. I believe we've fixed all such externally-reported issues at the moment.
JRUBY-888: Make regular distro/build create the all-in-one jar by default
We waffle back and forth on this. What's the answer? An all-in-one jar by default, or on request?
JRUBY-873: ant test thread tests sometimes run forever
I managed to get a trace on this one. It appears that some of the tests cause a deadlock between stopping a thread and other operations. But I don't believe anyone's seen it in normal execution yet (or at least nobody's reported it). I'm probably the only one familiar enough with the threading subsystem to fix it, but I'd like to know if anyone's seen other issues.
JRUBY-98: allow "/" as absolute path in Windows
-and-
JRUBY-36: Dir['...'] incompatibilities between Ruby and accross platforms
-and-
JRUBY-61: IO CRLF compatibility with cruby on Windows
-and-
JRUBY-644: Windows line delimiter (\r\n) cause position errors
These blasted Windows pathing and newline bugs just seem to hang on forever. Nobody wants to fix them. There must be someone out there that can help resolve these, yes?
JRUBY-884: Create and consolidate extension and non-standard options
There are a number of configuration options for JRuby. Some have their own flags, like -O to disable ObjectSpace and -C to force compilation of a target script. But many others are only configurable via Java system properties. What we really could use from the community is some idea which settings are worth adding special flags for. We can handle the rest. My personal opinion is that the new -J flag that allows passing flags to the JVM is actually enough.
By my estimation, the rest of the bugs are either almost done or are trivial enough they could be punted to a post 1.0 release. But if we can get these issues knocked down soon (starting with those top couple), 1.0 is going to be an extremely solid release.
Monday, May 14, 2007
Big Plans
I just came across this amusing entry from an old, now-defunct blog of mine on LiveJournal.
The last line pretty much says it all. I must have been really frustrated with my job at the time:
Someone should pay me to sit at home and do great things.(If you don't get the joke: I now work for Sun Microsystems, at home, on JRuby...so if JRuby's ever considered a "great thing", the prophecy will be fulfilled.)
And yes, I've seen the Microsoft news. I'd hate to be an OSS developer or apologist at Microsoft today. If Sun did something like this I'd resign.
Thursday, May 10, 2007
JSR-292 Summit Wrap-Up
Today Tom and I met with fellow language implementers Oti Humbel, Charlie Groves, and Tobias Ivarsson to start discussions about JSR-292 (invokedynamic) with John Rose, longtime VM implementer and new spec lead. It was an excellent discussion, with John listening patiently to the implementation challenges we've had on JRuby and Jython and offering suggestions. He took copious notes and posted them here.
We have the second edition of the JRuby on Rails talk tomorrow morning, so I'm signing out now. But check out John's post, it's a great first look at a promising start on 292 work.
Monday, May 07, 2007
"the solution is JRuby"
Big news today from ThoughtWorks Studios! Their collaborative development project management solution Mingle (think SourceForge, but pretty and usable) will be the first commercially-distributed Rails application to run on JRuby. Jon Tirsen summarizes the justification for this move in a Studios blog post.
The reasons are obvious ones to those of us who've poured our hearts and souls into making JRuby succeed, but they're excellent points to call out for the Ruby community at large. Mingle 1.0 will be released running with a Jetty web front-end and a Derby database back-end. Mingle 1.1 will be released as a WAR file (Rails-in-a-WAR courtesy of GoldSpike) targeting any app server and any database.
This is a big day for JRuby. ThoughtWorks is one of the most respected names in the Ruby world, and we're proud that they've chosen JRuby as their enterprise deployment solution. We're happy to have ThoughtWorkers like Ola Bini on the team, and look forward to collaborating with ThoughtWorks to continue improving JRuby into the future.
Wednesday, May 02, 2007
JRuby: The T-Shirt!
Yes, I've managed to secure a small order of JRuby t-shirts for giveaway at JavaOne and RailsConf. And they're pretty sweet, have a look at the logo on the front:
And the back will have in white the words:
include Java
How cool is that? The logo was designed and contributed by "Liz" (Liz, let me know if you want me to post contact info or anything...great work) and arranged by Damian Steer, longtime JRuby contributor. I handled the legwork of arranging the shirts, and managed to secure a little funding to get them paid for.
So! How does one get one of these slicko t-shirts? Well, the first round of giveaways goes to folks who've made contributions in the past year. And by contributions I mean either patches or some serious research and legwork on a bug. Obviously there's a core group of folks that should automatically get shirts, so contact me about it if you think you qualify and I'll try to set one aside.
Otherwise...You've got the next week or two to fix a bug, post a patch, and otherwise make yourself useful to the JRuby project. I'm gonna be pretty strict on this...if you want the goods, you've got to help out. May is 1.0 cleanup time, with an RC coming out soon and hopefully a final not too long after. So if you'd been considering helping out, now's the time!
Visit the JRuby homepage, check out the JRuby bug tracker, and grab JRuby source. The details of contributing are linked from the front page of the JRuby wiki.
Update: Oh yeah...how will you get the shirts. The easiest way will be to hunt down Tom and I at the JRuby booth (in Sun's section) between 1PM and 3PM every day. We'll verify that you're not a slacker and make the arrangements. We're looking forward to meeting contributors in person. Otherwise, email me and we'll try to make other arrangements.
For RailsConf, I'll just have the shirts at the hotel, and we'll figure something out. Maybe I'll just bring the box to our training and session there.
One JRuby on Rails Talk Already Full!
Tom and I are giving our JRuby on Rails talk twice at JavaOne next week, and while wrestling with ScheduleBuilder I discovered something surprising: the thursday session is already FULL! Perhaps I underestimated how many people would be interested in seeing us prattle on about the future of Java web development with Rails. I expected that two sessions was extremely optimistic of the scheduling committee. It appears I was quite wrong!
If you haven't signed up for the JRoR session yet, you better do it quick. I'd hate for any of my regular readers to miss out on attending the session. It's gonna be a great show!
Tuesday, May 01, 2007
Dynamic Languages Event at CommunityOne RedMonk Unconference
Ok folks, this is the beginning. I'm going to try, in the midst of JavaOne Day Zero chaos, to run the first big *open* JVM dynamic languages get-together.
It will be part of RedMonk's Unconference running parallel to CommunityOne on Monday, May 7th. It is a *FREE* event. Yes, I said *FREE*.
We're going to have a number of Sun engineers there, including Tom Enebo and myself, John Rose from the JVM, Alex Buckley, and more. There will be representatives from JRuby (besides Tom and I), Jython, SISC, Rhino, Groovy, and others. It's going to be our big opportunity to make sure the JVM continues on a path of solid support for dynamic (and other) languages. And lest ye forget, these discussions will directly influence Java 7...so it's a unique opportunity we must not waste.
Why now? Because interest in dynamic languages for general-purpose application development--especially on top of mainstream virtual machines like the JVM--has simply exploded in the past year. At JavaOne, there's an entire track devoted to Tools and Languages, where the party line in the past has always been "Java Java Java". There are more language-related talks than anyone could attend, where before they were few and far between. And perhaps most importantly, Microsoft today showed they're doing the exact same thing we want to do, announcing their CLR-based Dynamic Language Runtime.
But we have a different palette with which to paint. The JVM is truly open source. All the language implementations are truly open source. We have vast communities in the Java world and strong, energetic communities around each dynamic language project. And all those communities have deep roots in the open source world. Our world is tailor-made for collaboration, and now is the time.
So, on Monday, May 7th (good lord, less than a week away!) I invite language implementers and alternative JVM language enthusiasts to join us. The time has not been decided yet, but I've got two other sessions at 4:00 and 5:00, so it will be before that. Perhaps during the early afternoon period? There's also the possibility of making arrangements for a second discussion that week, maybe co-opting a BOF or a BOF room.
For those of you unable to make it, please try to find a proxy that can represent your interests, or make those interests known to me over email. And know also that I'll be in San Francisco from the 3rd until the 15th, and in Portland, Oregon for RailsConf from then until the 21st. I'd love to get together and discuss language implementation and the future of the JVM. And I love a good beer.
Friends, dynamic languages have truly arrived on the JVM. Let's get together to ensure they feel welcome.
Watch this space for more details.
Wednesday, April 25, 2007
What Would I (Will I?) Change About Ruby
The latest Ruby Blogging Contest hits close to home: What changes would make Ruby into a better language without making it into something that isn't Ruby?
As you might guess, I've got some pretty strong thoughts here. I'm not a heavy Rails user, and I'm not as heavy a Ruby user as I'd like to be. But implementing a Ruby interpreter and now compiler has taught me a few things about what's right and what's wrong with Ruby. I'm not going to complain about performance, whine that the C code is too hard to follow, or even attack C-based extensions. Those may be important issues, but they're all fixable in the long term without breaking anything that works today (or by providing reasonable substitutes). I'm also not going to go into language design ideas...I have mine, you have yours, Matz has his. But my money's on Matz to do the "right thing" with regards to actual language design.
What I'm talking about are a few really important changes to the Ruby runtime, libraries, and ecosystem. Take these as my educated opinions...and don't think too hard about whether I'll be working to change these things in JRuby and in the wider Ruby world.
1. Threading
This more than any other area probably means the most visible changes to Ruby. Ruby currently is green-threaded, as most of you know. JRuby implements native threads mainly because Java uses native threads...we just piggyback off the excellent work of the JVM engineers. And the developing Ruby 1.9, the future successor to the current version 1.8 C implementation, provides something in the middle: native threads with a giant lock, so threads won't run concurrently.
So in general, Ruby is trending toward support for native threads. But there's a problem...some of Ruby's current APIs are impossible to do safely with native threads (and in general, impossible to really do safely with green threads...Ruby just does them anyway). Threading needs to be improved, with support for concurrent execution and removal of operations that prevent that.
Specifically, the following operations and features are inherently unsafe, and are not supported by any mature threaded system:
- Thread#kill
- Killing one thread from another may leave its locks and resources in an unpredictable state. JRuby currently implements this by setting a kill flag on the target thread and waiting for it to die--basically asking the thread to "please die yourself"--but it's not deterministic and the thread could fail to die.
- Thread#raise
- Forcing another thread to raise an exception can have the same effect as kill, since the thread may not expect to handle the given exception and may not be able to release locks or tidy up resources. JRuby handles this similar to kill, by setting a field to contain the exception a target thread should "please raise", but again it's not deterministic and there's no way to guarantee the target thread will raise.
- Thread#critical=
- There is no way to deterministically force true concurrent threads to stop and wait for the the current thread, not to mention the horrendous race conditions that can result when locks are involved. As a result of the many critical problems with critical=, it is already slated to be removed in Ruby 1.9/2.0.
2. ObjectSpace
ObjectSpace is Ruby's view into the garbage-collected heap. You can use it to iterate over all objects of a particular type, attach finalizers to any object, look an object up by its object ID, and so on. In Ruby, it's a pretty low-cost heap-walker, able to dig up objects matching particular criteria for you on a whim. It sounds like it might be pretty useful, but it's used by very few libraries...and most of those uses can be implemented in other (potentially more efficient) ways.
JRuby implements ObjectSpace by keeping a separate linked list in memory of weak references to created objects. This means that for every ObjectSpace-aware object that's created, a weakref is added to this list. When the object is collected, the weakref removes itself from the list. Walking all objects of a particular type just involves walking that list. Reconstituting an object ID into the object it references is supported by a separate weak list (again, more memory overhead).
There are no plans currently for ObjectSpace to be removed from Ruby in a future version. But there's a problem...in addition to being pure overhead in JRuby (which you can turn off completely by using the -O flag), ObjectSpace limits evolving development of the Ruby garbage collector, breaks heap and memory transparency, and poses yet more problems for threading.
There are many issues here. First off, the JRuby thing. By having to add ObjectSpace governors for all objects in the system, JRuby pays a very large penalty. We're forced to do this because the JVM (and most other advanced garbage-collecting VMs) does not allow you to traverse in-memory objects nor retrieve the object that is associated with a given ID. In general this is because the JVM does all sorts of wonderful and magical things with objects and memory behind the scenes, and the ability to ask for all objects of a given type or pull an object based on some ID number at any time cripples many of these tricks.
The threading issues are perhaps more important. Imagine if you will a true concurrent VM, with many threads creating objects, maybe one or more threads collecting garbage, and synchronizing all this to guarantee the integrity and efficiency of the heap and garbage collector. There is absolutely no room in this scenario for those multiple threads to request lists of specifically-typed objects at any time, nor to provide an ID and expect its object to be presented to you. These features break encapsulation across threads, they violate security restrictions from thread to thread, and they require whole new levels of locking to ensure that while reading from the heap no other thread produces new objects and no garbage collection occurs. As a result, ObjectSpace harms Ruby by limiting the flexibility of its garbage collecting and threading subsystems, and should be eliminated.
3. $SAFE and tainting
Safe levels are a fairly simple, straightforward way to set a "security level" that governs what operations are possible for a given thread. By setting the safe level to various values, you can limit modification of Object, prevent IO, disallow creation of new methods or classes, and so on. Added to this is the ability to "taint" or "untaint" objects. Tainted objects are considered "unsafe", and so certain security levels will cause errors to be thrown when those objects are passed to safe-only operations.
JRuby has safe level and tainting checks in place, but it's almost assured they're not working correctly. We have never tested them, largely because practically no tests (or perhaps literally no tests) use safe levels or tainting, and we've had *exactly one* bug report relating to safe levels, just a couple weeks ago. And to further kill the possibility of JRuby ever supporting safe levels and tainting correctly, my work tonight to fix some safe level issues revealed that doing so would add a tremendous amount of overhead to almost all critical operations like method creation, module/class mutation, and worst of all, object creation.
At this point, safe levels will probably remain in their current half-implemented state for 1.0, but I think it's almost decided for us that safe levels and tainting will simply not be supported in JRuby. In their place, we'll do two things (which I'd recommend the C implementation consider as well:
- Recommend that people who really want "safe" environments use an approach like whytheluckystiff's Sandbox, which takes a more JVM-like approach to safety: it runs code in a true sandboxed sub-runtime with only "safe" operations even defined. In other words, not only is it disallowed to load in files or hit the network, it's physically *impossible* to do so. What makes this even better is that Sandbox is already supported in JRuby (gem "javasand") and JRuby out of the box allows a fine granularity of operations to be disabled in new runtimes.
- Implement safe levels like Java handles security restrictions, which we get to leverage since they're already being checked and enforced at the JVM level. We will not be able to map everything...for obvious reasons, checking tainted strings all the time or limiting class and method creation are unlikely to ever happen, but we can limit those operations that the JVM allows us to limit, like loading remote code, opening sockets, accessing local files, and so on. So it's highly likely JRuby's implementation of safe levels will map to clearly-defined sets of Java security restrictions in the near future.
Ruby is a very free-form community. Matz is the most benevolent dictator I've had the pleasure to work with, and most of the community are true free-thinking artists. It's like the hippie commune of the language world. Peace out, man.
But there's a problem here. Ruby needs guidance beyond VM and language design or the loose meanderings of its more vocal community members. It boils down to a few simple points:
- Ruby needs a spec. Anyone who believes this isn't true isn't paying attention. Now I'm not talking about a gold-standard legal document signed in blood by Matz and the chief stakeholders of the Ruby community. An officially sponsored, widely supported, and massively publicized community spec would work fine--and probably fit the community and the language better. But something needs to done quickly, since Ruby's "bus number" is dangerously low. A spec is not something to be feared...it's a guarantee that Ruby will live on into the future, that alternative implementations (like JRuby) can't intentionally introduce nasty incompatibilities (or at least, that they'd be easy to discover and easy to document), and perhaps most importantly...that the full glory and beauty of Ruby is published forever for all to see and explore, rather than dangerously trapped in very few minds.
- Ruby needs a non-profit governing body. I'm not necessarily talking about a council of elders here, I'm just talking about some legal entity to which OSS copyrights can be assigned, donations can be made, and from which projects and initiatives can be funded. Maybe this would be RubyCentral, maybe this would be some other (new) organization...I don't know that. But it would be a great help to the community and Ruby's future if there were some official organization that could act as caretaker for Ruby's future. I'm all set to sign over any JRuby copyrights I have to such an organization, to protect the future of Ruby on the JVM just like the future of the C implementation. How about you?
- Ruby needs you. Granted, this isn't really a change as such. You probably wouldn't be reading this if Ruby didn't already have you. But the Ruby community is at a big point in its lifetime...at risk of losing its identity, being eclipsed by newer projects, or even slipping deep, deep into the trough of disillusionment. What will prevent that happening is the community showing its strong ties, coming together to support official organizations and official documents, and above all, continuing to pour all our hearts into creating newer and better applications and libraries in Ruby, pushing the boundaries of what people think is possible.
Monday, April 16, 2007
Paving the Road to JRuby 1.0: Performance
Since it looks like Antonio Cangiano is going to delay the next running of the bulls until Ubuntu 7.04 is released, I figured the next JRuby 1.0 update could be about performance.
Performance is such a tricky area for Ruby. Folks outside the Ruby community happily malign its performance, fueled by both FUD and by some truths. Folks within the Ruby community either aren't affected by Ruby's performance (it's "fast enough") or they simply don't care (it's slow, but I still love it too much to leave). A small part of the Ruby community takes what is in my opinion a rather anti-Ruby stance: "write it in C" as a targeted solution for identified bottlenecks. I suppose reality lies somewhere inbetween all these views, with Ruby's performance certainly not being stellar in the general case, but reasonable and sometimes surprisingly good for specific cases.
Ruby 1.9 has raised the promise of a new bytecode-based interpreter engine--a Ruby "virtual machine" by some reckonings--with the goal of improving performance foremost on the minds of Ruby's developers. And again the performance question is rather complicated. Antonio's recent shootout, running only Ruby 1.9's chosen benchmarks against all other implementations, shows it doing extremely well. It comes out many times faster than Ruby 1.8 in almost every test, and no other implementation even comes close. The truth however doesn't change very much; for non-synthetic benchmarks (like running Rails) the situation only improves by about 15% for some tests, and for many other tests performance actually degrades. Of course Ruby 1.9 is still under heavy development, and many more improvements are ahead, but the wide range of results demonstrates again that benchmarking must be taken with a grain of salt.
Et tu, JRuby?
So then there's us and JRuby. JRuby's professed goal has never been to be a better Ruby; at best, we're trying to build the best Ruby possible on top of what we believe is the best VM in existence. And toward that end we've put most of our time into compatibility and correctness above all else, aiming for the goal of complete Ruby language compatibility and near-complete builtin-class compatibility, hoping for JRuby to someday be treated as "just another Ruby implementation" on which people can run their apps and design their clever frameworks and libraries. But in the past year, it's become apparent that we could actually exceed Ruby's performance for specific cases in the near term, and for general cases over time. So the target market for JRuby seems to be changing from "Ruby users that must use Java VMs, libraries, and servers" to "Ruby users that want a better-performing, more scalable implementation". And our noble quest for near-complete compatibility gets muddled with all these fiddly performance details.
But that's life, right? The goals you set out for yourself and your projects rarely align perfectly with the goals others set out for you. The trick is achieving a balance between what you want to do with your life and what others (like your community members or your employers) want you to do. Perhaps the successful developer is the one who can derive pleasure from both tasks.
On the road to JRuby 1.0, we've done our best to balance compatibility and performance. We are now nearing the end of the compatibility road, with Ruby language features nearly 100% and builtin classes almost as complete as we can make them on the JVM. The real reason for a JRuby 1.0 now is that we believe we're finally approaching "Ruby compatibility" for some high measure of compatibility, such that the vast majority of platform-agnostic Ruby code should run successfully on JRuby. And that's certainly no small feat, given that just a year ago we celebrated a mostly-broken cobbled-together Rails 1.1 app slowly handling CRUD operations. Today, people are deploying JRuby on Rails apps in production, and the game has only gotten more interesting.
So then, performance. You're all wondering what the answer is to this performance thing, aren't you? Is JRuby going to blow away all competition, including the nascent Ruby 1.9 and mid-term projects like XRuby, Rubinius, and Ruby.NET? It's certainly possible, but it's not our goal. Is JRuby going to be faster than Ruby 1.8 when 1.0 is released? For specific cases, I'd say yes...there's plenty of areas we already perform better than Ruby 1.8. For the general cases, it's hard to say. We perform well serving up Rails requests today, but only about 50-70% of Ruby 1.8's performance. And though we know where most of the bottlenecks lie, we're a little resource limited trying to fix them. Do we believe that we'll be faster than Ruby 1.8 in all general cases in the near future? Yes, we strongly believe that will happen.
Now of course I could ramble on and on about performance and put you all to sleep, but actual numbers will probably keep your interest better than my droning.
The Test
Like the shootout, I'm just running the Ruby 1.9 benchmarks here. We have not done any targeted optimization for these tests; there's no Fixnum magic or anything like that under the covers. What we have done is implement multiple general-purpose optimizations to speed method and block invocation, object creation, and interpretation. We've also spent a little more time getting these tests to compile successfully, but of course any work done on the compiler is generally applicable as well.
These results are all based on JRuby trunk code, revision 3480. I'm running Java 6 on a MacBook Pro 2.16GHz Core Duo, and all code was compiled to target Java 6.
For the first set of results the JRuby command executed basically amounts to the following:
JAVA_OPTS=-Xverify:none jruby SERVER -O [script.rb]
- JAVA_OPTS=-Xverify:none specifies not to verify classes on startup; this is a large part of the speed hit Java applications have when starting. We turn it off here to remove a little of that overhead from the benchmarks, since most of them are very short runs to begin with.
- SERVER specifies that JRuby should use the "server" VM, which takes a bit longer to optimize a bit more heavily when JITting Java code into native instructions. JRuby generally performs best under the server VM, though using it impacts startup time.
- -O disables ObjectSpace in JRuby. This may seem like cheating, but the truth is that when ObjectSpace is enabled we pay double or triple the object creation cost in JRuby since we have to track all objects separately. Ruby's ObjectSpace is essentially zero-cost...it's just a window into the memory manager. Since we don't have a low or zero-cost way to implement ObjectSpace, it falls into what I categorize as "optional incompatibility". If you don't need it, turn it off and JRuby performance will improve. You can call it cheating if you like...the truth is that practically no code actually depends on ObjectSpace.
TEST MRI JRubySome of these are rather surprising results. This is JRuby running in plain old interpreted mode, with no compilation involved. The majority of the tests still have JRuby slower than Ruby 1.8, but the gap has narrowed an incredible amount since last year. In only a few tests are we more than twice as slow as MRI, and on a couple we're almost twice as fast. If you'll imagine startup time removed from these numbers, and believe me when I say Java takes far more than 30-60 seconds to rev up to full speed, then the situation looks even better. What's more, we've got a good several weeks before the first 1.0ish release is scheduled (something betaish or RCish that proudly proclaims it's "done") and a bunch of great committers and community members eyeing performance metrics.
--------------------------------------
app_answer 0.584 2.239
app_factorial ERROR 4.248
app_fib 7.126 10.549
app_mandelbrot 2.346 10.300
app_raise 2.587 4.441
app_strconcat 1.829 2.141
app_tak 9.711 13.345
app_tarai 7.529 11.050
loop_times 5.475 6.903
loop_whileloop 9.982 11.797
loop_whileloop2 2.009 3.292
so_ackermann 13.726 26.132
so_array 7.257 8.801
so_concatenate 2.063 3.546
so_count_words 0.507 4.491
so_exception 4.342 8.346
so_lists 1.238 2.744
so_matrix 2.258 4.241
so_nested_loop 5.609 7.898
so_object 7.050 6.496
so_random 2.139 4.643
so_sieve 0.740 2.240
vm1_block 23.604 27.405
vm1_const 16.774 20.650
vm1_ensure 16.546 15.800
vm1_length 21.210 21.899
vm1_rescue 13.170 16.197
vm1_simplereturn 21.091 31.376
vm1_swap 25.114 17.949
vm2_array 6.049 5.690
vm2_method 13.528 17.759
vm2_poly_method 16.886 24.956
vm2_poly_method_ov 4.576 6.972
vm2_proc 7.060 7.797
vm2_regexp 4.421 9.353
vm2_send 4.332 8.198
vm2_super 4.992 7.944
vm2_unif1 3.838 6.095
vm2_zsuper 5.409 8.452
vm3_thread_create_join 0.019 1.592
Ok, you may be asking "what about JRuby's compiler?" Yes, there is a compiler in the works. It's primarily been my job to build out the compiler, though Ola has jumped in a few times to offer his excellent help. And progress has been slow but steady. You have to remember that in all the world, there's no known 100% complete Ruby compiler for a general-purpose VM. There's Ruby 1.9, but its bytecodes have been custom designed around Ruby. There's XRuby and Ruby.NET, but it's still unclear how complete they really are. So this is an open area of research and development. But the results are looking great so far.
For this test, both the ahead-of-time (AOT) compiler and the just-in-time (JIT) compiler modes are activated. This forces the target script to be compiled before execution and also compiles any methods hit heavily once execution gets going. The command amounts to the following:
JAVA_OPTS="-Djruby.jit.enabled=true -Xverify:none" /
jruby SERVER -O -C [script.rb]
- -Djruby.jit.enabled=true enables the JIT compiler. The default threshold at which a (compilable) method gets compiled is 50 invocations.
- -C tells JRuby to compile the target script before executing it. If the script can't be compiled, JRuby bombs out with an error.
TEST MRI JRubyAhh, now things look a bit different! In almost every case, JRuby performs better than Ruby 1.8. In the long running cases, the difference is even more obvious. Short runs still put Ruby 1.8 ahead, but I'm totally ok with that. Java, and by extension JRuby, has never had stellar short-run and startup performance. But we don't really have to if the heavy, long-running apps people actually use end up running faster.
--------------------------------------
app_factorial 0.029 3.459
app_fib 7.094 5.093
app_mandelbrot 2.340 9.011
app_strconcat 1.827 2.391
app_tak 9.714 5.394
app_tarai 7.515 4.642
loop_times 5.428 2.942
loop_whileloop 10.016 6.027
loop_whileloop2 2.012 2.191
so_ackermann 13.610 11.254
so_concatenate 2.043 2.396
so_lists 1.250 2.141
so_matrix 2.256 2.394
so_nested_loop 5.614 4.167
so_random 2.158 3.291
so_sieve 0.741 1.798
vm1_block 23.392 12.397
vm1_const 16.980 8.908
vm1_length 21.094 10.901
vm1_simplereturn 21.252 9.345
vm2_array 6.025 3.041
vm2_method 13.049 7.794
vm2_regexp 4.468 7.647
vm2_unif1 3.855 3.293
vm3_thread_create_join 0.017 1.389
Note also that this is the first real compiler we've had; it's not doing any optimization like Ruby 1.9's and Rhino's compilers, and it's almost certainly far from being efficient. I'm no compiler expert, and this is my first real attempt. These numbers already looking so good demonstrates that there's a grand adventure ahead of us: Ruby really can be made to perform well on the JVM. It just requires a little confidence and a little more effort.
That pretty much wraps up this installment. The bottom line: As we approach JRuby 1.0, our performance is better than it's ever been--faster than Ruby 1.8 for many specific cases, with stellar across-the-board performance right around the corner. And with other implementations like Ruby 1.9, XRuby, Rubinius, and Ruby.NET rapidly coming of age, Ruby's future is looking extremely solid.
script type="text/ruby"
Remember way back when the IRB applet was first thrust upon the world, and I promised that you truly could now use Ruby in the browser? At the time, it was still a little cumbersome, and generally you could only use Ruby by passing it directly into the applet.
Well Dion Almaer has taken it to the next awesome step:
<script type="text/ruby">
Check out his blog entry on how to enable true Ruby scripting on the client side, as well as the link to his demo page (still loads up the applet in the background, but that's a minor implementation detail).
Awesome.
Friday, April 06, 2007
Paving the Road to JRuby 1.0: Unicode
Well friends, the countdown has begun. Within the next several weeks, we're looking at getting a 1.0 release out in either beta or RC form. Of course most of you OSS folks know how arbitrary release numbers are...the day after we branch 1.0 we're going to start committing to 1.1, and the cycle will continue. But it seems like the last year is coming to a very solid milestone, and these stabilization points are necessary once in a while. So it comes to pass.
As part of "closing the door" on 1.0, we're weighing features and bugs and wishes to decide what makes the cut and what must wait for a future release. To help get the word out, I'll be doing a series of entries on where we are, where we'll be for 1.0, and where we're going in the future. First off, one of the biggest issues for Ruby--Unicode support--and what it will look like in JRuby 1.0.
Unicode
Straddling two worlds is complicated. Ruby is dynamically typed, with open classes, redefinable methods, eval...a very different sort of world than Java. If those really obvious items weren't tricky enough, Ruby has another wrench to throw into the works: it cares not for Unicode.
Of all issues, this is probably the most problematic to solve. Java, as you know, is always unicode, representing strings internally as UTF-16 and supporting most external encodings you could ever possibly want. Ruby, on the other hand, pays little attention to encodings most of the time, preferring to just treat strings as an array of bytes. If it's encoded, so be it. If not, so be it. The majority of Ruby's string operations work with bytes (though there are a few smart exceptions), so multibyte-encoded strings can be damaged if you have high expectations.
We've spent the past year weighing options. We want JRuby to be compatible with the C implementation, since it's widely deployed and apps depend on its sometimes quirky string behavior. We also want JRuby to integrate well with Java, and the boundary to be as seamless as possible. How does one accomplish that? We've gone through a couple options:
- At first, JRuby's string was represented with a String/StringBuffer internally. This allowed the original JRuby authors the most trivial path to a fully-working string implementation; almost all the heavy lifting was done for them, and for simple cases it worked fine. But the APIs did not conform to what Ruby applications expected, frequently returning 16 bit values for individual characters and reporting incorrect byte lengths for strings that couldn't encode into all 8-bit characters. It was broken, as far as Ruby code was concerned.
- The second option, which is what we currently have, is to do what Ruby does...treat all strings as byte[], and implement all operations to the letter of the law. This allows us to make even the lowest-level APIs conform exactly, and to be honest that's been necessary to get applications like Rails up and running. Ruby blurs the distinction between a byte[] and a string so completely that Rails actually now ships with its own multibyte helper, a library that stands a good chance of becoming widely deployed. So to support Ruby and all its many wonderful apps, we fell in line.
When crossing the boundary, Ruby will be assumed to use UTF-8 strings.
There are many benefits to this even beyond the fact that it took me 15 minutes to make it work. It allows Ruby strings to behave like Ruby apps expect them to. It allows Java strings to pass into and out of Ruby code without corruption. And it allows Ruby strings to pass into Java code with the only prerequisite (if you want to avoid garbled strings) being that they should be decodable UTF-8. That last item seems like a fairly reasonable requirement given that you're calling 100% unicode Java APIs, and in general the people that will care about this conversion are the ones already using unicode.
Of course, nobody calling these APIs would ever see this or have to do anything special...strings will pass back and forth and encode/decode as necessary, and things will generally just work. And there are performance tricks we can use to speed things up if that becomes a real issue. But it also does something no other proposed solution can: it solves the problem pretty darn well...in time for 1.0.
So for the record, this is the proposed solution we'll go with in the 1.0 timeframe:
- Ruby strings are byte[] and conform to Ruby string semantics
- Java strings passing into Ruby code will be encoded as UTF-8, with the implication that you should expect to be working with UTF-8 byte[] in the receiving code
- Ruby strings passing out of Ruby into Java libraries will be assumed to be UTF-8, and the resulting string on the Java side of the call will reflect that assumption.
- At some point, either before 1.0 or immediately after, there will be an installable gem that provides Java-native support for Rails' MultiByte library, which should provide a substantial performance boost there. As it is, the current pure Ruby version works almost 100%, so the extra performance should ultimately just be a bonus.
- We will start implementing a Ruby 2.0-compatible string, but it's unclear when would be a good time to flip that switch. I would predict that JRuby users will have a choice of which to use fairly soon, and we'll swap out behavior depending on what you want.
- It's also likely that we'll present clean ways to "just use Java strings" for developers that want to do that. This would not result in code that's compatible with stock Ruby, since we'd obviously have to "unicodify" many Ruby methods, but it fits into a larger goal of Ruby-enabling the Java platform as well as possible
- And of course, some combination of these three and the current solution is possible, given some time and thought. Ultimately it will be driven by you, the users of JRuby, and what you want to see done with it. Because that's what we here at Sun are trying to be all about...what users and developers want to be doing.
Saturday, March 31, 2007
ActiveRecord 100%, Performance Doubling, Java Support Improving
I haven't blogged a solid JRuby update recently...and this one isn't going to be as solid as I'd like, but it should let you all know that the wheels are still moving and the train is picking up speed. It's astounding how much progress is being made.
ActiveRecord Now Fully Supported
It's good to have such incredible community members and great core team members.
Ola Bini, who recently accepted an offer from ThoughtWorks, has been hitting Rails unit tests hard. And the module with the most failures has always been ActiveRecord. It's the most complicated and the weakest point in the JRuby on Rails story.
Until now.
1031 tests, 3839 assertions, 0 failures, 0 errorsThat headline should read "ActiveRecord Fully Supported on JRuby", for you press affiliates out there.
Of course there's a few caveats, but they're pretty minor in this case. That's zero failures and zero errors when running against MySQL, but it's certainly a de-facto standard in the Rails world. The next step is to get as many other databases to 0/0 as possible. And those are much smaller steps than this first one.
So what does this mean for JRuby on Rails users? Well in general, it means that the heart of your DB-driven JRoR apps--ActiveRecord--is now the best-supported piece of the puzzle. It also means that overall Rails support is well into the 99% range and still climbing.
Performance Continuing to Improve
In addition to the focus on getting Rails working well (which is really a focus on improving overall Ruby compatibility) we've been hitting performance much harder the past couple weeks. I especially have been splitting my time between interpreter and compiler work. As a result of all our efforts, the current trunk code is almost another 2X the performance of 0.9.8, released less than a month ago. The compiler also supports almost 2X the number of syntactic constructs it did in 0.9.8, and it's getting faster and more stable every day.
There are actually a number of benchmarks running faster than MRI (the C version of Ruby) even when interpreted, showing that we're finally making good on our promise to have interpreted JRuby run as fast as MRI. So it truly does seem that the compiler will be "bonus" performance once we work out the remaining bottlenecks in the system.
Java Support Looking and Feeling Better
There have been two areas of focus for Java support the past few months: rounding out features and syntax and improving performance. Both had some great jumps in JRuby 0.9.8, with a couple small fixes improving performance many times and with concrete/abstract class extension finally making it into a release.
Since 0.9.8 we've continued to make progress on both fronts. I've made a few additional small changes to improve overall performance, and Ola's been moving some especially critical pieces of code into Java, since they represented a lot of back-and-forth across that barrier. Where calling into Java code was previously as bad as twenty times slower than calling Ruby code, it's now a modest two times slower. We expect to improve that further, getting as close to Ruby invocation speed as possible.
We've also just added what should hopefully be the final revision to our various "import" syntaxes. Oddly enough, we've come full circle to "import":
# create a new constant "System" pointing at itAnd we're considering some improvements to how multiple classes are imported and how to specify an alternate constant name (for classes like String, which would be a bad idea to overwrite).
import java.lang.System
# use Java::syntax for less-common packages
import Java::my.weird.package.MyClass
You can expect to see this all solidifying as we run these last miles toward JRuby 1.0.
--
I will blog more on all this as we get into April, but it's truly shaping up to be Springtime for JRuby in JavaLand.
Friday, March 30, 2007
Can Your Ruby Do This?
# scripting.rb
include Scriptable
x = 'Hello'
y = 'world'
groovy <<G
def z = 'multilanguage'
println("${x}, ${z} ${y}")
G
javascript <<JS
z = 'multilanguage'
println(x + ', ' + z + ' ' + y)
JS
# both => "Hello, multilanguage world!"
Saturday, March 10, 2007
Ruby on Grails? Why the hell not?
I've spent some time this weekend looking over the Grails codebase. After a truly gigantic checkout (Grails bundles production JARs for all the libraries it integrates) and after swimming through a substantial, single-project sea of source files, I think I'm starting to follow it (though a tour by someone familiar with Grails internals would be nice). But there's something else I've learned in the process...
There's really no reason this couldn't use Ruby just as easily as Groovy.
Let's look at some really dumb numbers. Grails, as it turns out, is actually over 2/3 Java code, unlike Rails which is 100% Ruby. Grails contains around 436 Java source files for about 54kloc of code and 273 Groovy source files for about 23kloc of code. So Java outnumbers Groovy by a 2:1 ratio. And yes, Java's probably more verbose than Groovy in most cases. But my point stands, and I'll back it up with more...
Then there's the libraries. The Grails binary release weighs in at a whopping 26MB compressed and 49MB uncompressed (*wow*). But 31MB of that is third-party libraries completely independent of Groovy, things like Hibernate, XFire, Quartz, Apache Commons, and so on. Almost all of which (or perhaps all of which) are pure Java code. So the ratio is even more heavily toward Java.
These numbers and my exploration of the code base lead me to a surprising conclusion: Grails is not a "Groovy on Rails" by any means. It's a large number of popular, well-established Java libraries stitched together mostly by Java code with a thin layer at the front (20kloc is not much) of Groovy code for end-users to see and work with. And I guarantee you it's not the Groovy part that's most interesting here. Hell, you could write that part in Java if you followed enough "convention over configuration", and you certainly could write it in Ruby.
I decided to do this exploration after realizing there's nothing about Ruby or JRuby that couldn't be used to wire a bunch of Java libraries together--indeed, that's one of the biggest selling points of JRuby, the fact that you can write plain old Ruby code but call Java libraries almost seamlessly (with day-by-day improving integration points and performance numbers). So why not take Grails and do a port to Ruby? Groovy code isn't far off from Ruby in many ways, and everything you can do in Groovy you can do in Ruby (plus more), so the port should be pretty straightforward, right?
Then I find out that Grails is mostly Java code. And instead of being annoyed or repulsed by this revelation, I was intrigued. Why not make Grails work with Ruby as well? And really, why not make it work with any scripting language? Why limit that nicely-integrated back end by forcing everyone to use Groovy, regardless of preference? Have we learned nothing from a decade of "100% Pure Java"?
So I propose the following.
- I believe it would be possible to completely isolate Grails' tastier bits from Groovy. I don't claim it would be easy, but Grails is fairly well interfacified, injectificated, and abstractilicious, so making that dividing line clear and bold is certainly possible (and heck, that's the point of dependency injection and component frameworks anyway, isn't it?). Really, I could start implementing a few of those interfaces right now with Ruby code, but that's not the right approach.
- Grails' value is not in Groovy, but in the clean integration of established libraries behind the "Convention Over Configuration" mantra. There's nothing specifically Groovy (or Ruby) about that religion, and there's nothing Groovy does in Grails that another JVM language couldn't do just as easily. This is *truth*. And Grails would benefit tremendously by expanding to other languages. Being tied to Groovy alone will hold it back (just as a framework tied to any one language will be held back, Ruby included).
- I also believe that if the Grails project doesn't make an official effort in this direction, others are likely to take up that mantle themselves. Though the debates will forever rage about which JVM dynlang should carry us into the future, the same facts that have guided programming for half a century still apply today: No one language will be enough to carry us forward; no languages will survive indefinitely; and the language you'll use in ten years you've probably not even heard of yet. Did anyone expect Java to be the most widely-used, widely-deployed language ten years ago? Well today, it is. It won't always be.
- There's also a very interesting angle here for language enthusiasts. Nobody doubts that the Java platform is officially becoming multi-lingual, possibly a complete reversal from years past. But there are going to be growing pains here, and we're already feeling them on some projects at Sun. We have this wonderful platform with all these wonderful languages...and they all do things differently. We have a first stab at official "Scripting" support in JSR-223, but it only defines the integration point from Java to the language in question...only partially defining the integration points for those languages calling back to Java (a tricky problem) and defining *nothing* related to making those languages interoperate with each other. Again, Java becomes the integration point, and the bottleneck (as in "how do you pass a dynamic-typed object from one language to another by sending it through a statically typed language?"). The very exercise of making Grails (or any other framework) work equally well across multiple user-facing language interfaces will require better interoperability and more clearly-defined integration points for dynamic and static languages on the JVM.
- The Phobos guys at Sun are already tackling many of these same problems while expanding their JavaScript-mostly Rails-inspired web framework to other JSR-223 languages (notably, Ruby). So anyone taking on the Grails effort would certainly not be alone, and there are some damn smart engineers at Sun interested in tackling these very same problems.
--
So, enough of the inflammatory bull-baiting. I dare you to tell me why I'm wrong here. Even better, I dare you to look into it yourself. 75kloc of code, only a third of it Groovy. Established libraries you're sure to have used at some point. Clean integration, following the principals inspired and proven by Rails.
But fronted with your language of choice.
Make it so.
Tuesday, March 06, 2007
NetBeans 6 Ruby Support Even Better Than Expected
See, here's the deal. I'm not a very advanced IDE user. I used Eclipse for years and JDE before that, and I wouldn't ever have considered myself a "pro" with either. I learned what I needed to get the job done and not a whole lot else. Beyond that I never knew some of the coolest features, and was never so good at convincing people to make a switch. This actually has helped my migration to NetBeans, since it basically does all the things I was used to in Eclipse. But it has been problematic when I try to convey how cool the new NetBeans Ruby support really is.
It's super cool. It does things I've not seen any other IDE or editor do.
However, it's best not to take my advice, and Roumen Strobl has recorded two seriously excellent demonstrations of Rails and Ruby support in NetBeans. They make the sale way better than I could (though I now have many good ideas for how to sell it better in the future).
If you have any interest in Ruby editors or IDEs, you should take the time and watch these, especially the Ruby support.
Two Demos: JRuby on Rails and Advanced Ruby Editing in NetBeans!
Behind The Scenes: JRuby 0.9.8 Released!
Things have been moving quickly in JRuby-land, and we've just kicked out a major release in JRuby 0.9.8. Among the big features:
- Ruby classes can extend concrete/abstract Java classes and override methods
- New Java primitive array syntax
- Reimplementation of String, Numeric classes, and Array to be more correct and performant
- Significant bottlenecks have been identified. In some cases IO is 6.5x faster than previous releases. Java included classes are significantly faster than in the past.
- 220 Jira issues resolved since last release
- Ruby on Rails support
It was our original plan over two months ago that by Februrary's end we'd be passing enough of Rails' own test cases to call it "supported". And in a relative rarity for an open source project, we hit that milestone. It was actually early last week we started saying we officially support Rails, which at the time meant better than 95% of Rails test cases passing. By the time we cleaned up the remaining release items, we had passed 98%. And we're still climbing.
So what does this mean for a practical Rails user? Well, it means that the dream of deploying Rails apps on any Java-based server in any Java-based organization is another big step closer to coming true. I've actually been demoing Rails apps running in a WAR file on GlassFish for all my recent talks, and it's pretty solid. I don't imagine it's perfect, of course, but it's looking really very good. It will be "ready" by the time we hit 1.0.
A few folks have been wondering about the jump from JRuby 0.9.2 to JRuby 0.9.8. The simple answer is this: we're two releases away from 1.0. We wanted to make it obvious how much work has gone into 0.9.8 (as many SVN revisions in this release as the last three combined) and show our commitment to getting a 1.0 release out very soon. The 0.9.8 release will be followed by 0.9.9, which will be followed by...well, you get the picture. We're in the home stretch now.
The other core team members have reported on the release, so I won't cover all that. Here's a list of links:
Tom Enebo
Ola Bini
Nick Sieger
In related news, there's also a new release of ActiveRecord-JDBC, the gem-installable module enabling JDBC databases to be used by ActiveRecord. This is also very welcome, since there have been a number of bug fixes in trunk that weren't easily installable the past couple months. Now you should be able to run both Rails 1.2.2 and 1.1.6 well with JRuby, just using released code. It helps complete the puzzle.
So what's next for JRuby? Well, with the 0.9.8 release we've reached a really big milestone as far as Ruby compatibility, so we're likely to turn our attentions toward areas not directly related to Ruby 1.8. Specifically:
- Work will continue on the bytecode compiler, and I'm hoping to have the JIT permanently enabled within the next week
- Java integration will get a heavier focus, to start bringing us more in line with languages like Groovy for ease-of-integration. In general things are looking pretty solid right now, but we know there's a lot of work to do here. I'd like for us to come as close as possible to Groovy's level of tight integration as possible.
- We will also start working on native Unicode support, by exposing a Chars class like the one provided in Rails' MultiByte library. This will allow us to maintain compatibility with Ruby and provide solid Unicode support without introducing an incompatible set of features.
- Both Tom and Ola have expressed an interest in continuing work on our Ruby 2.0 bytecode engine. Ola would like to continue improving and expanding the bytecode support, while Tom would like to play with a second parser that goes straight to bytecode. Both of those will help feed into a potentially faster interpreted mode in the future, giving us many of the perf gains shown in Ruby 1.9 benchmarks.
- JRuby interpreted mode is roughly 3-4x as slow as Ruby 1.8
- JRuby compiled mode is roughly 1.5-2x faster than Ruby 1.8
- KRI (Ruby 1.9, "Koichi's Ruby Implementation") is many times faster than MRI for the posted benchmarks
- KRI is 2x or less faster than MRI for the general case
As always, we welcome contributors, bug reports, patches, and user anecdotes for JRuby. We love to hear about bugs, we want to know your opinion, and we respect and honor our community whenever we get the chance. Remember, JRuby is your project, and can only be successful with your help.
Visit www.jruby.org to learn more about the project, download the release, and get involved!
On to the next release!
Thursday, March 01, 2007
JRuby World Tour: Bangalore
After my adventures in Hyderabad, the next stop on the tour took me to the heart of India's IT industry, where I would present my TechDays talk again for Sun Microsystems' Indian Engineering Center. But I scheduled in a little free time as well, to do some exploring.
I arrived on Saturday the 24th after a short flight on Sahara Air. For once, my travel arrangements worked out, and I was able to buy my ticket upon arriving at the Hyderabad airport. A few hours later, I had landed in Bangalore. A hotel car was waiting for me (this is now only the second time I've had a car waiting with my name on a sign, and I think I like it), so I headed straightaway to the Richmond Hotel in central Bangalore.
The Richmond has a fairly old-world feel, apparently having been converted from an apartment or office building some time in the past. It had a musty smell, but was otherwise pretty clean and well-kept. I'm generally more comfortable in "clean and comfortable" hotels than in "pristine" hotels like the Novotel, so the Richmond was just fine. That night I made arrangements for the next day's tour, ordered in some room service, and caught up on sleep.
It still amazes me how everything has changed since blogging became mainstream. Even as short as five years ago, I had a devil of a time finding local tour guides for my travels. Generally the best bet was to fork over too much money to a paid guide or to travel with a tour group. Neither of those are really conducive to free-form exploration, so I've always been forced to strike out on my own. But those days are certainly behind us.
In response to my "World Tour" blog posts, I had no less than 5 different people offer to show me around Bangalore. They ranged from JRuby enthusiasts to casual readers, and for once I had more guide offers than I could use. So I settled on the first two to contact me: K "Venkat" Venkatsubramaniyan and Håkan Raberg.
Venkat was up first on Sunday the 25th. He had actually stopped into the JRuby IRC channel (#jruby on FreeNode) as well as emailing me. His plan for the day included a trip to a cultural/folk center outside the city, lunch and dinner at a couple places to be determined, and shopping stops for a few things I wanted to purchase. So we started out bright and early at 9AM.
We made the following stops, in order:
- Rajarajeshwari Temple in western Bangalore, where I paid my respects and was gifted with a wreath of flowers and a tilak on my forehead. It was a moving experience, even though I felt a bit out of place in my western clothes. And the classical architecture of this modern shrine was quite impressive.
- Janapada Loka (Folk-Cultural World), a collection of folk art museums and exihibits on the road from Bangalore to Mysore.
- Kamath Yathrinivas Restaurant, where I had my first traditional Indian meal, complete with banana-leaf plate and eating with my hands. It was very tasty.
- Nalli Silk Arcade, to purchase sarees for my wife. I think I was the only guy in there shopping, other than a few poor saps who were obviously dragged along by their wives.
- Cauvery Crafts Emporium on Mahatma Ghandi Road, a nice collection of handicrafts, clothes, and nick-nacks, but very much tailored to western tourists. I looked at pearls for my wife briefly here, but decided not to buy.
- Navaratna Jeweleries, up the road from Cauvery, where I found the pearls I wanted. I went back on Tuesday before flying home to have the string extended and a clasp put on.
- Garuda Mall and Bangalore Central Mall...we just kinda ended up here looking for a place to buy tea and wasting time before our evening's restaurant opened.
- Amaravathi Restaurant, another more traditional place (more banana leaves and eating with hands), but this time with some delicious spicy curries and a crab dish as tasty as it was difficult to eat.
Monday the 26th was my day on the spot at Sun Microsystems. Conveniently, Sun's office in Bangalore was only about two blocks from the Richmond, so I headed over at around 10AM to prepare for my 11AM talk.
For some reason, every Sun office I visit is a terrific maze of doors, offices, and cubicals. The Menlo Park office has all L-shaped buildings with multiple intersecting hallways. Santa Clara's offices are plus-shaped (I think) which is often even more confusing. The Bangalore office put them to shame. It is hosted on the 4th, 5th, 6th, and 7th floors of a circular building with a central unsecured walkway and multiple secure entries and exits as you progress around the outer ring of offices. What complicated things most was that hallways extended out from the center of the ring like spokes of a wheel, but the perpendicular hallways did not always curve to match the circle. This meant that hallways often met at bizarre angles, and it was impossible to maintain any sense of direction. I got lost more than once wandering around, and unlike the California offices, I had no guarantee that following a given hallway would eventually lead me to an outside wall. Bizarre.
Anyway, I did the same talk from TechDays for Sun folks from 11AM to 12PM. Unfortunately, I had lost favor with the demo gods. My NetBeans demonstration froze up and refused to come back. My Rails demo failed with an error I still haven't isolated. And because of a peculiar scheduling snafu, the last 15 minutes of my talk had to be condensed into five minutes, resulting in more errors and a very flustered presentation. What had gone perfectly a few days earlier at TechDays went very poorly at Sun. Luckily, it seems to have still been well-received, and the classic JIRB demo was enough to wow the crowd.
The rest of the day was spent meeting with various folks at the IEC, talking about their projects and how JRuby could help. There's some cool stuff coming out of Sun India, so keep your eyes and ears open over the next few months.
Monday night my new friend Hakan Raberg from ThoughtWorks arranged to meet with me and two other ThoughtWorkers for dinner. It turns out Hakan is an old friend of Anders Bengsston, one of the premier JRuby contributors in the old days before I joined the project. Such a small world.
As a nice change of pace, the ThoughtWorkers and I had dinner at Barbeque Nation, a trendy, modern-feeling restaurant featuring fairly traditional northern Indian cuisine. In other words, lots of meat. Barbeque Nation is one of those "cook it yourself" places, where the center of the table is a small charcoal stove and the meats are delivered mostly-cooked on metal skewers. After marinating with a choice of marinades and allowing the food to cook the last 10% of the way, a delicious entree results. To make it even more acceptable to a western eater: it's all-you-can-eat, with skewers continually delivered until you tell them to stop. For the first time in a week, I was stuffed. It's very hard to turn down delicious seasoned and grilled meats when they keep arriving in front of you.
The ThoughtWorkers also had a number of interesting projects going on, many related to Ruby and a couple likely to use JRuby in the near future. Hakan had even worked on a code-generated implementation of the JRuby selector table's switch statements, which he promised to tidy up and send me. It could be very useful for us to optimize user-created classes in the same way we're starting to optimize core classes, and it was actually a technique we considered doing ourselves. More on that as it develops.
Tuesday, I packed up and checked out of the hotel. I was a little sad to be leaving Bangalore, but since I still had an entire day before my flight I walked up to M.G. Road to take care of some last minute souvenir shopping. My first stop was at Navaratna Jewelers for the modifications to my wife's new pearl necklace I mentioned earlier. While I waited for the work to be completed, I visited a few shops off the main boulevard and purchased the following items to bring home:
- A small wooden Ganesha (about 4" tall), to bring success and prosperity to my office.
- A small wooden Emblem of India, atop a 4" wooden column with a felt base. It is a perfect reminder of my Indian adventure (and probably the coolest national emblem I know). This is my favorite souvenir by far.
- A small wooden puzzle box for my son.
- Two larger versions of the puzzle-box with hand-painted images of Indian women on the tops.
- Two silk ties with different images of dancing elephants on them.
- Two painted clay hippopotamuses, as gifts for whoever.
- A comic/cartoon book of stories of "Heroes of the Mahabharata", for my boy to read.
Comfortably full from lunch and loaded down with all my new possessions, I returned to the hotel to catch a cab to the airport. Goodbye, Bangalore...or so I thought.
Bangalore Airport was to be a baffling ordeal. People were lined up for a solid block to get into the terminal, mostly encumbered with a half-dozen bags each. To confuse even more, there was an option whereby non-flying visitors could pay for admission to the terminal. Luckily, I figured out I didn't have to pay, and after muscling my way to the entrance, I moved on to the second challenge in this travel gauntlet: Thai Airways had no reservation for me.
My original tickets to India were incorrectly issued to "Charles Onutter", an unfortunate combination of my middle initial and last name. Normally, this wouldn't have been a big deal, but apparently international flights are extremely strict about the name for which tickets are issued. If it doesn't match your passport exactly, you're out of luck and can't check in. In a flurry of activity the day before I left, I managed to arrange new tickets. But because it was such short notice, I would have to buy two of my tickets at the originating airports: Sahara Air from Hyderabad to Bangalore and Thai Airways from Bangalore to Bangkok. Sahara air went fine...either because they found my reservation or because they had plenty of room on the flight. But Thai Airways had cancelled my reservation because they overbooked the flight.
Luckily, after talking to the ticket agents and American Express's emergency travel service, I was able to purchase an unused seat for the flight. I would fly first-class to Bangkok. I rushed through security, ran to the gate, and finally settled into my seat. Onward to the next stop in the tour: Bangkok, Thailand!
Friday, February 23, 2007
The Great Hyderabad Adventure
I have many pictures to share, but they will have to wait for an updated post. Allow your imagination to escort you for now.
So! I had been trapped in the "green zone" at the Novotel Hyderabad for two solid days, and I think I just finally had enough of it. The Novotel is surrounded by about five acres of land, and the edge of that property terminates with a heavy fence (or wall in places) with either barbed wire or razor wire keeping the rabble out...or perhaps keeping the westerners in? I had taken a few strolls on the grounds, and after walking a few hundred meters from one end to the other this morning, I realized I was tired of living in the bubble. It was time to leave.
A number of folks had expressed interest in escorting me into and around Hyderabad, but nothing materialized. Since this was my last chance to see the city, and potentially the last time I'd be in Hyderabad for many years, I had to take matters into my own hands. I asked the front desk for a map, a guidebook, and a few recommendations. Then the transportation desk set up a cab ride to my first destination, Charminar, the square, four-towered heart of the city. It was a perfect if clichéd place to start my day of exploration.
Unfortunately, Charminar is also right in the heart of the "old city", the other half of Hyderabad housing the less privileged portion of the population. Charminar itself was certainly astounding. Even though the minarets were closed, the view from the top main floor was quite impressive. In each of the four directions, thoroughfares stretched off into the distance, revealing a bustling mass of humanity. I could see the swarming rickshaws, the teeming bazaar, and the hoards of Hyderabadi citizens going about their lives. It was humbling really. But it was a bit more personal once I descended back into those masses to find my next destination.
India has truly awakened. Here more than anywhere I have visited the contrast between technology and tradition is grossly apparent. Construction projects involve teams of mean working by hand, like ants assembling a colony. But every ant has a cell phone with text messaging and a camera. In the streets, beggars clutch at you constantly, but often speak extremely capable English and understand the western world. Above, signs advertise BizTalk Server and Oracle Training. Below, people sleep in the streets and beg for pennies in roaring traffic. And in the streets of Hyderabad this paradoxical turmoil boils unabated.
So I climbed back down the stairs of Charminar, a bit concerned for my future. Walking out the secured gate, I was immediately mobbed by beggars, young and old. A few (usually the youngest) spoke surprisingly good English. They asked where I was from, what I needed, where I was going. Some (the older members of the hoard) only knew what they needed to elicit a response from western hearts and minds..."food"..."baby"..."hungry"..."please". Why exactly had I decided to launch into this city alone? Ahh, for adventure, I remember. And what happens at this point in an adventure? When the odds and numbers are stacked against you? A gallant escape, with nary a rupee spared. How noble of me.
No, I did not make a contribution to the Hyderabadi poor. I certainly could have. Is it heartless of me? I don't think so; to have made one contribution would have certainly brought a torrent upon me, and it was already a worrisome situation. Instead, I made for a nest of rickshaws near the monument. One particularly bold youth, Abdul, stepped out of the crowd with a map of monuments, a plan for visiting them, and a welcoming smile. He was a salesman with exactly the product I needed, and for the next six hours he would be my guide. Off we went.
One of the previous would-be escorts had suggested Golkonda Fort as a place I should not miss. So naturally, with a friendly and aware guide at my side, that was the next place to visit.
Golkonda is the ruined former home of the Qutub Shahis, before they moved their home to what is now Hyderabad. It is an extensive maze of towers, walls, tunnels, temples, mosques, and overlooks atop a the highest hill in the area. After climbing several hundred feet to the summit, the sprawl of the city extends in all directions. No other view in Hyderabad gives such a complete picture of its size and extent. The fort itself is also impressive, with four-story defensive walls, a remarkable acoustic communication system, and numerous gardens tucked away behind tunnels and through towering arches. The climb to the top required a few breaks, during which Abdul told me what he knew of the fort's history in broken English and snapped photos of me at the major landmarks.
Eventually, we descended, and headed for the next stop on the tour: the Qutub Shahi Tombs.
There were seven Qutub Shahi kings, and they are all interned here in monumental tombs. Each tomb hosts an enormous dome (complete with perfect acoustics for singing, chanting, or tourists clapping and hooting), multiple entrances and side-tunnels, and inside the center, a symbolic (or perhaps more-than-symbolic?) monument to the hosted monarch. Most of the tombs contain a single such monument, but one had three: mother, father, and child. There are tombs here for the seven kings, many of their wives, and other relations. The writing on the monuments is in Arabic. That on the tour signs, in Telugu. Both Hindi and Muslim visit and pay their respects. And the city surrounds. I would say that the Qutub Shahi Tombs are at least as important a stop as the Golkonda Fort. If you are to visit the height of the kings' success, you should at least say an appropriate goodbye.
The rest of my adventure is somewhat less compelling, but perhaps still interesting. Hyderabad as we know it in the western world is really two cities to the Indians: Hyderabad and Secunderabad. In a peculiar quirk of fate, this pair is known to most Indians as "The Twin Cities", just like my home towns of Minneapolis and St Paul are known to Minnesotans (and many Americans) as "The Twin Cities". The very center of Hyderabad, bridging these two cities, is a large lake called Hussein Sagal. Arriving here, I have come full-circle to the heart of the "Twin Cities" on the other side of the world. Unfortunately, the interesting part of that story ends here. Hussein Sagal now appears to be little more than a tourist trap, firmly ensconced in the "new" part of the city. It is surrounded by miraculous science museums and modern family amusement parks. Put simply, I had no interest in anything more than driving past. Yes, it's a large lake in the middle of the city. Yes, there's a 60-foot-tall Buddha standing in the center. The noise of the traffic surrounding and the lights of the shopping malls and Pepsi signs tends to dampen any beauty or majesty that once was there. We moved on.
One item I had expressed interest in when talking to the front desk was "shopping". Unfortunately, we had different definitions of shopping. To me, shopping was visiting a "bazaar of the people", where I could browse sarees for my wife, hand-made toys for my son, and myriad other handicraft and localities. To the hotel, shopping meant "Macy's".
360s, $1 The final stop on my 8-hour Hyderabad tour was, sadly, the LifeStyle department store. It wasn't a bad store, really, since it did host a few local items and the prices were a little better than those at home. But it was a truly western-style store, with little interest to me as anything other than a landing zone. I wandered up to its second story, where I found discounted XBoxVCDs, and a coffee shop so grossly plastered with Windows Vista branding that they even had a "Windows Logo" menu prouding selling "Windows Vista WOWffee". Naturally, I ordered one due to the sheer absurdity of getting a free "Vista Wristlet", a white rubber bracelet emblazoned with the words "Windows Vista. The "Wow" starts here." To make it even more stomach-turning, it was a latte with orange flavoring. Delicious.
Of course I couldn't let the day end with a "WOWffee" and high-quality merchandise at low, low prices, so I called a fellow Sunny from the hotel to join me for a very satisfactory dinner. And then, after consuming excessive amounts of vegetable biriyani, chicken tikka masala, and garlic naan, I returned to the "green zone".
So what have I learned from my excursion?
- Exploring the poor districts of an unknown city alone can be frightening, but it certainly gives one perspective on the meaningless nonsense we labor over in the western world.
- Don't wait for someone to show you the world; just go see it yourself.
- The world is disappearing...so you better see it soon.
