JRuby Progress Updates: JRuby on Rails, IRB, and the Future

We've been very productive on JRuby over the past week. Progress is being made on many fronts, and I'm more excited than ever about JRuby's potential. Here's a few updates on Rails, IRB, and JRuby's future for those of you following along. It's a long post, but each subsection stands on its own.

JRuby on Rails

I've continued my work getting the Rails "generate" script to run with JRuby. The major hurdle we were coping with last time was parsing the database.yml file. First, a bit of background.

YAML, as most of you will know, is a markup language (or rather, YAML's not A Markup Language) used prominently in Ruby applications for configuration files and for some types of object persistence. YAML's creator, _why, originally wrote YAML parsers and libraries in the language of the target platform; in other words, the original YAML parser was pure Ruby code, written to use Ruby's compiler-compiler library RACC. Now Ruby's troubles with performance are fairly well-documented, but these issues were considerably more pronounced when doing such an intensive process as parsing a large YAML file. _why's solution was to write a new C library for parsing YAML called "syck". Syck did two things: first, it sped up YAML parsing considerably and allowed many languages to use the same parser via language plugin mechanisms; and second, it eliminated the need or availability for a pure Ruby YAML parser.

Enter JRuby. With YAML now being parsed almost exclusively using the C-based syck library, we have been forced to use an older version of the RACC-based pure Ruby parser. When starting work on Rails (and really, when playing with making RubyGems work) the complexity of the YAML parser brought out some problems in JRuby. That was a couple weeks ago.

Almost all of those problems have now been solved.

Our StringIO library (again, Ruby uses C code for StringIO to improve performance...see a pattern forming?) had been tested using all available test cases, but unfortunately those test cases did not cover the simple cases. When yaml.rb (the pure Ruby YAML parser we're using) started to make heavy use of StringIO, failures showed up. Tom Enebo is currently working on fixing the last of those failures, writing more extensive test cases at the same time. However, yaml.rb also contains a "stripped-down" version of StringIO for its own use. During my continued testing, I have been forcing it to use that version while Tom completes his fixes.

Other problems ranged from interpreter bugs (variable scoping, throw/catch not working, etc) to parser bugs (JRuby's parser did not take into account an "eval" called from within a block, which causes variables to be handled a bit differently). Those issues are now sufficiently resolved so that YAML does not show failures.

So back to Rails. Where the "generate" script originally got to the point of parsing database.yml and blew up, it now successfully parses that file and continues on. The next step in the "initialize_database" step of the initializer is to actually instantiate an ActiveRecord adapter based on database.yml. This is where the current failure lies, and where my attentions will be focused.

So to recap Rails progress, the "generate" script's call to the railties initializer successfully runs up to ActiveRecord instantiation, as well as successfully running a number of other initialize tasks. It's getting closer every day.

IRB

Oh, IRB, how we love thee. For those unfamiliar, IRB is the "interactive ruby" shell where you can enter in line-by-line Ruby code and immediately see results. Multi-line constructs like classes, methods, and modules are handled very elegantly, and therefore you can test some fairly complex bits of Ruby quickly and easily. It's a wonderful interactive environment for testing, learning, and experimenting with Ruby.

Unfortunately, it doesn't run under JRuby.

IRB is a very complicated beast. Running IRB results in almost every aspect of the underlying interpreter getting a good pounding; the parser is brutalized for parsing small snippits of code, the evaluator must translate that code into appropriate state changes, and any aspect of the Ruby language must be instantiable and callable interactively. Beyond even the Ruby aspects, IRB provides line-editing capabilities, tab completion, and command history features. Naturally, this presents many challenges for JRuby, and the ability to run IRB would be a huge demonstration of JRuby's maturity.

Running IRB under JRuby originally just blew up immediately; there were core bugs in the libraries and interpreter that prevented early stages of IRB's startup from completing successfully. Many of those issues were the same ones fixed for Rails' "generate" script, such as the parser/block issues and many interpreter bugs. Today I fixed another issue affecting both Rails and IRB, where throw/catch was not correctly passing back the symbol thrown. I was working on "generate", but remembered that I had stopped previous IRB work because of an apparent try/catch problem.

So I took a break from rails and attempted to start up IRB.

C:\rails>jruby C:\ruby\bin\irb
irb(#<irb::workspace:0x5a9c6e>):001:0>

To my amazement, IRB successfully started up. Although hopeful, I had always worried that there were core requirements of IRB that could never be satisfied by JRuby, and that even starting it up would never be possible. Seeing the IRB prompt comeup successfully was a huge relief to me and an unexpected nugget of joy. I'm so glad it happened in the morning; I'll be glowing all day.

Now don't get me wrong. IRB still doesn't work right. I naturally proceeded to type in the beginning of a class definition, and IRB blew up immediately after hitting enter. I never expected the prompt would just start working, and the blow up doesn't temper my joy in any way. There's still more work to do, but this is a very exciting milestone in my book. I now believe without a doubt that we will get IRB to run. The implications of successfully running such a complicated script in JRuby are tremendous, and finally reaching this milestone has made my day.

The Future

Ahh the future. Such a magical time. If not for the promise of the future, what point would there be in writing software. Truly, my greatest motivation for rolling out of bed each day is the possible future I will be walking into.

My hopes for JRuby's future are starting to take shape.

Recently, I encountered more issues with JRuby's performance being a bit lacking. Actually, let's just say it: JRuby is really slow right now. A microbenchmark recently posted to the ruby-talk mailing list implemented a brute-force Sudoku-solving aogorithm. The original poster's compared Ruby's performance to native C code; where the C code took seconds to run, the Ruby version of the algorithm took over half a minute.

Again, Ruby's struggles with performance are widely known. It's also obvious that Ruby's creators and developers are aware of these issues, since many core libraries are implemented in C and since Ruby 2.0 will boast a new interpreter and Vitual Machine as well as many VM features comparable to those in Java and .NET's runtimes.

Naturally, curious how JRuby would perform on this benchmark--and with full awareness that JRuby's performance is far from spectacular--I ran it and waited for a result.

And waited. And waited.

After a few minutes, I killed the VM, assuming that there was something broken in JRuby that prevented the algorithm from terminating successfully. I did a bit of debugging, traced into the very depths of JRuby's evaluator, and found nothing. As far as I could tell, progress was being made and the algorithm was moving forward. My findings warranted another run.

JRuby took over 800 seconds to complete the benchmark, around 13.6 minutes.

I will admit the realization that JRuby is an order of magnitude slower than C Ruby came as a bit of a shock to me. There are many definitions of slow; Ruby's "slow" is for most purposes "fast enough". Java's "slow" is in most cases much faster than is required, and in some cases faster than native C code. JRuby's "slow", it would seem, is a different beast altogether.

However, I am reactionary. Such disappointment immediately sours my stomach and gives me a headache. Could I have been wrong about JRuby's potential? Will this never work?

Performance is a unique problem in JRuby. Since we do not have the option of running native C code for any libraries, and since reimplementing core features in pure Java is both time-consuming and not in the spirit of what we're trying to accomplish, performance concerns have taken a back seat to functionality, compatibility, and correctness. Performance problems are not easily isolated, and never easily solved. However...I love a challenge.

The redesign of JRuby's interpreter over the past several months has been focused on two things: enabling missing features like continuations and green threading; and providing a Java-friendly design that could more easily transition to optimized interpreters and eventually bytecode compilation. What I've essentially been doing amounts to painstaking refactoring of all JRuby's functional guts, from the AST-walking evaluator to the class and object implementations to the threading, framing, scoping, and call mechanisms. All these areas were originally written and designed based on Ruby 1.6 code; there were flashes of OO genius, but the mostly procedural approach of Ruby's C code shined bright throughout JRuby. As you might guess, this is certainly the easiest way to port a language interpreter to any platform: reimplement the same code in your target language of choice. As you might also guess, this does not generally take advantage of that target language's best features.

In JRuby's case, a major missing piece was the inability to longjmp, C's function for leaping from one call stack to another. longjmp is heavily used (understatement!) in Ruby for everything from threading to continuations to exception handling. Missing longjmp in Java presents a very large hole when porting Ruby C. Many creative attempts to mimic longjmp were therefore created: exception-based flow control allowed loop keywords like 'next' to throw control back to a higher-level loop construct; a recursive evaluator repeatedly called itself for new AST nodes, ever-deepening the stack but always keeping lower nodes within the context of higher ones; exception-based "return sleds" allowed returns to bubble their results back up to the appropriate recipient; and on and on. Many of these approaches were extremely novel, worthy of their own papers and accolades. Indeed, several of them have shown up in academic papers and PhD theses in some form or another.

Unfortunately, these features still tried to mimic the way C code worked, which was never 100% achievable. longjmp is an extremely powerful tool that requires the capability to store, retrieve, and manipulate your own call stack. Java provides no such capability, and while exceptions do allow us to escape the stack--mimicking one aspect of longjmp--there is no ability to restore that stack. A new approach was needed.

Enter the JRuby redesign. In October of 2005, I began the process of unraveling JRuby's code with a number of design goals in mind:

1. The new interpreter must be iterative, rather than recursive, so escaping and restoring the stack are possible. This would enable continuations and green threading.
2. The JRuby code must be drastically cleaned up and simplified, and there must be a clear separation of concerns to allow future implementations of key subsystems.
3. JRuby must continue to work with no functional regression throughout this redesign.

The first two points are fairly straightforward. The new interpreter design enables us to provide all the required Ruby language features in a much more Java-friendly way. It also helps qualify JRuby as a real "VM", or at least a micro-VM layer on top of the JVM. I'm planning to start documenting this new design (since it has evolved over time and out of necessity), but it's fairly well-understood within the JRuby team.

The third goal, however, continues to be a serious pain-in-the-ass.

Ruby as a language and as a platform is poorly-specified. There is no conclusive specification; indeed the best spec is the incomplete (but still astounding) documentation provided by Dave Thomas's "Pickaxe" book, Programming Ruby. Given this lacking, the only way a Ruby interpreter can be determined to conform to the Ruby Way is by actually running it. Primarily, this means unit tests.

The Rubicon project was spawned out of a set of unit tests Dave and the PragProg folks created while writing the first edition of "Pickaxe". It tested out many of the features and scriptlets demonstrated in the book, and provided a wide but fairly shallow set of test cases to excercise Ruby features. Rubicon today exists as the "rubytests" project on RubyForge, where it has languished in recent years. Nobody likes writing tests after-the-fact, and the value of such tests is dubious.

JRuby makes heavy use of Rubicon, as well as some of Ruby's and our own internal unit tests, to ensure compatiblity and prevent regression. Anything not covered by those tests or by applications that run on JRuby remain unknown, untested areas until discovered by a new script or application. However, they're the best we've got right now. By implementing a Ruby that can run all or most of those tests as well as as few key applications, we can cobble together over time a pretty good Ruby. Current efforts to run more advanced applications like Rails or IRB are driven by the fact that those test cases do not excercise enough of JRuby to be conclusive, and the more we run the better we get.

When the redesign began, it was immediately apparent that without continually running those test cases and applications we would be diving down the rabbit hole with no insurance; refactoring an entire interpreter is obviously extremely dangerous without a language spec or appropriate unit tests. Goal #3 above became an absolute necessity.

As a result, after every major VM change these past months we have continued to run test cases and scripts to ensure that regressions are prevented or kept to a bare minimum. JRuby's codebase is not terribly large; a wholesale refactoring would not normally take months to complete. However with the added restriction that it must continue to work, the time-to-implementation increased tremendously. In addition, and of primary importance to performance, tradeoffs had to be made between "doing things right" and "doing things fast". Things had to get worse before they could get better.

JRuby's performance was no great shakes before the refactoring, but the 0.8.2 release appears to be as much as 30% faster than the current HEAD version in some scenarios. While such a decrease in speed is worrisome, it comes with the fact that the new VM will enable performance-enhancing optimizations in ways the original never could.

My interest in those enhancements was revitalized by the poor benchmark results. Perhaps one of the most impotant is JRuby's eventual ability to compile Ruby code into Java bytecode. After a long discussion with my good friend Kelly, I believe we have devised a way to make compilation happen without sacrificing goal #1 above. More on that in a future post.

I also started looking to isolate the performance problems. Immediately, I started looking at the redesigned interpreter engine. To make a long story short, the current interpreter has more overhead than the original because rather than recursing for additional nodes in the AST, it "trampolines" from one to the next. Each node encountered is associated with a number of instructions; those instructions are executed in sequence, allowing the Java call stack to remain at the same level and enabling the potential for continuations and green threading (since we can now step away from one instruction sequence and into another, efffectively doing what longjmp does for C). This flexiblity initially comes with decreased performance since the instruction fetch cycle, decoding that instruction into sub-instructions, maintaining a cursor within the AST and instruction sequence, and double-dispatching for each instruction all add overhead.

Small changes in the interpreter can have a drastic effect on performance, and so to put my mind at ease I went ahead with a couple optimizations I had put off. Specifically, I reworked the way flow-control, return values, and exception handling worked, reducing the number of calls and objects created. The results were very promising: a subset of the sudoku benchmark improved by roughly 9%. Since this small change only represented one tiny aspect of the interpreter, my fears have been temporarily put to rest.

Based on my reexamination of the interpreter and on the results of this small optimization, I do not believe that JRuby's performance issues will be a problem much longer. I'm also confident that we can begin improving performance rather than degrading it, since the current interpreter is only a few steps off from its eventual structure. Combining future interpreter optimizations with potentially compiling many or all pure Ruby methods to Java bytecode means we should see drastic improvements in the coming months. Will we ever run as fast as Ruby 1.8 or 2.0? Will we run faster? Time will tell.