Friday, November 02, 2007

Top Five Questions I Get Asked

I'm at RubyConf this weekend, the A-list gathering of Ruby dignitaries. Perhaps one day I'll be considered among them.

At any rate, here's the top five questions I get asked. Perhaps this will help people move directly on to more interesting subjects :)

#1: Are you Charles Nutter

Yes, I am. I look like that picture up in the corner of my blog.

#2: How's life been at Sun/How's Sun been treating you?

Coming to Sun has been the best professional experience of my life. When they took us in over a year ago, many worried that they'd somehow ruin the project or steer it in the wrong direction (and I had my occasional concerns as well). The truth is that they've remained entirely hands-off with regards to JRuby while simultaneously putting a ton of effort into NetBeans Ruby support and committing mission-critical internal projects to running on JRuby. Beyond that, I've become more and more involved in the growing effort to support many languages on the JVM, through changes to Java and the JVM specification, public outreach, and cultivating and assisting language-related projects. It's been amazing.

Along with this question I often get another: "What's it like to work at Sun?" Unfortunately, this one is a little hard for me to answer. Tom Enebo and I work from our homes in Minneapolis, so "working at Sun" basically means hacking on stuff in my basement for 16 hours a day. In that regard, it's like...great.

#3 What's next for JRuby?

Usually this comes from people who haven't really been following JRuby much, and that's certainly understandable. We had our 1.0 release just five months ago, and we've had two maintenance releases since then plus a major release coming out in beta this weekend. So there's a lot to keep track of.

JRuby 1.0 was released in RC form at JavaOne 2007 and in final form at Ruby Kaigi 2007 in Tokyo. It was focused largely on compatibility, but had early bits of the Ruby-to-bytecode compiler (perhaps 25% complete). Over the summer, we simultaneously worked on improving compatibility (adding missing features, fixing bugs) and preparing for another major release (with me spending several months working on the compiler and various performance optimizations). That leads us to JRuby 1.1, coming out in beta form this weekend. JRuby 1.1 represents a huge milestone for JRuby and for Ruby in general. It's now safe to say it's the fastest way to run Ruby code in a 1.8-compatible environment. It includes the world's first Ruby compiler for a general-purpose VM, and the first complete and working 1.8-compatible compiler of any kind. And it has finally reached our largest Rails milestone to date: it runs Rails faster than Ruby 1.8.x does.

What comes after JRuby 1.1? Well, we need to finally clean up our association/integration with the other half of JRuby: Java support. Java integration works great right now--probably good enough for 90% of users. But JRuby still doesn't quite fit into the Java ecosystem as well as we'd like. Along with continuing performance and compatibility work, JRuby 2.0 should finally run the last mile of Java integration. Then, maybe we'll be "done"?

#4 What do you think of [Ruby implementation X]?

I usually hear this as "what bad things do you want to say about the other implementations", even though people probably don't intend it that way. So, for the record, here's my current position on the big five alternative Ruby implementations.

Rubinius: Evan and I talk quite a bit about implementation design, challenges of implementing Ruby, and so on. JRuby uses Rubinius's specs for testing, and Rubinius borrows/ports many of our tests for their specs. I've even made code contributions to Rubinius and Sun sponsored the Ruby Hackfest in Denver this past September. We're quite cozy.

Technical opinion? It's a great idea to implement Ruby in Ruby...I'd rather use Ruby than Java in JRuby whenever possible. However the challenge of making it perform well is a huge one. When all your core classes are implemented in Ruby, you need your normal Ruby execution to be *way* faster than Ruby 1.8 to have comparable application performance...like orders of magnitude faster. Think about it this way: if each core class method makes ten calls to more core class methods, and each of those methods calls ten native methods (finally getting to the VM kernel), you've got 100x more method invocations than an implementation with a "thicker" kernel that's immediately a native call. Now the situation probably isn't that bad in Rubinius, and I'm sure Evan and company will manage to overcome performance challenges, but it's a difficult problem to solve writing a VM from scratch.

We all like difficult problems, don't we?

XRuby: XRuby is the "other" implementation of Ruby for the JVM. Xue Yong Zhi took the approach of writing a compile-only implementation, but that's probably the only major difference from the JRuby approach. XRuby has similar core class structure to JRuby, and when I checked a few months ago, it could run some benchmarks faster than JRuby. Although it's still very incomplete, it's a good implementation that's probably not getting enough attention.

Xue and I talk occasionally on IM about implementation challenges. I believe we've both been able to help each other.

Ruby.NET: Ruby.NET is the first implementation of Ruby for the CLR, and by far the most complete. Wayne Kelly and John Gough from the University of Queensland ran this Microsoft-funded project when it was called "Gardens Point Ruby.NET Compiler", and Wayne has continued leading it in its "truly open source" project form on Google Code. I exchanged emails many times with Wayne about strategies for opening up the project, and I involved myself in many of the early discussions on their mailing list. I'm also, oddly enough, listed as a project owner on the project page. I'd like to see them succeed...more Ruby options means Ruby gains more validation on all platforms, and that means JRuby will be more successful. Ironic, perhaps, that long-term Ruby and JRuby success depends on all implementations doing well...but I believe that to be true.

IronRuby: IronRuby is the current in-progress implementation of Ruby for the CLR, primarily being developed by Microsoft's John Lam and his team. IronRuby is still very early in their implementation, but they've already achieved one big milestone: they have their Ruby core classes available as a public, open-source project on RubyForge, and you, dear reader, can actually contribute. In my book, it qualifies as the first "real" open source project out of Microsoft since it finally recognizes that OSS is a two-way street.

Technically, IronRuby is also an interesting project because it's stretching Microsoft's DLR in some pretty weird directions. DLR was originally based on IronPython, and it sounds like there's been growing pains supporting Ruby. Ruby is a much more complicated language to implement than Python, and from what I've heard the IronRuby team has had to diverge from DLR in a few areas (perhaps temporarily). But it also sounds like IronRuby is helping to expand DLR's reach, which is certainly good for MS and for folks interested in the DLR.

John and I talk at conferences, and I monitor and participate in the IronRuby mailing list and IRC channel. We Rubyists are a close-knit bunch.

Ruby 1.9.1/YARV: Koichi Sasada has done the impossible. He's implemented a bytecode VM and compiler for Ruby without altering the memory model, garbage collector, or core class implementations. And he's managed to deliver targeted performance improvements for Ruby that range from two times to ten times faster. It's an amazing achievement.

Ruby 1.9.1 will finally come out this Christmas with Koichi's YARV as the underlying execution engine. I talk with Koichi on IRC frequently, and I've been studying his optimizations in YARV for ideas on how to improve JRuby performance in the future. I've also offered suggestions on Koichi's Fiber implementation, resulting in him specifying a core Fiber API that can safely be implemented in JRuby as well. Officially, Ruby 1.9.1 is the standard upgrade path for Rubyists interested in moving on from Ruby 1.8 (and not concerned about losing a few features). JRuby will support Ruby 1.9 execution in the near future, and we already have a partial YARV VM implementation. JRuby and YARV will continue to evolve together, and I expect we'll be cooperating closely in the future.

#5 How's JRuby's Performance?

This is the first question that finally gets into interesting details about JRuby. JRuby's performance is great, and improving every day. Straight-line execution is now typically 2-4x better than Ruby 1.8.6. Rails whole-app performance ranges from just slightly slower to slightly better, though there have been reports of specific apps running many times faster. There's more work to do across the board, but I believe we've finally gotten to a point where we're comfortable saying that JRuby is the fastest complete Ruby 1.8 implementation available.

See also my previous posts and posts by Ola Bini and Nick Sieger for more information on performance. There's more detail there.

13 comments:

AkitaOnRails said...

Excelent post, I'll translate this into brazilian portuguese for my blog as well. Very insightful. I am eager to try the new 1.1 release. Congrats!

Anonymous said...

Two question :

1) Is JRuby team planning to merge more code from Rubinius (beyond the specs) into JRuby at some point in the roadmap?

2) Is JRuby going to go beyond the ruby language, adding features that are not in the MRI? What do you think about a future with several Ruby-VM-dialects (alà Smalltalk) ?

Thx.

Anonymous said...

When you say

JRuby's performance is great, and improving every day. Straight-line execution is now typically 2-4x better than Ruby 1.8.6.

is that with JRuby 1.0.x or 1.1b?

Thanks for your hard work in bringing Ruby to the JVM!

Charles Oliver Nutter said...

Anonymous: 1.1 is the big performance release. 1.0.2 is still mostly interpreted, and usually about 2-3x slower than Ruby 1.8.6.

Isaac Gouy said...

Going from JRuby 1.0 to 1.1b1 on the benchmarks game looks like this

binary-trees 10.8% faster
chameneos 13.0% faster
cheap-concurrency slower 27.3%
fannkuch 39.1% faster
fasta 35.8% faster
k-nucleotide slower 29.0%
mandelbrot 29.8% faster
meteor-contest 37.4% faster
n-body 68.5% faster
nsieve slower 24.4%
nsieve-bits 29.3% faster
partial-sums slower 4.5%
pidigits slower 13.5%
recursive 48.8% faster
regex-dna slower 3.1%
reverse-complement 15.5% faster
spectral-norm 19.8% faster
startup slower 72.3%
sum-file slower 50.6%

Charles Oliver Nutter said...

isaac: Great, thanks for updating the numbers! It will give us some area to examine before 1.1 final.

I'm confused why some numbers are still slower than Ruby though. I was unable to find any shootout benchmark that ran slower for me in JRuby than in Ruby. Perhaps I need to set up the same shootout bootstrap process to see for myself why the numbers are lower than I'd expect.

Charles Oliver Nutter said...

isaac: I discovered something interesting about the benchmarks: because they're .ruby instead of .rb, the (admittedly weak) logic in the compiler is not parsing the filename into a class name correctly, resulting in it failing to compile and falling back on the interpreter. I would expect this to severely impact every benchmark in the suite.

It's a JRuby bug, so this is just a heads-up for you and for readers, but we'll have it resolved before 1.1 final.

Isaac Gouy said...

From my perspective that's great - the reason I bothered testing the beta was on the off-chance it might hit some odd bug ;-)

Charles Oliver Nutter said...

isaac: and since you did, I'm quite happy about it :)

Charles Oliver Nutter said...

pedro:

1) We are actively exploring the possibility of incorporating Rubinius's pure-ruby core classes for experimentation, but probably not for shipping, final versions of JRuby until we can overcome the resulting performance hit.

2) Outside of Java integration, which is of course entirely different from Ruby, we have no plans to extend Ruby in incompatible ways. I believe the other implementers are largely adhering to the same modus operandi.

Giles Bowkett said...

How about Merb on MRI vs. Merb on JRuby?

Charles Oliver Nutter said...

Giles: in terms of performance? I don't believe anyone's tried it. As far as I know, since Mongrel works in JRuby just fine, Merb should work as well. It's on a list of things I want to play with, but haven't had time.

Isaac Gouy said...

Now with .rb extensions the JRuby 1.0 to 1.1b1 improvement on the benchmarks game looks like this

binary-trees 21.5% faster
chameneos 46.3% faster
cheap-concurrency slower 11.7%
fannkuch 72.5% faster
fasta 41.8% faster
k-nucleotide 16.3% faster
mandelbrot 63.6% faster
meteor-contest 39.6% faster
n-body 71.9% faster
nsieve slower 34.7%
nsieve-bits 57.5% faster
partial-sums 28.1% faster
pidigits slower 11.4%
recursive 59.9% faster
regex-dna slower 1.0%
reverse-complement 24.7% faster
spectral-norm 41.2% faster
startup slower 72.5%
sum-file slower 25.8%

JRuby is faster than Ruby on all but pidigits, regex-dna, reverse-complement and startup (hello world) is so slow that must be effecting everything.