Friday, May 12, 2006

And They Said JRuby was Dead...

Things are looking very good for JavaOne.

The past several months have been a period of furious development on the JRuby project, with milestone after milestone flying by. Four months ago we couldn't run most basic Ruby apps and were still wrestling with basic 1.8 compatibility issues. Since that time we've gotten IRB working, Rake running, RubyGems almost working, and then managed to process a simple Rails request. All the while, our existing libraries have grown in stability and completeness, and we've attracted many new contributors to the project. This stone is gathering no moss.

JavaOne is a going to be a big event for us, but we wanted something more. What would the week before a major conference be without some buzz-worthy announcements to make?

Hold on to your hats.

Where We've Been

JRuby has been around for around five years. It was originally based on Ruby 1.6 C code, and was pretty much a direct port from C to Java. Over those years, up until Fall of 2005, the focus was mainly on basic compability issues, updating libraries and language semantics to 1.8 levels, and generally just making things work well enough for use. All told, it was a pretty good port, but there was a long way to go. There were a few Java apps that used it for scripting, but the tide was turning away from JRuby with the release of ever-more-complex Ruby applications that were increasingly difficult to run on a partial implementation.

Starting this past Fall, and really gaining steam since the start of 2006, our focus shifted from simple 1.8 compatibilities and interpreter tweaks to much more ambitious goals. We began to tackle the killer apps for Ruby one by one, fixing what broke and getting a much more intimate knowledge of both Ruby and JRuby internals than we had before. First came early runs of Rake. They worked, but required changes to the way external scripts launched. Then something a bit more lofty: I disappeared into my office for a week to get IRB working...and work it eventually did. Then we realized that RubyGems would be a great target; while it's not yet working 100%, it's now really close.

Then came Rails.

The Big Ticket

There is little doubt that Rails is the app that's selling Ruby today. Ruby has become, in many circles, synonymous with Ruby on Rails. While there's a discussion to be had about whether a single killer app should hold so much sway, it's impossible to argue that Ruby would be where it is today without Rails dragging it into the spotlight. Rails has, more than any other app, brought Ruby to the world.

We just had to get a bite of that apple.

Getting Rails to run in JRuby is a very challenging task. For those of you that know what Rails does, imagine the code that's required to do that. Think of all the magic necessary to get things so slick and transparent. Look at the size of the Rails codebase which, while far smaller than a beastly Java library, is positively enormous compared to the typical Ruby app. If you've ever seen the code, you know what kinds of tricks and traps there are in the Rails code, and how many dark alleys you might venture down when debugging a problem deep within its guts.

Now imagine running it on an interpreter that breaks in weird and indeterminate ways.

We have fixed hundreds of issues on the road to getting Rails working. Every step of the way, we have had to pore through the Rails code, digging for that one irreducible test case that demonstrates a bug, sometimes sifting through tens of files, thousands of lines of code. It has been an extreme challenge. However...

What Would You Do With JRuby on Rails?

Today, we can announce that a simple end-to-end Rails application has successfully run under JRuby.

As reported on Tom's blog, a little CRUDified cookbook application can now deliver a form and process submissions correctly. Using an ActiveRecord-to-JDBC adapter (yes, you read that right) donated by Nick Sieger, the app is able to generate a form, display a table of results, and process creates and deletes. All this magic, all working on JRuby. We're ecstatic.

Can It Truly Be?

See for yourself. It's even wrapped in a little servlet that invokes Rails FastCGI-style, keeping it loaded in memory for additional requests. It's the culmination of all our experiments, bug fixes, and hard work over the past several months. And it sure feels good.



Onward and Upward

Of course this demo is slightly canned. We coded to the test, and the average Rails app still isn't going to work out of the box. There's plenty of little things missing, like reliable session management, more ActiveRecord magic, and various rendering flaws, to list a few we know about. But as a technology preview, running Rails essentially unmodified--even on a simple end-to-end app--is our biggest milestone yet.

With the recent successes, we plan to target the end of summer for full, general-purpose Rails support. With more contributors and more time to work, that timetable could move up, but there's also the compiler work that has begun in earnest. It's going to be a great summer for coding, and a great year for dynamic(-typed) lanaguages on the JVM.

Oh, and one last note for the perennial skeptics and naysayers out there: Rails on the JVM is coming sooner than you think.

A Few Other Updates

Here's another couple items to hold you off until the next round of milestones:

IRB glitches resolved
A remaining IRB bug that caused declared classes to be scoped under Kernel (as in class A would be scoped as Kernel::A) has been resolved. I had to get this fixed to demo IRB at JavaOne

RubyGems works, sorta
Tom discovered that if a known good RubyGems install (from a C Ruby installation, for example) was copied into JRuby's dirs, gems would require and load correctly. So although our RubyGems and gem install code isn't quite there yet, actually using gems seems to be working. Huzzah!

Improved codegen DSL
John Lam, of the RubyCLR project, pointed me to a nice little eval trick that has helped simplify my code-generation DSL a bit. By eliminating most of the block params, it's a bit less verbose and certainly prettier:

class_bytes = ClassBuilder.def_class :public, "FooClass" do
def_constructor :public do
call_super
return_void
end

def_method :public, :string, "myMethod", [:void], [:exception] do
call_this GenUtils.array_cls(:string), "getStringArr"
call_this :string, "getMessage"
return_top :ref
end

def_method :private, :string, "getMessage", [:void] do
construct_obj :stringbuffer, [:string] do
constant "Now I will say: "
end

call_method :stringbuffer, "append", :string, :stringbuffer do
constant "Hello CodeGen!"
end
call_method :string, "toString", :void, :stringbuffer
return_top :ref
end

def_method :public, GenUtils.array_cls(:string), "getStringArr" do
construct_array :string, 5 do |i|
constant "string \##{i}"
end

array_set 2, :string do
constant "replacement at index 2"
end

return_top :ref
end
end
I'll be going full-speed-ahead on compilation after JavaOne, you can count on that.

YAML parser improved?
One of our favorite contributors, Ola Bini, now has a working version of a YAML 1.1-compliant parser which is able to load fairly large files. There's still some work to do, but if all goes well we may be able to migrate to it in the near term.

Dynamic languages press conference
The JRuby team (Tom and I) will be part of a JavaOne Day-1 press conference on the future of dynamic languages on the JVM. Along with the Quercus guys from Caucho (PHP for the JVM) and Roberto Chinnici from Sun (doing cool stuff with JSR223), we'll have our own ten minutes to show the press how cool JRuby really is. It oughta be a fun time, and should help get JRuby a bit wider exposure.

This is just too much fun.

15 comments:

Anonymous said...

Once again: Great article and great work of you JRuby-guys!!!

With the engagement you are showing this will definitly be a success! One thing about donations, marketing and so on: Are you working at sun? Or is sun in contact with you guys? Taking into account the effort that microsoft is putting into dynamic langs on .net it would be cool if sun donates you and puts some marketing power behind cool projects like this one. I think it would be quite interesting to hear for non insiders if sun cares about projects like yours or if they leave the future of dynamic langs on the jvm solely to the open source community

Regards Jan Prill

Anonymous said...

This is damn cool... Good work guys!

Anonymous said...

This is awesome news! Well done guys, keep it up!

Anonymous said...

great work.

but why?

why is it better to run a RoR app on JVM than to run it on plain Ruby?

i blogged about this here: http://www.nekomancer.net/blog/archives/ruby-on-rails-on-a-jvm-why

Charles Oliver Nutter said...

Re: Anonymous "why?"

I responded on your blog, but there's one reason I neglected to mention:

Because we want to!

JRuby is not just a project to accomplish the goal of a JVM-based Ruby implementation. It's also a great challenge to those of us for whom day-to-day Java or J2EE development has become mundane. Now we're able to kill two birds with one stone: bring one of our favorite langages into that day-to-day world, and have fun making it happen.

Anonymous said...

Re: Charles Oliver Nutter said... Re: Anonymous "why

(i now chose an identity. "Re: Anonymous" looks so depressing :-( )

thanks for the comments.

first: the "because it's fun" is already a sufficient reason :)

now regarding your points: they basically fall into either 'interact with java' or 'run in java'.

i understand the 'interact with java' part.

could you elaborate a bit more on the 'run in java' part?

you many times mention 'java servers'. do you mean J2EE EJB containers? or "java.exe myclass" type applications?

is JRuby able to run inside an EJB container? what happens with the threads or IO?

Charles Oliver Nutter said...

Gabor: JRuby in its current form might require some tweaks to run in an EJB container, but it's not far off. Threading and IO do need to be tightly controlled, but Rails itself does not use threads and the only IO it needs we can easily provide. Most of the issues preventing JRuby from running inside a container today are superficial, and won't be issues much longer.

Anonymous said...

We would definately use this. We've got an existing Java app which is successfully used in the Enterprise, and we really want to write additional functionality in Rails - starting with low-value but fun stuff but possibly migrating more and more of the core app to Rails over time. This will be so much easier if Rails is "just another" JVM app, rather than having to explain why we're installing this apparently faddish Ruby on Rails onto someone's mission critical enterprise server.

I'm not saying that Rails on native Ruby isn't appropriate, just that if we can stick it into our core code then it is our problem how the code got written, whereas if we introduce another app server, we've got some convincing to do which is expensive and somewhat risky to our relationship with the client.

Blending Rails and J2EE is important for another reason, apart from customer acceptance. It is going to get very hard to convince developers to keep working in Java, given how much fun Rails is. We're hitting that problem already, the developers aren't really that keen on working on the (mortgage-paying) Java app, they all want to write the addons in Rails (which produce no revenue).

Keep up the good work! If we ever get any cash, we'd be very happy to make a contribution of some kind - as I suspect would a lot of other people with Java-based apps.

Anonymous said...

I've been able to get some gems installed ... I have all the gems I need in a given directory, and you have to issue to gem install command when in that directory, it worked for me :-)

insight:~/gems $ jruby /usr/bin/gem install rails-1.1.2 --local --backtrace --debug --no-rdoc --no-test --no-wrappers
Attempting local installation of 'rails-1.1.2'
Successfully installed rails, version 1.1.2

I had to hack up some of the Ruby code in fileutils.rb and in some of the rubygems files, but it wasn't anything major ...

Anonymous said...

Congratulations! Well done indeed.

Anonymous said...

Unbelievable good news, you guys rock. I'm really looking forward to the great potential of JRuby, given that the advance of YARV is quite slow and a JIT is still in dream. Also glad to hear Ruby.net and JRuby team help each other. Ruby.net is for sure important for the expanding adoption of Ruby as well.

Anonymous said...

Any thoughts about getting JRuby's continuations to be serializable? This would be interesting to a number of folks.

Charles Oliver Nutter said...

Serializable continuations have been mentioned before. Since we do control all state necessary to save off a continuation, it would theoretically be possible to serialize as well. However, we have not started thinking about this much yet (since as of now we don't support continuations either).

Anonymous said...

Charles, I'm happy to hear it is on your radar. I'm thinking Ruby would be a very pleasant language to do control flow programming, ala how Rhino is used for Cocoon's flowscript.

Anonymous said...

I love JRuby! Great project! Keep going!