Saturday, October 31, 2009

Missed RubyConf? Attend Qcon's Ruby Track!

This year, RubyConf reportedly reduced their attendee cap to 250 people (Update: actually 450 people), after hosting a 500 to 600-person conference last year. As you can imagine, this meant a lot of people that wanted to attend were not able to get tickets. To complicate matters, the RubyConf registration site happened to go live during the middle of the night EU time, and by the time most Europeans woke up it was already sold out. What's a Rubyist to do?

Well there's another option. Working with Ryan Slobojan of InfoQ and the organizers of Qcon San Francisco, I'll be hosting a one-day Ruby track the day before RubyConf! Qcon's main conference runs Wednesday through Friday, with my track on Wednesday, November 18th.

And now the REALLY good news! Because we wanted this to be a fallback for folks that could not attend RubyConf, we realized that the full conference fee was simply too high (ranging from $1500 up). So to make it possible for people to attend just the one day Ruby track, you can register with the code "rubywednesday" to get a drastically reduced $350 one-day conference pass. And to sweeten the deal even more, you can pop over to other tracks and attend the keynotes that day. Yes Virginia, there is a Santa Claus!

Those of you attending RubyConf are also welcome to attend this one-day track as well; most of the presentations won't overlap. Here's the lineup, including a special opening presentation by Yukihiro Matsumoto himself!

10:20 - "The Many Facets of Ruby" track opening by me

Ruby has seen a dramatic growth in popularity over the last few years, and there are now many facets to the Ruby story - multiple implementations, game-changing web frameworks, and large-scale use in enterprise solutions. Join us as we explore many aspects of Ruby in today's world.

10:30 - "Why we love Ruby?" by Yukihiro Matsumoto
Why we love Ruby? I have asked myself this question repeatedly. In this presentation, I will disclose my answer as of 2009. The keyword is QWAN.

11:45 - "Basking in the Limelight" by Paul Pagel
Limelight is Ruby on the desktop. Build applications with multiple windows, or just one window. Take control of the desktop, or play nicely with the desktop. Create fun animated games, or productive business apps. Develop rich internet applications, or unwired apps to make you rich. Publish your apps on the internet, or keep them for you and your friends. Do all this, writing nothing but Ruby code, in Limelight.

13:45 - "You've Got Java in my Ruby" by Thomas Enebo
JRuby is now well-established as a popular alternative implementation of Ruby.But why would you want to use it? How can it help you? This talk will detail some of the more interesting differences and advantages of using JRuby. Expect to get a better understanding of how Java makes a faster and more stable Ruby as well as how you can leverage Java features as an extra set of tools for your project.

15:00 - "Rails 3" by Yehuda Katz
I don't have a full abstract for this, but it's what you might expect...an overview of why Rails 3 is really "growing up" the framework, making it more clearly componentized and easier to adapt to more complicated (dare I say "enterprise") applications in the future. In working with Yehuda I know he's also paid special attention to performance.. Rails 3 is going to be excellent.

16:30 - "Rails in the Large: How Agility Allows Us to Build One of the World's Biggest Rails Apps" by Neal Ford
While others have been debating whether Rails can scale to enterprise levels, we've been demonstrating it. ThoughtWorks is running one of the largest Rails projects in the world, for an Enterprise. This session discusses tactics, techniques, best practices, and other things we've learned from scaling rails development. I discuss infrastructure, testing, messaging, optimization, performance, and the results of lots of lessons learned, including killer rock-scissors-paper tricks to help you avoid babysitting the view tests!


I think it's going to an outstanding track, and I'd probably pay the $350 just to see Matz speak if I knew I wouldn't get another chance for a long time. Limelight looks like an outstanding way to build rich client apps using JRuby, and of course you know I like JRuby. Tom will show some of the latest advancements we've done in JRuby, including the ability to produce "real" Java classes at runtime for integrating with Java frameworks better. Rails 3 I've described above, but you really have to see Yehuda present it himself. And of course everyone would like to know how to scale Rails to the moon...Neal knows his stuff.

Here's the track page: The Many Facets of Ruby

And the full conference schedule for Wednesday.

And finally, the registration page (don't forget to use code "rubywednesday").

I really hope to see you all there, so you can get your Ruby conference fix this fall. Tell your friends and let me know if you have any questions!

Friday, September 11, 2009

Announcing JRubyConf 2009

Good news everybody! We're finally going to have JRubyConf!

After over 3 years of heavy development, dozens of deployments and hundreds of users, it's time for a conference for JRuby users. We've talked about it on the JRuby mailing lists, polled users, and seen other JRubyists do the same. And the chorus slowly grew: you all wanted JRubyConf more and more.

Now, thanks to Engine Yard, who's producing the conference, and to sponsors EdgeCase and ThoughtWorks, we'll host the first ever JRubyConf the day after RubyConf, on Sunday November 22nd. This should allow folks attending RubyConf to also attend JRubyConf and not have to schedule a separate trip.

So here's the details:

What: JRubyConf 2009 (the first ever!)
When: Sunday, November 22nd; the day after RubyConf 2009
Where: Same hotel as RubyConf, the Embassy Suites at San Francisco Airport. You don't even have to switch locations!
Why: Because we love you!
Price: FREE!!!
We've been putting together a wide range of talks and speakers, from the JRuby core team members to real-world users; from the latest on deploying Rails to gaming and desktop development. It's going to be a fast-paced event with something for everyone, and best of all, it's FREE!

Space is limited for the event, and you will have to register separately from RubyConf to secure your seat (but you don't have to go to RubyConf to attend JRubyConf!). Check out www.jrubyconf.com for information, registration, and so on, and be quick about it!

See you at JRubyConf 2009!

Wednesday, August 26, 2009

Introducing Surinx

In June of this year, I spent a few hours formulating a dynamic cousin to Duby. Duby, if you don't remember, is a static-typed language with Ruby's syntax and Java's type system. Duby supports all Ruby's literals, uses local type inference (only argument types *must* be declared), and runs as fast as Java (because it produces nearly identical bytecode). But with the advent of invokedynamic, Duby needed a playmate.


Enter "Juby". Juby is intended to be basically like Duby, in that it uses Java's types and Ruby's syntax. But it takes advantage of the new invokedynamic opcode to be 100% dynamic. Juby is a dynamic Duby, or perhaps a dynamic Java with Ruby syntax. It's not hard to comprehend.

But the name was a problem. Juby sounds like "Jewbie", a pejorative term for a Jewish person. So I sent out a call for names to the Twitterverse, ultimately ending up with far more names than I could choose from.

The name I have chosen, Surinx, has a simple story attached. You see, when James Gosling created Java, he originally named it "Oak", after the tree outside his window. So I followed his lead; the tree (a bush, really) outside my window is a lilac, Syringa vulgaris. The genus "Syringa" is derived from New Latin, based on a Greek word "surinx" meaning "shepherd's pipe" from the use of the Syringa plant's hollow stems to make pipes. Perhaps Surinx is building on the "hollow stem" of the JVM to produce something you can smoke (other dynamic languages) with. Combined with its cousin "Duby", we have quite a pair.

And in other news, the simple Surinx implementation of "fib" below (identical to Ruby) manages to run fib(40) in only 7 seconds on current MLVM (OpenJDK7 + invokedynamic and other tidbits), a five-fold improvement over JRuby's fastest mode (jruby --fast).

def fib(a)
if a < 2
a
else
fib(a - 1) + fib(a - 2)
end
end

Given that JRuby has started to support invokedynamic, the solid performance of Surinx bodes very well for JRuby's future.

Please welcome Surinx to your language repertoire!

Sunday, August 02, 2009

Which Deployment for JRuby on Rails?

We often get the question "which deployment option is the best for JRuby on Rails?" The truth is that it depends on what you need out of deployment.


If you have a fairly straightforward Rails app without a lot of service dependencies and a greenfield deployment target, your best bet is probably the GlassFish gem right now. It performs really well, can handle high loads and high concurrency, and automatically detects Rails' threadsafe mode, scaling better when it's turned on. I'm no longer a Sun employee, and I still think the GF gem is an outstanding piece of work. Here's my howto on GF gem + JRuby on Rails + Apache. Update: Here's information on using Capistrano with the GlassFish gem.

If you have an existing Java EE or web container like Tomcat, JBoss, GlassFish, WebLogic, or WebSphere, you want to look into Warbler. Warbler packages your application as a .war file, suitable for deployment on any of these standard servers. The Warbler wiki is the best place to learn about deploying with Warbler.

If you're looking for something that's somewhat greenfield but also needs more advanced services like scheduled asynchronous jobs, web services, and some level of EE integration, you should look at JBoss's TorqueBox, a customized JBoss specially tailored for deployment of Rack-based apps (like Rails) on JRuby.

If you're looking for a hosting provider that can take your app and make it "just work", then you should look into Engine Yard's JRuby cloud offering. We don't yet have it 100% ready to go, but it won't be long and it will be fantastic. For now you can give us some direction and input on what that hosting/cloud should look like.

All told, there's a lot of great options for JRuby deployment, and no one of them is going to be perfect for everyone. Choose wisely, and join the JRuby mailing lists if you have questions.

Saturday, August 01, 2009

Return of Ruboto!

It's been a while since I was able to work on JRuby's Android support, but tonight I managed to finally circle back. And I've got something much more impressive working today: a real IRB application!



(And yes, this works just fine on the phone too)

It turned out to be incredibly easy to get this working. I'm not using any stinking plugin because they all seemed to just get in my way. So I generated a dummy application using the "android" tool, dropped the jruby jar in "libs", wrote up a quick interactive console, built and signed it, and that's all there was to it.

JRuby turns out to work very well for this sort of thing because we have an interpreter, so we can parse and execute code dynamically. Hooray for interpreted support!

I had to tweak a couple things to work around shortcomings in Android:

  • I had to edit the dx tool to allow up to a 1024M heap, since JRuby's jar has a ton of stuff in it
  • I had to catch and swallow an ArrayIndexOutOfBounds exception coming out of Dalvik's enum support. Bug!
This is of course a proof-of-concept. Writing full applications in Ruby isn't far behind, but we'll need a couple adjustments to JRuby to support it well:
  • Ability to run 100% precompiled with no runtime code generation
  • Strip out parser, interpreter, compiler, and bytecode-generation bits to shrink the jar
  • Tidy up the AOT compiler and wire it into the app build process
  • Generate some Ruby stub logic for the Android APIs, so they'll work well from Ruby
  • Strip down the weirder and wilder Ruby features (eval, etc) to allow fastest-possible execution
I know how to do all of this.

I've pushed ruboto-irb to Github so you can check it out and play with it. I welcome contributors :)

Ruboto lives!

Update: Good news, everyone!

First, the two bugs I've encountered have both been previously reported and are due to be fixed in an upcoming Android release. They are the enum bug and the reflection bug.

Second, someone going by the handle of "Psycho" reports in the comments that the next version of the Android Scripting Environment (ASE) will include JRuby support! Of course I'm interested in more than just scripting applications with JRuby...I'd like to be able to write applications using only Ruby code, so I'll continue working on this. But JRuby support seems to be coming in from all directions.

Tuesday, July 28, 2009

JVM Language Summit Call for Participation

I should have blogged this sooner, but things have been a little...crazy...lately.


The JVM Languages Summit is coming up for its second year. The event last year was spectacular; representatives of all the major languages and several minor ones showed up and talked about their plans, their history, and their desires from the JVM. And JVM engineers from the three major vendors (Sun, IBM, Oracle) sat there and dutifully took notes. It was a great meeting of minds, and an incredibly uplifting event for those of us invested in the JVM.

It's also not just an event for implementers; if you're keen on the nitty-gritty details of JVM languages and want to help improve them, promote them, or otherwise relate to them in some way, you should be here.

Hope to see you at the Summit!


=== CALL FOR PARTICIPATION -- JVM LANGUAGE SUMMIT, September 2009 ===

http://jvmlangsummit.com/

Dear colleague;

We are pleased to announce the the 2009 JVM Language Summit to be held at Sun's Santa Clara campus on September 16-18, 2009. Registration is now open for speaker submissions (presentations and workshops) and general attendance.

The JVM Language Summit is an open technical collaboration among language designers, compiler writers, tool builders, runtime engineers, and VM architects. We will share our experiences as creators of programming languages for the JVM and of the JVM itself. We also welcome non-JVM developers on similar technologies to attend or speak on their runtime, VM, or language of choice.

The format this year will be slightly different from last year. Attendees told us that the most successful interactions came from small discussion groups rather than prepared lectures, so we've divided the schedule equally between traditional presentations (we're limiting most presentations to 30 minutes this year) and "Workshops". Workshops are informal, facilitated discussion groups among smaller, self-selected participants, and should enable "deeper dives" into the subject matter. There will also be impromptu "lightning talks".

We encourage speakers to submit both a presentation and a workshop; we will arrange to schedule the presentation before the workshop, so that the presentation can spark people's interest and the workshop will allow those who are really interested to go deeper into the subject area. Workshop facilitators may, but are not expected to, prepare presentation materials, but they should come prepared to guide a deep technical discussion.

The Summit is being organized by the JVM Engineering team; no managers or marketers involved! So bring your slide rules and be prepared for some seriously geeky discussions.

The registration page is now open at:

http://registration.jvmlangsummit.com/

If you have any questions, send inquiries to inquire@jvmlangsummit.com.

We hope to see you in September!

Monday, July 27, 2009

JRuby's Importance to Ruby, and eRubyCon 2009

I'm going to be speaking about JRuby again this year at eRubyCon, in Columbus OH. I just got back from Rails Underground, which reminded me how much I love the smaller regional Ruby conferences. So I'm totally pumped to go to eRubyCon this year.


The idea of "Enterprise Ruby" has become less repellant since Dave Thomas's infamous keynote at RalsConf 2006. There are a lot of large, lumbering organizations out there that have yet to adopt any of the newer agile language/framework combinations, and Rails has most definitely led the way. I personally believe that in order for Ruby to become more than just a nice language with a great community, it needs to gain adoption in those organizations, and it needs to do it damn quickly. JRuby is by far the best way for that to happen.

There's another aspect to adoption I think has escaped a lot of Rubyists. In 2006 and 2007, Ruby gained a lot of Java developers who were running away from bloated, over-complicated frameworks and the verbosity and inelegance of Java. When I asked at Ruby conferences in 2005, 2006, and 2007 how many people had done Java development in a former life, almost everyone in the room raised their hands. When I've asked the same question in 2008 and 2009, it's down to less than half the room. Where did they go?

The truth is that the Java platform now has reasonably good answers to Ruby in Groovy, Scala, and Clojure, and reasonably good answers to Rails in Grails and Lift. And yet many Rubyists don't realize how important it is for JRuby to continue doing well, many still seeing it as simply "nice to have" while dismissing the entirety of the Java platform as unimportant to Ruby's future. It's an absurd position, but I blame myself for not making this case sooner.

I believe that JRuby is the most crucial technology for Ruby's future right now. Regardless of how fast or how solid the C or C++ based Ruby implementations get, the vast majority of large organizations are *never* going to run them. That's the truth. If we can leverage JRuby to grab 1-2% of the Java market, we'll *double* the size of the Ruby community. If we completely lose the Java platform to alternatives, Rubyists may not have the luxury of remaining Rubyists in the future. It's that big a deal.

So I hope you'll come by eRubyCon and hear what we've been working on in JRuby and what we have planned for the future, especially our work on making JRuby a stronger JVM citizen. I'm certain to expand on the Hibernate-based prototype code I showed at Rails Underground, and hope to have some additional, never-before-seen demonstrations that will shock and amaze you. And if there's time, I'll demonstrate my two research pets, the "Ruby Mutant" twins Duby and Juby.

See you there!

Sunday, May 31, 2009

Your JavaOne 2009 Ruby Guide

Here's a list of talks about Ruby or that mention/relate to Ruby at CommunityOne and JavaOne 2009. Some of these are about other languages, since I just did a dumb search for any mention of "ruby".


Add your suggestions in comments to help narrow down which talks people should go see.

CommunityOne:

S304128

Developing on the OpenSolaris™Operating System

David Miner, Sun Microsystems; Nicholas Solter, Sun Microsystems

Monday

June 01

10:50 AM - 11:40 AM

Esplanade 305

S307894

Sun GlassFish™ Portfolio: Where Sun's Application Platform Is Going

Eduardo Pelegri-Llopart, Sun Microsystems, Inc.

Monday

June 01

10:50 AM - 11:40 AM

Hall E 135

S304001

Pragmatic Identity 2.0: Invoking Identity Services with a Simplified REST/ROA Architecture

Pat Patterson, Sun Microsystems, Inc.; Daniel Raskin, Sun Microsystems, Inc.; Ron Ten-Hove, Sun Microsystems, Inc.

Monday

June 01

11:50 AM - 12:40 PM

Gateway 102-103

S304141

Programming Languages and the Cloud

Ted Leung, Sun Microsystems, Inc.

Monday

June 01

11:50 AM - 12:40 PM

Gateway 104

S304267

Beyond Impossible: How JRuby Has Evolved the Java™ Platform

Charles Nutter, Sun Microsystems, Inc.

Monday

June 01

1:40 PM - 2:30 PM

Hall E 134

S304040

Social-Enable Your Web Apps with OpenSocial

Dave Johnson, IBM

Monday

June 01

4:00 PM - 4:50 PM

Esplanade 300

S311290

JRuby Rails Workshop

Arun Gupta, Sun Microsystems, Inc.; Jacob Kessler, Sun Microsystems; Vivek Pandey, Sun Microsystems, Inc.; Nick Sieger, Sun Microsystems, Inc

Tuesday

June 02

9:00 AM - 5:00 PM

Breakout Room 7

S311294

Cloud Computing and Storage in Practice

Tim Bray, Sun Microsystems, Inc.; Chris Kutler, Sun Microsystems, Inc.

Wednesday

June 03

1:30 PM - 5:00 PM

Breakout Room 2


JavaOne:

PAN-5348
Script Bowl 2009: A Scripting Languages ShootoutPanel SessionRoberto Chinnici, Sun Microsystems, Inc.; Thomas Enebo, Sun Microsystems, Inc. ; Rich Hickey, Clojure;Guillaume Laforge, SpringSource; Raghavan Srinivas, Self; Dick Wall , Google; Frank Wierzbicki, Sun Microsystems, Inc.Tuesday
June 02
10:50 AM - 11:50 AM
Gateway 104
TS-4164
Clojure: Dynamic Functional Programming for the JVM™ MachineTechnical SessionRich Hickey, ClojureTuesday
June 02
12:10 PM - 1:10 PM
Hall E 133
TS-4487
The Feel of ScalaTechnical SessionBill Venners, Artima, Inc.Tuesday
June 02
3:20 PM - 4:20 PM
Gateway 104
TS-5015
Welcome to RubyTechnical SessionYehuda Katz, Engine YardTuesday
June 02
4:40 PM - 5:40 PM
Gateway 104
TS-5216
Toward a Renaissance VMTechnical SessionBrian Goetz, Sun Microsystems, Inc.; John Rose, Sun MicrosystemsTuesday
June 02
6:00 PM - 7:00 PM
Hall E 133
BOF-4434
Hacking JRubyBOFOla Bini, ThoughtWorksTuesday
June 02
8:30 PM - 9:20 PM
Gateway 104
BOF-5058
JRuby Experiences in the Real WorldBOFLogan Barnett, Happy Camper Studios; David Koontz, JumpBoxTuesday
June 02
9:30 PM - 10:20 PM
Gateway 104
TS-5413
JRuby on Rails in Production: Lessons Learned from Operating a Live, Real-World SiteTechnical SessionNick Sieger, Sun Microsystems, IncWednesday
June 03
11:05 AM - 12:05 PM
Gateway 104
TS-4921
Dynamic Languages Powered by GlassFish™ Application Server v3Technical SessionJacob Kessler, Sun Microsystems; Vivek Pandey, Sun Microsystems, Inc.Wednesday
June 03
11:05 AM - 12:05 PM
Hall E 133
TS-4955
Comparing Groovy and JRubyTechnical SessionNeal Ford, ThoughtWorks Inc.Wednesday
June 03
2:50 PM - 3:50 PM
Gateway 104
BOF-4682
Performance Comparisons of Dynamic Languages on the Java™ Virtual MachineBOFMichael Galpin, eBayWednesday
June 03
6:45 PM - 7:35 PM
Esplanade 300
TS-5385
Alternative Languages on the JVM™ MachineTechnical SessionCliff Click, Azul SystemsThursday
June 04
9:30 AM - 10:30 AM
Gateway 104
TS-4012
Pragmatic Identity 2.0: Simple, Open, Identity Services Using RESTTechnical SessionPat Patterson, Sun Microsystems, Inc.; Ron Ten-Hove, Sun Microsystems, Inc.Thursday
June 04
10:50 AM - 11:50 AM
Esplanade 307-310
TS-4961
"Design Patterns" for Dynamic Languages on the JVM™ MachineTechnical SessionNeal Ford, ThoughtWorks Inc.Thursday
June 04
10:50 AM - 11:50 AM
Gateway 104
TS-5354
Exploiting Concurrency with Dynamic LanguagesTechnical SessionTobias Ivarsson, Neo TechnologyThursday
June 04
1:30 PM - 2:30 PM
Gateway 104
TS-5033
Scripting Java™ Technology with JRubyTechnical SessionThomas Enebo, Sun Microsystems, Inc. ; Charles Nutter, Sun Microsystems, Inc.Thursday
June 04
2:50 PM - 3:50 PM
Gateway 104
TS-3955
Monkeybars: Tools-Enabled Swing Development with JRubyTechnical SessionLogan Barnett, Happy Camper Studios; David Koontz, JumpBoxFriday
June 05
12:10 PM - 1:10 PM
Esplanade 302

Thursday, May 14, 2009

BiteScript 0.0.2 Scripting Examples

I just released BiteScript 0.0.2, which mainly fixes some issues defining packages and non-public classes.

BiteScript is basically just a simple DSL for generating JVM bytecode. I use it in Duby and now in the "ruby2java" compiler we'll be using to turn Ruby classes into Java classes.

I've blogged about BiteScript here before, but I realized today I never posted any simple "hello world" examples. So here's a few of them, all using the command-line "scripting" mode.

First, the simplest version:

main do
ldc "Hello, world!"
aprintln
returnvoid
end

Obviously this is using a predefined "aprintln" macro, since there's no "aprintln" opcode on the JVM. Here's a longer version that shows how a macro would be defined, and accepts one argument
import java.lang.System
import java.io.PrintStream

macro :aprintln do
getstatic System, :out, PrintStream
swap
invokevirtual PrintStream, println, [Object]
end

macro :aprint do
getstatic System, :out, PrintStream
swap
invokevirtual PrintStream, print, [Object]
end

main do
ldc "Hello, "
aprint
aload 0
aaload 0
aprintln
returnvoid
end

And of course this is just Ruby code, so you can just use Ruby to alter the generation of code:
main do
5.times do
ldc "Wow!"
aprintln
end
returnvoid
end

These "BiteScripts" can all be either run with the "bite" command or compiled with the "bitec" command:
$ bite examples/using_ruby.bs 
Wow!
Wow!
Wow!
Wow!
Wow!

$ bitec examples/using_ruby.bs

$ javap -c examples/using_ruby
Compiled from "examples.using_ruby.bs"
public class examples.using_ruby extends java.lang.Object{
public static void main(java.lang.String[]);
Code:
0: ldc #9; //String Wow!
2: getstatic #15; //Field java/lang/System.out:Ljava/io/PrintStream;
5: swap
6: invokevirtual #21; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
9: ldc #9; //String Wow!
11: getstatic #15; //Field java/lang/System.out:Ljava/io/PrintStream;
14: swap
15: invokevirtual #21; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
18: ldc #9; //String Wow!
20: getstatic #15; //Field java/lang/System.out:Ljava/io/PrintStream;
23: swap
24: invokevirtual #21; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
27: ldc #9; //String Wow!
29: getstatic #15; //Field java/lang/System.out:Ljava/io/PrintStream;
32: swap
33: invokevirtual #21; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
36: ldc #9; //String Wow!
38: getstatic #15; //Field java/lang/System.out:Ljava/io/PrintStream;
41: swap
42: invokevirtual #21; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
45: return

}

This last example shows the resulting JVM bytecode as well.

Future plans for BiteScript include making it have better error detection (right now it just falls back on the JVM bytecode verifier, which is not the most descriptive thing in the world) and improving the API to more easily handle all the various combinations of class, field, and method modifiers. I'd also like to make it detect if you're doing bad things to the stack to save you the hassle of interpreting verification errors that may not happen until runtime.

Anyway, give it a try and feel free to contribute; the code is all Ruby, wrapping the ASM bytecode library, so anyone that knows Ruby can tweak it. The project page and wiki are hosted at Kenai.com: http://kenai.com/projects/jvmscript

And if you're not up on the JVM or JVM bytecodes, the JVM Specification is an easy-to-read complete reference for code targeting the JVM, and here is my favorite JVM opcode quickref.

Help JRuby by Fixing RubySpecs

A number of you have asked how you can help JRuby development. Well there's actually an easy way: fix RubySpec failures.

You may have noticed we periodically update our RubySpec "stable" revision number, and usually have to file a few bugs. This isn't because we don't want to fix those issues...on the contrary, we would love to fix them. We just don't have enough manpower, and there's usually harder issues we need to tackle first.

But most of the failures are easy to fix, and a lot of JRuby newcomers have gotten their feet wet fixing them. So here's a short guide on how to run the specs and fix them quickly.

1. Get a JRuby working copy and build it

This is as simple as 'git clone git://github.com/jruby/jruby.git', then 'cd jruby' and 'ant'. You'll need Apache Ant 1.7 and Java 5/1.5 or higher (grab "Java SE Development Kit" from the Java SE downloads page.

2. Run the CI spec run

We have a clean spec run that should be clean for you before you start. Just run "ant spec-short" and it will pull the mspec and rubyspec repositories, roll them to the stable versions, and run all known good specs. Now you're ready to investigate specific failures.

3. Run specific spec files with bugs reported

You can look in Jira under the "RubySpec" category, or look under spec/tags/ruby for "tag" files listing failing specs and Jira bug numbers. Once you find something you'd like to investigate, run that spec file using "bin/jruby spec/mspec/bin/mspec <path/to/spec/file>". For example to set the Range#initialize failures I just reported, run "bin/jruby spec/mspec/bin/mspec spec/ruby/core/range/initialize_spec.rb".

Now you can proceed to fixing it.

4. Identify where the problem is.

Most of the core classes are pretty easy to locate. Any classes in the "core" specs will have a Java class named Ruby, like RubyArray, RubyRange, and so on. They're generally located in src/org/jruby. If you know any Java, these files are pretty easy to follow, and we're standing by on IRC or on the mailing list to hold your hand at the start.

5. Create a patch and submit it to the bug

Once you have a working fix, you can go ahead create a patch, either with "git diff > somefile.patch" or by committing it to your local repository and using "git format-patch -1" to create a formatted patch for the topmost commit. Some git-fu may be necessary, so I usually just use "git diff".

That's all there is to it! You'll be a JRuby contributor in no time!

fork and exec on the JVM? JRuby to the Rescue!

Today David R. MacIver pinged me in #scala and asked "headius: Presumably you guys have spent quite a lot of time trying to make things like system("vim") work correctly in JRuby and failing? i.e. I'm probably wasting my time to attempt similar?"

My first answer was "yes", since there's no direct way to exec a program like vim (which wants a real terminal) and have it work on the JVM. The JVM's process launching gives the newly-spawned processes the child side of piped streams, which you then have to manually pump (which is what we do in JRuby's system, backtick, and exec methods). Under these circumstances, vim may start up, but it's certainly not functional.

But then I got to thinking...if you were doing this in C, you'd fork+exec and all would be happy. But we can't fork+exec on the JVM..OR CAN WE?

As you should know by now, JRuby ships with FFI, a library that allows you to bind any arbitrary C function in Ruby code. So getting fork+exec to work was a simple matter of writing a little Ruby code:

require 'ffi'

module Exec
extend FFI::Library

attach_function :my_exec, :execl, [:string, :string, :varargs], :int
attach_function :fork, [], :int
end

vim1 = '/usr/bin/vim'
vim2 = 'vim'
if Exec.fork == 0
Exec.my_exec vim1, vim2, :pointer, nil
end

Process.waitall

Running that with JRuby (I tried master, David tried 1.3.0RC1, and 1.2.0 works too) brings up a full-screen vim session, just like you'd expect, and it all just works. No other JVM language can do this so quickly and easily.

We'll probably try to generalize this into an optional library JRubyists can load (require 'jruby/real_exec' or similar) and perhaps add fork and exec to jna-posix so that the other JVM languages can have sweet, sweet process launching too.

JRuby rocks.

Update: The biggest problem with using fork+exec in this way is that you can't guarantee *nothing* happens between the fork call and the exec call. If, for example, the JVM decides to GC or move memory around, you can have a fatal crash at the JVM process level. Because of that, I don't recommend using fork + exec via FFI in JRuby, even though it's pretty cool.

However, since this post I've learned of the "posix_spawn" function available on most Unix variants. It's basically fork + exec in a single function, plus most of the typical security and IO tweaks you might do after forking and before execing. It's definitely my recommended alternative to fork+exec for JRuby, and to make that easier I've bundled it up as the "spoon" gem (gem install spoon) which provides spawn and spawnp to JRuby users directly. Here's an example session using Spoon to launch JRuby as a daemon. If you just need fork+exec on the JVM, posix_spawn or the Spoon gem are the best way to do it.

Wednesday, May 13, 2009

JRuby Nailgun Support in 1.3.0

I've merged changes into master (to be 1.3 soon) that should make Nailgun easier to use. And 1.3 will be the first release to include all NG stuff in the binary dist.

  • jruby --ng-server starts up a server. You can manage it however you like
  • jruby --ng uses the Nailgun client instead of launching a new JVM for the command you run. You'll just need to run make in tool/nailgun to build the ng executable (already built for you on Windows).
Future improvements will include having --ng start up the server for you if you haven't started it, passing signals through (don't expect signals to work at all right now), better management of threads and commands, and so on. But it's a good start, and people can try to play with it and report issues more easily now.

Heres a sample session:
~/projects/jruby ➔ cd tool/nailgun/ ; make ; cd -
Building ng client. To build a Windows binary, type 'make ng.exe'
gcc -Wall -pedantic -s -O3 -o ng src/c/ng.c
ld warning: option -s is obsolete and being ignored
/Users/headius/projects/jruby

~/projects/jruby ➔ jruby --ng-server
NGServer started on all interfaces, port 2113.
^Z
[1]+ Stopped jruby --ng-server

~/projects/jruby ➔ bg
[1]+ jruby --ng-server &

~/projects/jruby ➔ jruby --ng -e "puts 1"
1

~/projects/jruby ➔ time jruby -e "puts 1"
1

real 0m0.609s
user 0m0.482s
sys 0m0.119s

~/projects/jruby ➔ time jruby --ng -e "puts 1"
1

real 0m0.073s
user 0m0.010s
sys 0m0.018s

Update: For those not familiar, "NailGun is a client, protocol, and server for running Java programs from the command line without incurring the JVM startup overhead. Programs run in the server (implemented in java), triggered by the client (written in C), which handles all I/O."

Update 2: It's been brought to my attention that we no longer ship a pre-configured Makefile for the ng client, so your build command line should actually look like:
~/projects/jruby ➔ cd tool/nailgun/ ; ./configure ; make ; cd -
Alternatively, you can run "ant build-ng" in the JRuby root, which will do largely the same thing for you using Ant.

Thursday, April 30, 2009

Stand and Be Counted

In the spirit of Nick Sieger's short statement on the recent uproar over Matt Aimonetti's "pr0n star" talk, I'm posting my one and only blog response to the whole thing.

Unlike Nick, I too often have used this blog as a soapbox. And too often I've ground my personal axe against projects that may or may not have deserved it. I'm human, I'm passionate and proud of my work, and I'm defensive of what we've accomplished, so I don't think this is surprising. I also see the same passion and pride in the Ruby community at large, and it's why I'm much more interested in attending Ruby conferences than Java conferences, where many attendees just seem to be going through the motions. And I know I've crossed a line at times, making or taking things too personal, and hopefully I've apologized or corrected myself whenever that's happened. If not, mea culpa.

But there's a disturbing trend in the Ruby community I haven't had to deal with since high school: in preference to open inclusion, more and more Rubyists seem to choose exclusivity.

This recent firestorm has continued in large part, I believe, because of the poor initial response by folks involved. Rather than recognize that there are people with different views, taking offense at different ideas and images, some decided to say "fuck you, this is who I am" and further alienate those people. I certainly don't expect we as passionate individuals won't commit occasional faux pas, especially when trying to be funny or provocative and especially when coming from different backgrounds that may be more or less accepting of certain behaviors. But to claim no responsibility for an obvious mistake, indeed to claim it's somehow the fault of the offended, or American sensibility, or political correctness...well that's just sophomoric.

I think to some extent we can understand (but not excuse) such behavior by realizing that the Ruby (or perhaps the Rails) community is largely a very *young* community. That's a large part of why this community is so passionate, why they're so committed to their ideals, why they're so opinionated, why they're so much more fun to hang out with than many 30-year programmers from other communities who've had the life sucked out of them. It's also a reason so many in the Ruby (or perhaps the Rails) community seem to act like they're in high school, forming cliques, sitting at their own tables, snubbing the new kids or the weird kids or anyone they perceive as "trying to be cool."

Have you been invited to any exclusive Ruby communities? I've been invited to a couple, and without exception I've found the idea offensive every time. In some cities, there are now multiple tiers of Ruby group: one for the proles, where anyone is welcome and everyone is either new to Ruby, a little weird, or both; and then perhaps one or two levels of more "exclusive" groups, usually more "advanced" and sometimes invite-only but generally exclusionary in some way.

There's also a technical "coolness" exclusivity many projects have had to cope with. Folks working on JRuby and IronRuby, for example, have had to deal with perceptions that they're either less "cool" because of their platform of choice or at least somehow less "Ruby" because they're not following the same golden path everyone else follows. Or perhaps their employers are out to take over Ruby, or they're going to infect Ruby with a bunch more "new" people who don't "get it". All the while the folks that use and work on these projects are working just as hard as anyone else to bring Ruby to the world, staying true to what makes Ruby special, and largely going against the grain in their original communities as well. Being snubbed, mocked, or attacked is often their reward.

You start to see a pattern here, yes?

So let's spell it out. I like the Ruby community because it's filled with people who love playing with new technology, without biases and prejudices getting in the way. My closest friends in the community are people like me, who find it repugnant that being opinionated has been too often equated with being rude and boorish, exclusionary and sophomoric, or simply mean. We are all here because of our love of technology, all here because we didn't feel like we fit in other places that weren't so passionate about beautiful code and fresh ideas. We are all here because we don't care if you're male or female, religious or irreligious, young or old, experienced or inexperienced, beautiful or plain, conservative or liberal, tall or short, fat or thin, foreign or domestic, gay or straight, black or white, or any grey areas in-between. We are all here because we love that more and more people like us join the community every day...the same people some of us immediately judge and box into their own subcool subgroups.

I don't want to join your damn clique. I don't think it's ok to set people aside or treat them like dirt because they don't believe what you believe or because they have their own way of thinking and acting or because they're not as worldly and mature and oh-so-smug as you are. I don't believe in "rock stars" and I don't believe that dubious title gives anyone the right to be an asshole to others or to have free reign to act any way they choose. I don't care what kind of car you drive, what house you live in, or what clothes you wear...and I sure as hell don't care how many people follow you on Twitter.

What I do care about is whether you're interested in sitting down and hacking out some code, looking at new projects with an open mind, helping someone new (maybe me) improve their skills, being part of something larger than yourself. If you promise not to treat me like a weirdo or a rock star, I promise to talk openly about your ideas, to show you the heart and soul of my code, and to freely share my thoughts...no matter who you are. I hope you'll attend my presentations and/or try out my projects, and in exchange I'll try to do the same the same for you. I hope you'll walk up to me at conferences and tell me about whatever "crazy" or "stupid" idea you have, and I guarantee to listen since it's probably not as crazy or stupid as you think. And I expect you to do the same for everyone else in the community and not treat me or anyone else any differently.

Now, let's move forward and get back to hacking and having fun!

Saturday, April 25, 2009

Setting up Typo on JRuby

I figured I'd give Typo a try on JRuby today. It has been working for quite a while, but with the GlassFish gem improving so much I thought it would be good to write up an updated walkthrough. It's pretty simple.

BTW, is Typo still the preeminent blog package for Rails? I certainly don't want to be out of fashion.

Prerequisites:

  1. MySQL already set up and working, with TCP sockets enabled (or I guess you can use SQLite too)
  2. Java (sudo apt-get install sun-java6-jdk or whatever's appropriate for your platform)
  3. JRuby (download, unpack, put bin/ in PATH)
  4. Appropriate gems installed (rails, activerecord-jdbcmysql-adapter, glassfish or mongrel)
The process:
  1. Download Typo. The gem unfortunately tries to install native extensions like sqlite3 and mysql (I sure wish they wouldn't do that!)
  2. Unpack the Typo zip wherever you want your blog site to live and cd into that directory
  3. Edit config/database.yml.example to your liking, replacing "mysql" with "jdbcmysql" and save it as config/database.yml
  4. Create the database:
    jruby -S rake db:create RAILS_ENV=production
  5. Migrate the database:
    jruby -S rake db:migrate RAILS_ENV=production
  6. Run the server:
    glassfish -p <port> -e production [and whatever other options you want]
    or
    jruby script/server -p <port> -e production
  7. Set up Apache to point at your new Typo instance (optional)
That's all there is to it! You'll want to be the first one to hit your new blog, so you can set up your admin account and lock down the server.

Perhaps it's time I finally moved my blog off Blogger and on to a JRuby-hosted server, eh?

Suggestions, improvements to this process? Add to comments and I'll update the post.

Apache + JRuby + Rails + GlassFish = Easy Deployment!

It occurred to me today that a lot of people probably want a JRuby deployment option that works with a front-end web server. I present for you the trivial steps required to host a JRuby server behind Apache.


Update: It's worth mentioning that this works fine with JRuby + Mongrel too, though Mongrel doesn't automatically multithread without this patch and Rails' production.rb config.threadsafe! line uncommented. The GlassFish gem will automatically multithread with several JRuby instances (in the same server process) by default or a single JRuby instance with config.threadsafe! uncommented.

Prerequisites:
  1. Apache with mod_proxy_http enabled (sudo a2enmod proxy_http on Ubuntu)
  2. Java (sudo apt-get install sun-java6-jdk or the openjdk flavors if you like)
  3. JRuby (download, unpack, put bin/ in PATH)
  4. gems appropriate to run your app with JRuby (e.g. rails, activerecord-jdbcsqlite3-adapter, etc)
  5. production DB all set to go
Ok. I'm no Apache expert, so I'm sure there's some tweaking necessary for this. Please add your tweaks and suggestions in comments. But basically, all you need to do is run your app using the GlassFish gem and set up Apache to proxy to it. It's that simple.
  1. Install the glassfish gem
    gem install glassfish
  2. From your application directory, run glassfish with these options:
    glassfish -p <port> -e production -c <context> -d
  3. Add ProxyPass and ProxyPassReverse lines to Apache (whereever is appropriate on your system) for the GlassFish server instance. For example, if <port> is 9000 and <context> is foo:
    ProxyPass /foo http://localhost:9000/foo
    ProxyPassReverse /foo http://localhost:9000/foo
  4. Reload Apache config, however is appropriate for your system
You'll now be able to access your app via http://servername/foo, and requests will all proxy to the GlassFish server instance.

This doesn't do anything to manage the server instance, but since GlassFish can start up as a daemon now, it should be easy to wire into whatever you normally use for that.

A few caveats:
  • I had some trouble getting mod_proxy to allow requests to proxy through. There's probably a right way I'm not doing, so I won't say what I did. If you know the ideal mod_proxy config for this sort of thing, post it in comments
  • Although the GlassFish gem is really nice already, we're still working minor kinks out of it and it may still have minor bugs. If you run into something, let us know and we'll get it fixed (and of course, you can use this mod_proxy setup with Mongrel too, if you like).
  • I'd love to get some help putting together something that manages servers similar to Phusion Passenger, because that's really the only missing piece here.
I'll update this post as suggestions come in. Enjoy!

Update: Dberg suggests the following improvement:
the one thing you want to remember to do also is to exclude the static assets that normally get served out of rails. To do this simply add some proxypass exclude lines like

ProxyPass /images !
ProxyPass /javascripts !
ProxyPass /stylesheets

Then make sure DocumentRoot is set to the right place for these files and you will get a slight performance boost !
Thanks for that!

Wednesday, April 22, 2009

The Future: Part One

There's been a lot of supposition about the future lately, and I've certainly read and succumbed to the temptation of the prognosticator. I'm not going to comment on any of the players or put forward my own suppositions about what might happen. What I will do here is talk about what *should* happen.

It's apparent that the Java platform is at a crossroads. One path leads to irrelevance, be it through general apathy that important technologies are getting sidelined, or through active emigration due to bureaucratic processes and waterfall platform evolution. The other path leads to a bright, open future, where polyglots romp and play with fresh new languages and developers have freedom to use whatever tools they feel are necessary for a given job. Given the large investment many of us have in this platform, we need to start talking now about which direction we want to go.

I've been doing a lot of thinking about the future of the Java platform and my future developing for it, and I've come up with a few things I believe must happen soon to ensure the future of the JVM and the applications and languages built for it.

Languages

As you might expect from me, the first area involves supporting many languages on the JVM. Over the past three years we've seen a spectacular transformation take place. The "Java" VM has become a truly multi-language platform. People are putting into production applications running all sorts of languages, often writing absolutely no Java to do so. And this is how things must be...this is how the platform is going to survive.

As I see it there are currently three primary players in the language domain: JRuby, Scala, and Groovy (in no particular order). I include these three due to the relative completeness of their implementations, vague popularity metrics, and availability of production use cases. These three languages, one static and two dynamic, bear further exploration.

JRuby represents the promise of off-platform languages being brought to the JVM. Rather than creating a language specifically tailored to fit well into the JVM's type system and limitations, we have managed to take a drastically different language and implement enough of it to be the only "alternative" implementation realistically considered for production applications. And in doing so we've had to stretch the platform. We've created native binding libraries, wired in POSIX functions the JDK doesn't provide, implemented our own core types like Strings, Arrays, and regular expressions, and done all this while managing to deliver the best performing compatible Ruby implementation available. And where the other two primary language contenders largely pull developers from other areas of the Java platform, JRuby actually brings in many developers that might not otherwise ever use the JVM. JRuby stretches and grows the platform.

Groovy is the second dynamic language under consideration. Groovy represents taking the best features of a number of dynamic languages and wiring them into a Java-like syntax that's easy for existing Java developers to learn. Groovy provides a solid dynamic language foundation without breaking Java type-system requirements like ahead-of-time compiled classes and static method signatures, allowing it to enlist directly in newer APIs that depend on those requirements. And while many developers come to Groovy from other areas of the Java platform, they might also have completely left the platform if not for Groovy. Groovy provides for Java developers a "dynamic layer" that doesn't require them to learn a new platform and a new set of libraries. And so Groovy's strength is in maintaining the platform and stemming the bleeding of developers to off-platform dynamic languages.

Scala, it must be stated, is the current heir apparent to the Java throne. No other language on the JVM seems as capable of being a "replacement for Java" as Scala, and the momentum behind Scala is now unquestionable. While Scala is not a dynamic language, it has many of the characteristics of popular dynamic languages, through its rich and flexible type system, its sparse and clean syntax, and its marriage of functional and object paradigms. The supposed weakness of Scala as being "too complicated" or "too rich" can be countered by developing coding standards, creating more robust editors and tools, and by better educating polyglots on how best to use Scala. Scala represents the rebirth of static-typed languages on the JVM, and like JRuby it has also begun to stretch the capabilities of the platform in ways Java never could.

Among the secondary players I include language implementations like Jython, Clojure, and Rhino. While still in early days of adoption (or in Jython and Rhino's cases, early days of re-awakening), they represent similar aspects to the primary three languages. For purposes of discussion, we'll leave it at that for now.

In order for the platform to embrace these and many future languages, several things need to happen:

  • Ongoing work on these languages must be funded in such a way as to avoid product lock-in. Funding them by tying them to specific technologies will only fracture development communities, and likely damage any existing open-source contributor base.
  • There must be paid support offerings for these languages outside of specific products. For people interested in running JRuby on Rails, Grails, or Liftweb, there must be support channels they can follow, regardless of product bundling attempts. In the JRuby world, for example, we receive many requests for paid support, either in the form of hired guns to optimize an application or through targeting resources to critical bugs. And so far, we have been unable to offer such support.
  • We must continue the work started on the OpenJDK and MLVM projects. OpenJDK has enabled such projects as Shark (Hotspot on LLVM) and the MLVM (JVM extensions for non-Java language features), and we must see these efforts through to completion.
  • Finally, we need a "languages czar" that can coordinate the technical aspects of these various projects and help direct resources where needed. This would largely be a community-cultivating role, but as a sort of "open-source manager" to ensure development efforts are both cooperating and not being needlessly duplicated.
I believe it's absolutely vital that these tasks be met, or we risk the future of the platform entirely. And losing must not be an option, lest we fall back into proprietary alternatives controlled by a single entity.

More to come.

Monday, April 13, 2009

JRuby Moves to Git

We have successfully migrated JRuby development to Git!

The main repository is on the JRuby kenai.com project, but most folks will just want to clone the official mirror on Github. The mirror lags by no more than five minutes, and we'll eliminate that lag soon.

Kenai: git://kenai.com/jruby~main
Github: git://github.com/jruby/jruby.git

The Github repository is also attached to the official Github "jruby" user.

The CI server has already been updated to point at the new repository on kenai, and nightly builds have been running for several days. If you just want to grab a current nightly snapshot, you can do that from github's capabilities or by visiting the nightly build page:

http://jruby.headius.com:8080/hudson/job/jruby-dist/

The old SVN repository on Codehaus is now defunct for JRuby development and will not be updated any more. We will likely remove the JRuby branches and trunk soon and replace them with a README pointing to the new repository locations.

Why Git?

We've known for a long time that we wanted to move to a distributed SCM, and had narrowed it down to Mercurial and Git.

For a long time Mercurial was the front-runner, partly because we were more familiar with it and partly because kenai.com, the site where we're moving JRuby's project hosting (and a JRuby on Rails-based site, btw), only supported Subversion and Mercurial.

But a few things changed our minds over the past couple months:
  • Kenai added git support
  • We realized that we'd get more Rubyists contributing if we had a mirror on Github
  • We became more comfortable with Git
Ultimately, the move to Git mostly came down to politics: Rubyists like Git better, and we're a Ruby-related project. Had we been Jython, we'd probably have chosen Mercurial.

Enjoy!

Thursday, April 02, 2009

How JRuby Makes Ruby Fast

At least once a year there's a maelstrom of posts about a new Ruby implementation with stellar numbers. These numbers are usually based on very early experimental code, and they are rarely accompanied by information on compatibility. And of course we love to see crazy performance numbers, so many of us eat this stuff up.

Posting numbers too early is a real disservice to any project, since they almost certainly don't represent the eventual real-world performance people will see. It encourages folks to look to the future, but it also marginalizes implementations that already provide both compatibility and performance, and ignores how much work it has taken to get there. Given how much we like to see numbers, and how thirsty the Ruby community is for "a fastest Ruby", I don't know whether this will ever change.

I thought perhaps a discussion about the process of optimizing JRuby might help folks understand what's involved in building a fast, compatible Ruby implementation, so that these periodic shootouts don't get blown out of proportion. Ruby can be fast, certainly even faster than JRuby is today. But getting there while maintaining compatibility is very difficult.

Performance Optimization, JRuby-style

The truth is it's actually very easy to make small snippits of Ruby code run really fast, especially if you optimize for the benchmark. But is it useful to do so? And can we extrapolate eventual production performance from these early numbers?

We begin our exploration by running JRuby in interpreted mode, which is the slowest way you can run JRuby. We'll be using the "tak" benchmark, since it's simple and easy to demonstrate relative performance at each optimization level.
# Takeuchi function performance, tak(24, 16, 8)
def tak x, y, z
if y >= x
return z
else
return tak( tak(x-1, y, z),
tak(y-1, z, x),
tak(z-1, x, y))
end
end

require "benchmark"

N = (ARGV.shift || 1).to_i

Benchmark.bm do |make|
N.times do
make.report do
i = 0
while i<10
tak(24, 16, 8)
i+=1
end
end
end
end

And here's our first set of results. I have provided Ruby 1.8.6 and Ruby 1.9.1 numbers for comparison.
Ruby 1.8.6p114:
➔ ruby bench/bench_tak.rb 5
user system total real
17.150000 0.120000 17.270000 ( 17.585128)
17.170000 0.140000 17.310000 ( 17.946869)
17.180000 0.160000 17.340000 ( 18.234570)
17.180000 0.150000 17.330000 ( 17.779536)
18.790000 0.190000 18.980000 ( 19.560232)

Ruby 1.9.1p0:
➔ ruby191 bench/bench_tak.rb 5
user system total real
3.570000 0.030000 3.600000 ( 3.614855)
3.570000 0.030000 3.600000 ( 3.615341)
3.560000 0.020000 3.580000 ( 3.608843)
3.570000 0.020000 3.590000 ( 3.591833)
3.570000 0.020000 3.590000 ( 3.640205)

JRuby 1.3.0-dev, interpreted, client VM
➔ jruby -X-C bench/bench_tak.rb 5
user system total real
24.981000 0.000000 24.981000 ( 24.903000)
24.632000 0.000000 24.632000 ( 24.633000)
25.459000 0.000000 25.459000 ( 25.459000)
29.122000 0.000000 29.122000 ( 29.122000)
29.935000 0.000000 29.935000 ( 29.935000)

Ruby 1.9 posts some nice numbers here, and JRuby shows how slow it can be when doing no optimizations at all. The first change we look at, and which we recommend to any users seeking best-possible performance out of JRuby, is to use the JVM's "server" mode, which optimizes considerably better.
JRuby 1.3.0-dev, interpreted, server VM
➔ jruby --server -X-C bench/bench_tak.rb 5
user system total real
8.262000 0.000000 8.262000 ( 8.192000)
7.789000 0.000000 7.789000 ( 7.789000)
8.012000 0.000000 8.012000 ( 8.012000)
7.998000 0.000000 7.998000 ( 7.998000)
8.000000 0.000000 8.000000 ( 8.000000)

The "server" VM differs from the default "client" VM in that it will optimistically inline code across calls and optimize the resulting code as a single unit. This obviously allows it to eliminate costly x86 CALL operations, but even more than that it allows optimizing algorithms which span multiple calls. By default, OpenJDK will attempt to inline up to 9 levels of calls, so long as they're monomorphic (only one valid target), not too big, and no early assumptions are changed by later code (like if a monomorphic call goes polymorphic later on). In this case, where we're not yet compiling Ruby code to JVM bytecode, this inlining is mostly helping JRuby's interpreter, core classes, and method-call logic. But already we're 3x faster than interpreted JRuby on the client VM.

The next optmization will be to turn on the compiler. I've modified JRuby for the next couple runs to *only* compile and not do any additional optimizations. We'll discuss those optimizations as I add them back.
JRuby 1.3.0-dev, compiled (unoptimized), server VM:
➔ jruby --server -J-Djruby.astInspector.enabled=false bench/bench_tak.rb 5
user system total real
5.436000 0.000000 5.436000 ( 5.376000)
3.655000 0.000000 3.655000 ( 3.655000)
3.662000 0.000000 3.662000 ( 3.662000)
3.683000 0.000000 3.683000 ( 3.683000)
3.668000 0.000000 3.668000 ( 3.668000)

By compiling, without doing any additional optimizations, we're able to improve performance 2x again. Because we're now JITing Ruby code as JVM bytecode, and the JVM eventually JITs JVM bytecode to native code, our Ruby code actually starts to benefit from the JVM's built-in optimizations. We're making better use of the system CPU and not making nearly as many calls as we would from the interpreter (since the interpreter is basically a long chain of calls for each low-level Ruby operation.

Next, we'll turn on the simplest and oldest JRuby compiler optimization, "heap scope elimination".
JRuby 1.3.0-dev, compiled (heap scope optz), server VM:
➔ jruby --server bench/bench_tak.rb 5
user system total real
4.014000 0.000000 4.014000 ( 3.942000)
2.776000 0.000000 2.776000 ( 2.776000)
2.760000 0.000000 2.760000 ( 2.760000)
2.769000 0.000000 2.769000 ( 2.769000)
2.768000 0.000000 2.768000 ( 2.769000)

The "heap scope elimination" optimization eliminates the use of an in-memory store for local variables. Instead, when there's no need for local variables to be accessible outside the context of a given method, they are compiled as Java local variables. This allows the JVM to put them into CPU registers, making them considerably faster than reading or writing them from/to main memory (via a cache, but still slower than registers). This also makes JRuby ease up on the JVM's memory heap, since it no longer has to allocate memory for those scopes on every single call. This now puts us comfortably faster than Ruby 1.9, and it represents the set of optimizations you see in JRuby 1.2.0.

Is this the best we can do? No, we can certainly do more, and some such experimental optimizations are actually already underway. Let's continue our exploration by turning on another optimization similar to the previous one: "backtrace-only frames".
JRuby 1.3.0-dev, compiled (heap scope + bracktrace frame optz), server VM:
➔ jruby --server -J-Djruby.compile.frameless=true bench/bench_tak.rb 5
user system total real
3.609000 0.000000 3.609000 ( 3.526000)
2.600000 0.000000 2.600000 ( 2.600000)
2.602000 0.000000 2.602000 ( 2.602000)
2.598000 0.000000 2.598000 ( 2.598000)
2.602000 0.000000 2.602000 ( 2.602000)

Every Ruby call needs to store information above and beyond local variables. There's the current "self", the current method visibility (used for defining new methods), which class is currently the "current" one, backref and lastline values ($~ and $_), backtrace information (caller's file and line), and some other miscellany for handling long jumps (like return or break in a block). In most cases, this information is not used, and so storing it and pushing/popping it for every call wastes precious time. In fact, other than backtrace information (which needs to be present to provide Ruby-like backtrace output), we can turn most of the frame data off. This is where we start to break Ruby a bit, though there are ways around it. But you can see we get another small boost.

What if we eliminate frames entirely and just use the JVM's built-in backtrace logic? It turns out that having any pushing/popping of frames, even with only backtrace data, still costs us quite a bit of performance. So let's try "heap frame elimination":
JRuby 1.3.0-dev, compiled (heap scope + heap frame optz), server VM:
➔ jruby --server -J-Djruby.compile.frameless=true bench/bench_tak.rb 5
user system total real
2.955000 0.000000 2.955000 ( 2.890000)
1.904000 0.000000 1.904000 ( 1.904000)
1.843000 0.000000 1.843000 ( 1.843000)
1.823000 0.000000 1.823000 ( 1.823000)
1.813000 0.000000 1.813000 ( 1.813000)

By eliminating frames entirely, we're a good 33% faster than the fastest "fully framed" run you'd get with stock JRuby 1.2.0. You'll notice the command line here is the same; that's because we're venturing into more and more experimental code, and in this case I've actually forced "frameless" to be "no heap frame" instead of "backtrace-only heap frame". And what do we lose with this change? We no longer would be able to produce a backtrace containing only Ruby calls, so you'd see some JRuby internals in the trace, similar to how Rubinius shows Rubinius internals. But we're getting respectably fast now.

Next up we'll turn on some optimizations for math operators.
JRuby 1.3.0-dev, compiled (heap scope, heap frame, fastops optz), server VM:
➔ jruby --server -J-Djruby.compile.frameless=true -J-Djruby.compile.fastops=true bench/bench_tak.rb 5
user system total real
2.291000 0.000000 2.291000 ( 2.225000)
1.335000 0.000000 1.335000 ( 1.335000)
1.337000 0.000000 1.337000 ( 1.337000)
1.344000 0.000000 1.344000 ( 1.344000)
1.346000 0.000000 1.346000 ( 1.346000)

Most of the time, when calling + or - on an object, we do the full Ruby dynamic dispatch cycle. Dispatch involves retrieving the target object's metaclass, querying for a method (like "+" or "-"), and invoking that method with the appropriate arguments. This works fine for getting us respectable performance, but we want to take things even further. So JRuby has experimental "fast math" operations to turn most Fixnum math operators into static calls rather than dynamic ones, allowing most math operations to inline directly into the caller. And what do we lose? This version of "fast ops" makes it impossible to override Fixnum#+ and friends, since whenever we call + on a Fixnum it's going straight to the code. But it gets us another nearly 30% improvement.

Up to now we've still also been updating a lot of per-thread information. For every line, we're tweaking a per-thread field to say what line number we're on. We're also pinging a set of per-thread fields to handle the unsafe "kill" and "raise" operations on each thread...basically we're checking to see if another thread has asked the current one to die or raise an exception. Let's turn all that off:

JRuby 1.3.0-dev, compiled (heap scope, heap frame, fastops, threadless, positionless optz), server VM:
➔ jruby --server -J-Djruby.compile.frameless=true -J-Djruby.compile.fastops=true -J-Djruby.compile.positionless=true -J-Djruby.compile.threadless=true bench/bench_tak.rb 5
user system total real
2.256000 0.000000 2.256000 ( 2.186000)
1.304000 0.000000 1.304000 ( 1.304000)
1.310000 0.000000 1.310000 ( 1.310000)
1.307000 0.000000 1.307000 ( 1.307000)
1.301000 0.000000 1.301000 ( 1.301000)

We get a small but measurable performance boost from this change as well.

The experimental optimizations up to this point (other than threadless) comprise the set of options for JRuby's --fast option, shipped in 1.2.0. The --fast option additionally tries to statically inspect code to determine whether these optimizations are safe. For example, if you're running with --fast but still access backrefs, we're going to create a frame for you anyway.

We're not done yet. I mentioned earlier the JVM gets some of its best optimizations from its ability to profile and inline code at runtime. Unfortunately in current JRuby, there's no way to inline dynamic calls. There's too much plumbing involved. The upcoming "invokedynamic" work in Java 7 will give us an easier path forward, making dynamic calls as natural to the JVM as static calls, but of course we want to support Java 5 and Java 6 for a long time. So naturally, I have been maintaining an experimental patch that eliminates most of that plumbing and makes dynamic calls inline on Java 5 and Java 6.
JRuby 1.3.0-dev, compiled ("--fast", dyncall optz), server VM:
➔ jruby --server --fast bench/bench_tak.rb 5
user system total real
2.206000 0.000000 2.206000 ( 2.066000)
1.259000 0.000000 1.259000 ( 1.259000)
1.258000 0.000000 1.258000 ( 1.258000)
1.269000 0.000000 1.269000 ( 1.269000)
1.270000 0.000000 1.270000 ( 1.270000)

We improve again by a small amount, always edging the performance bar higher and higher. In this case, we don't lose compatibility, we lose stability. The inlining modification breaks method_missing and friends, since I have not yet modified the call pipeline to support both inlining and method_missing. And there's still a lot of extra overhead here that can be eliminated. But in general we're still mostly Ruby, and even with this change you can run a lot of code.

This represents the current state of JRuby. I've taken you all the way from slow, compatible execution, through fast, compatible execution, and all the way to faster, less-compatible execution. There's certainly a lot more we can do, and we're not yet as fast as some of the incomplete experimental Ruby VMs. But we run Ruby applications, and that's no small feat. We will continue making measured steps, always ensuring compatibility first so each release of JRuby is more stable and more complete than the last. If we don't immediately leap to the top of the performance heap, there's always good reasons for it.

Performance Optimization, Duby-style

As a final illustration, I want to show the tak performance for a language that looks like Ruby, and tastes like Ruby, but boasts substantially better performance: Duby.
def tak(x => :fixnum, y => :fixnum, z => :fixnum)
unless y < x
z
else
tak( tak(x-1, y, z),
tak(y-1, z, x),
tak(z-1, x, y))
end
end

puts "Running tak(24,16,8) 1000 times"

i = 0
while i<1000
tak(24, 16, 8)
i+=1
end

This is the Takeuchi function written in Duby. It looks basically like Ruby, except for the :fixnum type hints in the signature. Here's a timing of the above script (which calls tak the same as before but 1000 times instead of 5 times), running on the server JVM:
➔ time jruby -J-server bin/duby examples/tak.duby
Running tak(24,16,8) 1000 times

real 0m13.657s
user 0m14.529s
sys 0m0.450s

So what you're seeing here is that Duby can run "tak(24,16,8)", the same function we tested in JRuby above, in an average of 0.013 seconds--nearly two orders of magnitude faster than the fastest JRuby optimizations above and at least an order of magnitude faster than the fastest incomplete, experimental implementations of Ruby. What does this mean? Absolutely nothing, because Duby is not Ruby. But it shows how fast a Ruby-like language can get, and it shows there's a lot of runway left for JRuby to optimize.

Be a (Supportive) Critic!

So the next time someone posts an article with crazy-awesome performance numbers for a Ruby implementation, by all means applaud the developers and encourage their efforts, since they certainly deserve credit for finding new ways to optimize Ruby. But then ask yourself and the article's author how much of Ruby the implementation actually supports, because it makes a big difference.

Update, April 4: Several people told me I didn't go quite far enough in showing that by breaking Ruby you could get performance. And after enough cajoling, I was convinced to post one last modification: recursion optimization.
JRuby 1.3.0-dev, compiled ("--fast", dyncall optz, recursion optz), server VM:
➔ jruby --server --fast bench/bench_tak.rb 5
user system total real
0.524000 0.000000 0.524000 ( 0.524000)
0.338000 0.000000 0.338000 ( 0.338000)
0.325000 0.000000 0.325000 ( 0.325000)
0.299000 0.000000 0.299000 ( 0.299000)
0.310000 0.000000 0.310000 ( 0.310000)

Woah! What the heck is going on here? In this case, JRuby's compiler has been hacked to turn recursive "functional calls", i.e. calls to an implicit "self" receiver, into direct calls. The logic behind this is that if you're calling the current method from the current method, you're going to always dispatch back to the same piece of code...so why do all the dynamic call gymnastics? This fits a last piece into the JVM inlining-optimization puzzle, allowing mostly-recursive benchmarks like Takeuchi to inline more of those recursive calls. What do we lose? Well, I'm not sure yet. I haven't done enough testing of this optimization to know whether it breaks Ruby in some subtle way. It may work for 90% of cases, but fail for an undetectable 10%. Or it may be something we can determine statically, or something for which we can add an inexpensive guard. Until I know, it won't go into a release of JRuby, at least not as a default optimization. But it's out there, and I believe we'll find a way.

It is also, incidentally, only a few times slower than a pure Java version of the same benchmark, provided Java is using all boxed numerics too.