Saturday, March 10, 2007

Ruby on Grails? Why the hell not?

I've spent some time this weekend looking over the Grails codebase. After a truly gigantic checkout (Grails bundles production JARs for all the libraries it integrates) and after swimming through a substantial, single-project sea of source files, I think I'm starting to follow it (though a tour by someone familiar with Grails internals would be nice). But there's something else I've learned in the process...

There's really no reason this couldn't use Ruby just as easily as Groovy.

Let's look at some really dumb numbers. Grails, as it turns out, is actually over 2/3 Java code, unlike Rails which is 100% Ruby. Grails contains around 436 Java source files for about 54kloc of code and 273 Groovy source files for about 23kloc of code. So Java outnumbers Groovy by a 2:1 ratio. And yes, Java's probably more verbose than Groovy in most cases. But my point stands, and I'll back it up with more...

Then there's the libraries. The Grails binary release weighs in at a whopping 26MB compressed and 49MB uncompressed (*wow*). But 31MB of that is third-party libraries completely independent of Groovy, things like Hibernate, XFire, Quartz, Apache Commons, and so on. Almost all of which (or perhaps all of which) are pure Java code. So the ratio is even more heavily toward Java.

These numbers and my exploration of the code base lead me to a surprising conclusion: Grails is not a "Groovy on Rails" by any means. It's a large number of popular, well-established Java libraries stitched together mostly by Java code with a thin layer at the front (20kloc is not much) of Groovy code for end-users to see and work with. And I guarantee you it's not the Groovy part that's most interesting here. Hell, you could write that part in Java if you followed enough "convention over configuration", and you certainly could write it in Ruby.

I decided to do this exploration after realizing there's nothing about Ruby or JRuby that couldn't be used to wire a bunch of Java libraries together--indeed, that's one of the biggest selling points of JRuby, the fact that you can write plain old Ruby code but call Java libraries almost seamlessly (with day-by-day improving integration points and performance numbers). So why not take Grails and do a port to Ruby? Groovy code isn't far off from Ruby in many ways, and everything you can do in Groovy you can do in Ruby (plus more), so the port should be pretty straightforward, right?

Then I find out that Grails is mostly Java code. And instead of being annoyed or repulsed by this revelation, I was intrigued. Why not make Grails work with Ruby as well? And really, why not make it work with any scripting language? Why limit that nicely-integrated back end by forcing everyone to use Groovy, regardless of preference? Have we learned nothing from a decade of "100% Pure Java"?

So I propose the following.

  • I believe it would be possible to completely isolate Grails' tastier bits from Groovy. I don't claim it would be easy, but Grails is fairly well interfacified, injectificated, and abstractilicious, so making that dividing line clear and bold is certainly possible (and heck, that's the point of dependency injection and component frameworks anyway, isn't it?). Really, I could start implementing a few of those interfaces right now with Ruby code, but that's not the right approach.
  • Grails' value is not in Groovy, but in the clean integration of established libraries behind the "Convention Over Configuration" mantra. There's nothing specifically Groovy (or Ruby) about that religion, and there's nothing Groovy does in Grails that another JVM language couldn't do just as easily. This is *truth*. And Grails would benefit tremendously by expanding to other languages. Being tied to Groovy alone will hold it back (just as a framework tied to any one language will be held back, Ruby included).
  • I also believe that if the Grails project doesn't make an official effort in this direction, others are likely to take up that mantle themselves. Though the debates will forever rage about which JVM dynlang should carry us into the future, the same facts that have guided programming for half a century still apply today: No one language will be enough to carry us forward; no languages will survive indefinitely; and the language you'll use in ten years you've probably not even heard of yet. Did anyone expect Java to be the most widely-used, widely-deployed language ten years ago? Well today, it is. It won't always be.
  • There's also a very interesting angle here for language enthusiasts. Nobody doubts that the Java platform is officially becoming multi-lingual, possibly a complete reversal from years past. But there are going to be growing pains here, and we're already feeling them on some projects at Sun. We have this wonderful platform with all these wonderful languages...and they all do things differently. We have a first stab at official "Scripting" support in JSR-223, but it only defines the integration point from Java to the language in question...only partially defining the integration points for those languages calling back to Java (a tricky problem) and defining *nothing* related to making those languages interoperate with each other. Again, Java becomes the integration point, and the bottleneck (as in "how do you pass a dynamic-typed object from one language to another by sending it through a statically typed language?"). The very exercise of making Grails (or any other framework) work equally well across multiple user-facing language interfaces will require better interoperability and more clearly-defined integration points for dynamic and static languages on the JVM.
  • The Phobos guys at Sun are already tackling many of these same problems while expanding their JavaScript-mostly Rails-inspired web framework to other JSR-223 languages (notably, Ruby). So anyone taking on the Grails effort would certainly not be alone, and there are some damn smart engineers at Sun interested in tackling these very same problems.
And I guess in the end it's a question of how visionary we all can be. Can the Grails team envision a future for Grails that doesn't require the use of Groovy? Are you (and am I) as a non-Groovy developer willing to help them get there? And in the end, isn't this what thought leaders on a multi-lingual platform are supposed to be doing?

--

So, enough of the inflammatory bull-baiting. I dare you to tell me why I'm wrong here. Even better, I dare you to look into it yourself. 75kloc of code, only a third of it Groovy. Established libraries you're sure to have used at some point. Clean integration, following the principals inspired and proven by Rails.

But fronted with your language of choice.

Make it so.

14 comments:

Daniel Spiewak said...

This is actually a really cool idea. I don't have much experience with either rails or grails, but I am familiar enough with the two to know that they each have their respective strengths. Offering grails running on JRuby would only open the door to non-groovy enthusiasts (like myself).

Jürg Lehni said...

You should have a look at Helma too. It is a similar system that has been around and evolving since 9 years or so. It is based on Java and the Rhino JavaScript engine, also offering the advantage of being able to integrate Java libraries in a script driven environment, thanks to Rhino's Java bridge.

And some of the features of Ruby on Rails that make it so successfull have already been in Helma since a long time, e.g. database abstraction, clean MVC, etc.

As the important parts are implemented in Java, it performs better than Rails (maybe similar to the performance found in Grails).

I think it would be great to offer a general framework for integrating Scripting languages that goes much faster than what javax.script currently does, so all these frameworks could all of a sudden offer more than one language. Being the author of a Rhino base project too (Scriptographer.com), I am currently looking into the possibilities for Jython and JRuby, and already started the discussion on the Jython mailing list. What we would need is the possibility to write Java classes that do not need to know about what Scripting language they are used in, and to formulate a set of rules of how they are wrapped (e.g. creation of bean properties, hints that an objects behaves like a dictionary or like a list, etc).
JRuby's JavaUtilities.extend_proxy goes into that direction, in Rhino one can achieve similar things through an overridden WrapFactory that creates custom wrapper objects for certain Java instances, but it would be really nice to find a common framework that works among all engines.

Also, an abstraction for callback methods on objects would be needed, so native Java classes could define getters and setters for callback methods, and call these if they are set, regardless of the scripting language they are formulated in.

As a further step, one could imagine a common framework for wrappers among all scripting languages, and implementations of such wrappers for each of the language's native objects, so they could be used in the other languages, including access to fields and execution of mehtods. Imagine a setup where you have some scripts written in Ruby (JRuby), some in JS (Rhino), and some in Python (Jython), and they would all share the global scope, and could use objects defined in the other language. The advantage of the java bridge would then also apply to libraries written in these languages. Now that's a goal for Java based dynamic languages for 2007!

Vita Rara said...

I've been working on integrating Groovy with Strus 2 and Spring. I have taken two different approaches:

1. Instantiating my actions in Struts 2, and then having Spring wire their dependencies.

2. Having Spring instantiate the actions, and wire their dependencies.

Spring has growing support for scripted beans, including dynamic recompilation. (ie. You don't need to recomile/redeploy your web app to see the change.)

The advantage of using Spring to instantiate the beans, is that they can be written in any scripting language supported by Spring. So, you could have a JRuby action, a Groovy service bean, a JRuby DAO, and a Java domain entity using JPA or Hibernate for example.

My issue right now is that Spring seems to wrap all scripted beans in a proxy. Therefore you need to have them implement an interface for any methods you want to publicly expose. (That's a PITA for simple things like a Struts 2 Action, which are highly dynamic.) Also, Spring does not support "prototype" scripted beans currently. Although I've submitted a patch for that to their JIRA.

Take a look at my latest ramblings on this subject.

Donald said...

Why not? Supporting other dynamic languages would make grails stronger.

It would also make a significant statement: the developers of groovy and grails do not suffer from the same hegemonic tendencies as other similar dynamic language projects. Imagine how the rails creator would response to this same question. Which raises an interesting question about defending groovy from the hegemony of other scripting languages...

Would there be any reason to provide multiple language support for the build or other parts of the framework that are written in Groovy?

Would there be any benefit to using Java less in Grails? The developers seem to have given a lot of thought to when they should use groovy over Java. In fact, it would be interesting to hear more about their criteria, because it is definitely easier to write Groovy code than Java. Is it for performance? Is it because it seems like the right thing to do?
It's hard to say, I've written builders in Groovy and Java, and I'd like to think the choice is a matter of taste and context.

Raphaël Valyi said...

I'm using both traditionnal J2EE stuff from GRails at work (Struts(2), Spring, Hibernate, Apache commons...) and Rails stuff at home.

My point is that Rails is just much better as an MVC framework + most of the other features web development requires.

First, it's a lot DRYer so the learning curve is much faster (and this is due to language level feature such as closures and multiple implementation inheritance through mixins Ruby provides where Java will have to use overkill decorators/factory patterns leading to obscur APIs)

Second, there is absolutely no justification for a faster language in the MVC wiring of web development. the MVC wiring is not where the time is spent.

Tird, the "static types" argument is crappy for the MVC and the mapping stuff as those are full of static flaws: HQL validity is not enforced at compilation time (with free IDE at least), neither is the type of your queries, neither are your JSP properties/EL expressions in your views, neither is your Spring wiring. Considering all this Rails+ easy unit testing just makes more sense.

I could only find some advantages of Hibernate for existing very complex mappings where ActiveRecord won't make it. Still ActiveRecord miht be easier (thus more efficient) in 90% of the use cases.

So I think Struts and all other J2EE MVC frameworks are really unproductive compared to the MVC from Rails.

Finally, I think your proposal might only be good at trying to increase the JRuby momentum by attracting developpers from GRails into a common synergy. Also, more importanty, this could open the door to a smooth transition to Rails in legacy web apps where Hibernate mappings are common and could be reused for more Rail oriented MVC.

Peace.

Raphaël Valyi.

Graeme Rocher said...

Hi Charles,

I would be happy to give you a tour of the codebase.

In terms of supporting Ruby, its just a matter of resources. See my post in response:

http://graemerocher.blogspot.com/2007/03/charles-nutter-ruby-on-grails-story.html

;-)

Cheers
Graeme

Justin Forder said...

You are right that something like Grails, built on the same Java frameworks, could be done in JRuby. My guess is that it wouldn't be Grails - or else it wouldn't be a natural use of Ruby. I feel that Grails should be given space to grow up in its own way, becoming a showcase for Groovy, just as Rails is for Ruby.

I'd rather think about what - apart from language preference - might cause someone to choose Grails rather than Rails on JRuby. Can Rails/JRuby be adjusted (e.g. via Java-specific plugins) to reduce the difference? For example, if I want to mix carefully tuned Spring MVC/Spring/Hibernate application content with admin pages using a lighter-weight framework against the same database - Grails supports this, with one O/R mapping; could Rails/JRuby?

Regarding the use of Java in Grails, that seems fine to me. In a Groovy application there's less of an obstacle to doing parts in Java than there would be for doing parts of a Ruby application in C. Given the openness of Ruby, it would be possible (if a bit scary) for a JRuby/Java plugin to replace parts of the Rails implementation with more efficient Java code - have you been at all tempted to do this?

Keep up the excellent work - it's truly impressive, as is your colleagues' work on NetBeans support for Ruby/JRuby.

Raphaël Valyi said...

Justin,
here are some comments about what you asked:
"if I want to mix carefully tuned Spring MVC/Spring/Hibernate application content with admin pages using a lighter-weight framework against the same database"

Spring:
I think you could launch Spring from your config/environment.rb to initialize your beans with Spring before your JRUBY on Rails apps use them. But that would only help for the J2EE stuff, because for Rails already has some kind of IOC (via Ruby mixins or dynamic redefinitions) support wich is more mannageable than Spring.

Struts MVC:
If you use Struts or an other MVC framework in your JRuby on Rails app, really I tink you couldn't use a lot of Rails then, may be only ActiveRecord. That'll be interesting to discover if you can make the to action dispatcher co-exist to server both JSP/MVC and RHTML/MVC though. Does anybody know about that point? Still I think Rails MVC outperform any other J2EE MVC.

Hibernate:
You got it, that's one of the best reasons to bridge Rails and the J2EE world. As far as I know Rails, that would be totally easy to use Hibernate as the ORM layer instead of ActiveRecord. I already used Rails almost without ActiveRecord (with a legacy data base) and I think wrapping a n Hibernate bean into an ActiveRecord object (or even skip it would be easy) and very welcome if your mapping is dirty.
For instance, you could forward all the methods from your ActiveRecord/other wrapper instance to the corresponding Hibernate bean via the method_missing ruby method (I did similar things already between several ActiveRecord objects for instance).

Beside the speed of the J2EE stack itself, using ActiveRecord-JDBC could also may be allow us to use a faster database: Hypersonic to name it. Swapping my DB to this one (all working in your RAM if you have enough RAM) is one of the first things I'll try if I can switch to JRuby on Rails.

Regards and long life to JRuby on Rails.

Anonymous said...

Why hibernate and not the JPA spec (who cares if this is kodo, toplink, hibernate).

Why Xfire and not the JaxWS Java EE (SE) standard?

Scott said...

I just had dinner with Charles a couple of weeks ago at a No Fluff, Just Stuff show in Milwaukee. Unless his thoughts have drastically changed in the interim, I don't think that he is "bull baiting" as he implies.

As we talked, we both came to the same conclusion: JRuby is for folks who like the Ruby dialect. Groovy is for folks who like the Java dialect. Plain and simple. JRuby is less an effort to integrate Ruby with Java than a project that allows copy/paste source code compatibility with C/Ruby on the JVM with only minor exceptions. Groovy offers the same copy/paste compatibility with Java.

If you have an existing Ruby on Rails app and you want to run it on the JVM, JRuby is a great choice. If you have experience with well established Java libraries like Spring and Hibernate, but are envious of the scaffolding and convention over configuration of your Rails brethren, Grails fits the bill.

Grails is nowhere close to a straight port of Rails to another language. Inspired by: yes. Ported from: no. Grails (and Groovy) respect the idioms of the Java language. While a JRuby port of Grails is an interesting thought experiment, I think that it kinda misses the point. That doesn't mean you shouldn't try it, but I wouldn't hold out hope for widespread Java/Groovy community support any more than Ruby folks are dying for a Java version of Rails. Both frameworks are reflective of the strengths of their respective underlying languages.

And speaking of respect, Charles' posts to the Groovy mailing lists have been exceedingly polite and well thought out. Certainly no one would be naive enough in this day and age to argue that programming languages are a zero sum game -- that one is inherently better than all the rest.

Charles is doing great work for JRuby. But not all roads lead to Ruby. One man's JRuby is another man's Groovy. Or Jython. Or JavaScript. Or best of all -- any one of them, used in the right circumstances.

Graeme Rocher said...

Nice post Scott. I think that reflects the situation very well.

schwabsauce said...

It's great to see so many hints that I'll probably want to be using Java in some forms quite a bit in the coming decade. I had recently thought I'd avoid it for the most part in favor of Ruby and Objective C. The improvements to Java itself and the beans are almost as compelling as the proliferation of languages coming to the vm.

I think Ruby on Grails will be a very strong move that helps to combine the momentum of Ruby with the mass of Java to bring more standardization around our theories about drying up the web. If Python and Helma and others end up fully compatible, so much the better. Unlike some here, I would generally suspect that all this noise would bring a lot more users to Groovy, in web context and others.

Anonymous said...

gjyj

rs.martin said...

"Ruby on Grails? What the hell?"

One of the strengths of Grails is that it provides Rails-like RAD for people who have no desire to use an archaic, stomach-churning language such as Ruby.

Language design has come on a long way since Ruby came along and in that time we have learnt a lot about transparency and maintainability as features that emerge from subtleties in the language. Rails is popular despite Ruby not because of it.

This, of course, actually reinforces your point about the value of a RAD web framework that is independent of the language choice. The framework should allow the language used to model the domain and craft the views and controllers to be pluggable.

Groovy, Ruby, pure-Java, even an XML language with a domain-modeling GUI on top -- all of these and more would find legions of supporter who would benefit from it all being backed by a unified and familiar framework and deployment model.