Wednesday, April 22, 2009

The Future: Part One

There's been a lot of supposition about the future lately, and I've certainly read and succumbed to the temptation of the prognosticator. I'm not going to comment on any of the players or put forward my own suppositions about what might happen. What I will do here is talk about what *should* happen.

It's apparent that the Java platform is at a crossroads. One path leads to irrelevance, be it through general apathy that important technologies are getting sidelined, or through active emigration due to bureaucratic processes and waterfall platform evolution. The other path leads to a bright, open future, where polyglots romp and play with fresh new languages and developers have freedom to use whatever tools they feel are necessary for a given job. Given the large investment many of us have in this platform, we need to start talking now about which direction we want to go.

I've been doing a lot of thinking about the future of the Java platform and my future developing for it, and I've come up with a few things I believe must happen soon to ensure the future of the JVM and the applications and languages built for it.

Languages

As you might expect from me, the first area involves supporting many languages on the JVM. Over the past three years we've seen a spectacular transformation take place. The "Java" VM has become a truly multi-language platform. People are putting into production applications running all sorts of languages, often writing absolutely no Java to do so. And this is how things must be...this is how the platform is going to survive.

As I see it there are currently three primary players in the language domain: JRuby, Scala, and Groovy (in no particular order). I include these three due to the relative completeness of their implementations, vague popularity metrics, and availability of production use cases. These three languages, one static and two dynamic, bear further exploration.

JRuby represents the promise of off-platform languages being brought to the JVM. Rather than creating a language specifically tailored to fit well into the JVM's type system and limitations, we have managed to take a drastically different language and implement enough of it to be the only "alternative" implementation realistically considered for production applications. And in doing so we've had to stretch the platform. We've created native binding libraries, wired in POSIX functions the JDK doesn't provide, implemented our own core types like Strings, Arrays, and regular expressions, and done all this while managing to deliver the best performing compatible Ruby implementation available. And where the other two primary language contenders largely pull developers from other areas of the Java platform, JRuby actually brings in many developers that might not otherwise ever use the JVM. JRuby stretches and grows the platform.

Groovy is the second dynamic language under consideration. Groovy represents taking the best features of a number of dynamic languages and wiring them into a Java-like syntax that's easy for existing Java developers to learn. Groovy provides a solid dynamic language foundation without breaking Java type-system requirements like ahead-of-time compiled classes and static method signatures, allowing it to enlist directly in newer APIs that depend on those requirements. And while many developers come to Groovy from other areas of the Java platform, they might also have completely left the platform if not for Groovy. Groovy provides for Java developers a "dynamic layer" that doesn't require them to learn a new platform and a new set of libraries. And so Groovy's strength is in maintaining the platform and stemming the bleeding of developers to off-platform dynamic languages.

Scala, it must be stated, is the current heir apparent to the Java throne. No other language on the JVM seems as capable of being a "replacement for Java" as Scala, and the momentum behind Scala is now unquestionable. While Scala is not a dynamic language, it has many of the characteristics of popular dynamic languages, through its rich and flexible type system, its sparse and clean syntax, and its marriage of functional and object paradigms. The supposed weakness of Scala as being "too complicated" or "too rich" can be countered by developing coding standards, creating more robust editors and tools, and by better educating polyglots on how best to use Scala. Scala represents the rebirth of static-typed languages on the JVM, and like JRuby it has also begun to stretch the capabilities of the platform in ways Java never could.

Among the secondary players I include language implementations like Jython, Clojure, and Rhino. While still in early days of adoption (or in Jython and Rhino's cases, early days of re-awakening), they represent similar aspects to the primary three languages. For purposes of discussion, we'll leave it at that for now.

In order for the platform to embrace these and many future languages, several things need to happen:

  • Ongoing work on these languages must be funded in such a way as to avoid product lock-in. Funding them by tying them to specific technologies will only fracture development communities, and likely damage any existing open-source contributor base.
  • There must be paid support offerings for these languages outside of specific products. For people interested in running JRuby on Rails, Grails, or Liftweb, there must be support channels they can follow, regardless of product bundling attempts. In the JRuby world, for example, we receive many requests for paid support, either in the form of hired guns to optimize an application or through targeting resources to critical bugs. And so far, we have been unable to offer such support.
  • We must continue the work started on the OpenJDK and MLVM projects. OpenJDK has enabled such projects as Shark (Hotspot on LLVM) and the MLVM (JVM extensions for non-Java language features), and we must see these efforts through to completion.
  • Finally, we need a "languages czar" that can coordinate the technical aspects of these various projects and help direct resources where needed. This would largely be a community-cultivating role, but as a sort of "open-source manager" to ensure development efforts are both cooperating and not being needlessly duplicated.
I believe it's absolutely vital that these tasks be met, or we risk the future of the platform entirely. And losing must not be an option, lest we fall back into proprietary alternatives controlled by a single entity.

More to come.

20 comments:

Javier said...

good review of the actual situation and trends of JVM although i am a clojure fan and would it on top of the JVM realm :-P

opinali said...

Great post. I for one, play in the static languages league and would love to see something like Scala taking the baton from Java (especially since the evolution of Java hits brick walls like we see once again in Java 7 - no hope for closures, reified generics, properties etc.) I've been learning JavaFX even not being a GUI guy, just for the sheer pleasure of using a languge that keeps what I love from Java but adds type inference and other modern traits like some support for FP. And I echo your words on Scala's complexity - the language is already a little hard for less experienced developers, I hope they don't go the way of Haskell which was a hugely promising language but became a playground of PhDs in post-modern typesystems and completely died as a viable mainstream language (wonder if Haskell' has any chance to revert that).

I hope Oracle understand the enormous value of what you and other alternative-JVM-languages hackers have been doing. Ditto for other major players, IBM for example seems to be supporting Groovy as one of their "sMash" languages; Google is also being very helpful to make several languages work in their AppEngine for Java.

dysinger said...

You forgot Clojure. There are folks doing clojure in production and it surpases Scala on github.com for repositories. It's a close second behind Scala on wefollow.com.

dysinger said...

I guess I commented too fast. You did mention clojure. Lots of people like Clojure, including myself (just as much as I like JRuby) and there's lots going on in the community.

Alex Kane said...

I enjoyed reading this, thanks for posting. Looking forward to more posts on the future of the JVM.

James Iry said...

@Charles - I'm very curious about the czar thing. Even relatively homogenous projects with a well recognized leader like the Linux kernel project are just shy of rampant anarchy. How can a language czar have any real influence across multiple communities with different goals?

Andy said...

Will the Language Czar have to face scrutiny regarding whether or not they paid their taxes? If so, there's, unfortunately, little hope in the position being filled.

Philip Jenvey said...

The JVM has come a long way in becoming a multi-language platform. Unfortunately Microsoft's .NET is still a little bit ahead here. They started earlier and fund many more .NET language related developers.

I mean, the .NET DLR work excites John Rose. It excites me too, and I hate Microsoft as much as anyone could. This is kind of a sad situation: the JVM, with so much potential in this area, playing catch up to the CLR. Meanwhile a key player like Charlie is reduced to almost groveling for more resources.

James E. Ervin, IV said...

I think you are right on. In fact, I think this transition has been in the works for a while or at least has been needed for a while. The Java platform includes the libraries, JVM and the languages built on top of it. Java as a language is beyond mature, in fact the mess of generics (though I love them in the simplest use cases like collections) shows that Java 5.0 should have been another language entirely, leaving Java 1.4 as the conclusion.

The success of the platform recently has been the fact that it has enabled development of frameworks and languages on top of it. This trend should continue and be pushed harder. The technical merits of that path are obvious.

Still, I think that it would be more useful for someone to make a stronger business case for this path. What I mean is, on the one hand, Sun's stewardship of the Java platform has been good, but other companies make far more money than Sun does off of it. This is not only *unfair* but I think unsustainable. Companies and organizations that derive business value (read cash,scratch,moola,dead presidents) from the Java technology should be the ones also devoting resources to improving Java. Of course, this is a common problem, I know Eclipse for one has it too and maybe more so. The point is how we might be able to correctly point the economic incentives in a way that helps focus resources down the path we would like to see Java go.

Btw, I have no idea how to accomplish this. :)

Anonymous said...

The JVM is not a good fit for dynamic languages, although it could be changed.

http://www.infoq.com/news/2007/05/java-dlr

James E. Ervin, IV said...

Well anonymous, that article was from 2007 and the JVM has changed in the meantime. I believe the author of this blog is the main force behind that effort.

Herve said...

Excellent post as always. This is clearly the way Oracle should go. However, they should not abandon pure Java (not JVM) work, because there is a huge installed base, and I also still think that Java is still a good language. I still love to code in Java ;)

@Philip Jenvey: What I saw of the DLR code some time ago did not convince me. This could change I agree but for now I still need to be convinced ;)

Luke said...

I really liked this post. We're using Lift/Scala as the foundation for Innovation Games online, our serious games platform. I'm very pleased with the results. Of course, I'm biased, because my server team is Dan O'Leary (full-time) and David Pollak (half-time) and they're two of the most incredibly talented developers and architects I've ever known. We're looking forward to watching the growth of the games put a lot of great stress on Lift/Scala games.

Regards,

Luke Hohmann
CEO, Enthiosys, Inc.
615 National Ave, Ste 230
Mountain View, CA 94087
m: +1-408-529-0319
lhohmann@enthiosys.com
www.innovationgames.com: The seriously fun way to do serious work -- seriously.

Miranda said...

Very nicely put Charles.

Ken Kousen said...

Great post. I find it very amusing, though, that you neglected to mention JavaFX. That's my experience, too, but Sun obviously wouldn't agree, judging by the topics at the upcoming JavaOne conference. :)

Anonymous said...

James, I don't think JSR 192 is finished yet, and invokedynamic isn't available in any JVMs is it? Actually, I was comparing it to the Strongtalk vm, what a shame that Sun didn't choose to finish it instead of having its authors work on Java.

Andrew Whaley said...

Good unbiased post and I think I probably agree with your conclusions re: scala.

Personally, I prefer Scheme and have used SISC to build web applications on the JVM which is a breeze with SISCWeb but there is a performance hit. I notice that neither Scala or Clojure support first class continuations which is a real shame as I would consider them essential for any sensible web program. If the JVM platform could be extended to support native continuations and tail-call optimisation it would open it up to some great languages.

Rich Dougherty said...

@Andrew: Scala 2.8 will support first-class delimited continuations, thanks to a clever compile-time transformation. But it would be even better if the JVM support them natively, of course.

Daniel Serodio said...

You've neatly summed up my thoughts about the future of Java. I've used (a little) Groovy, but I really miss the refactoring and code-completion comfort that only static types can give.

Scala apparently has managed the unite the best of both type systems (dynamic and static types) and programming paradigms (OO and functional).

omega-3 said...

Thanks for sharing such nice information on Java platform. I am doing programming in Java and this information will be very useful to me. I like this information. It is also a very useful information for IT student. Well It is nice to post here. I like this site.