Sunday, August 24, 2008

Zero to Production in 15 Minutes

There still seems to be confusion about the relative simplicity or difficulty of deploying a Rails app using JRuby. Many folks still look around for the old tools and the old ways (Mongrel, generally), assuming that "all that app server stuff" is too complicated. I figured I'd post a quick walkthrough to show how easy it actually is, along with links to everything to get you started.

Here's the full session, in all its glory, for those who just want the commands:

~/work ➔ java -Xmx256M -jar ~/Downloads/glassfish-installer-v2ur2-b04-darwin.jar
~/work ➔ cd glassfish
~/work/glassfish ➔ chmod a+x lib/ant/bin/*
~/work/glassfish ➔ lib/ant/bin/ant -f setup.xml
~/work/glassfish ➔ bin/asadmin start-domain
~/work/glassfish ➔ cd ~/work/testapp
~/work/testapp ➔ jruby -S gem install warbler
~/work/testapp ➔ jruby -S gem install activerecord-jdbcmysql-adapter
~/work/testapp ➔ vi config/database.yml
~/work/testapp ➔ warble
~/work/testapp ➔ ../glassfish/bin/asadmin deploy --contextroot / testapp.war
Now, on to the full walkthrough!

Prerequisites

JRuby 1.1.3 or higher

None of the steps in the main walkthrough require JRuby, since Warbler works fine under other Ruby implementations. But if you want to install and test against the JDBC ActiveRecord adapters, JRuby's the way. And in general, if you're deploying on JRuby, you should probably test and build on JRuby as well. Go to www.jruby.org under "Download!" and grab the latest 1.1.x "bin" distribution. I link here JRuby 1.1.3 tarball and JRuby 1.1.3 zip for your convenience. Download, unpack, put in PATH. That's all there is to it.

Java 5 or higher

Most systems already have a JVM installed. Here's some links to OpenJDK downloads for various platforms, in case you don't already have one.
  • Windows, Linux, Solaris: Download the JDK directly from Sun's Java SE Downloads page. I typically download the JDK (Java Development Kit) because I find it convenient to have Java sources, compilers, and debugging tools available, but this walkthrough should work with the JRE (Java Runtime Environment) as well. Linux and Solaris users should also be able to use their packaging system of choice to install a JDK.
  • OS X: The 32-bit Intel macs can't run the Apple Java 6, so you'll want to look at the Soylatte Java 6 build for OS X to get the best performance. A small warning...it doesn't have Cocoa-based UI components, so it will use X11 if you start up a GUI app.
  • BSDs: FreeBSD users should check the FreeBSD Java downloads page. I believe there's a port for FreeBSD and package/port for OpenBSD but I couldn't dig up the details. We have had users on both platforms, though, so I know they work fine.
  • Others: There's basically a JDK for just about every platform, so if you're not on one of these just do a little digging. All you need to know is that it needs to be a full Java 5 or higher implementation.
Rails 2.0 or higher

Hopefully by now most of you are on a 2.x version of Rails. This walkthrough will assume you've got Rails 2.0+ installed. If you're using JRuby, it's a simple "jruby -S gem install rails" or if you've got JRuby's bin in PATH, "gem install rails" should do the trick. Note that the Warbler (described later) should work in any Ruby implementation, since it's just a packager and it includes JRuby.

Step One: The App Server

The words "Application Server" are terrifying to most Rubyists, to the point that they'll refuse to even try this deployment model. Of course, the ones that try it usually agree it's a much cleaner way to deploy apps, and generally they don't want to go back to any of the alternatives.

Much of the teeth-gnashing seems to surround the perceived complexity of setting a server up. That was definitely the case 5 years ago, but today's servers have been vastly simplified. For this walkthrough, I'll use GlassFish, since it's FOSS, fast, and easy to install.

I'm using GlassFish V2 UR2 (that's Version 2, Update Release 2) since it's very stable and by most accounts the best app server available, FOSS or otherwise. Not that I'm biased or anything. At any rate, it's hard to argue with the install process.

1. Download from the GlassFish V2 UR2 download page. The download links start about halfway down the page and are range from 53MB (English localization) to 93MB (Multilanguage) in size.

2. Run the GlassFish installer. The .jar file downloaded is an executable jar containing the installer for GlassFish as well as GlassFish itself. The -Xmx specified here increases the memory cap for the JVM from its default 64MB to 256MB, since the archive gets unpacked in memory.
~/work ➔ java -Xmx256M -jar ~/Downloads/glassfish-installer-v2ur2-b04-darwin.jar
glassfish
glassfish/docs
glassfish/docs/css
glassfish/docs/figures
...
glassfish/updatecenter/README
glassfish/updatecenter/registry/SYSTEM/local.xml
installation complete
~/work ➔
Before the unpack begins, the installer will pop up a GUI asking you to accept the GlassFish license.
Read the license or not...it's up to you. But to accept, you need to at least pretend you read it and scroll the license to the bottom.
The installer will proceed to unpack all the files for GlassFish into ./glassfish.

3. Run the GlassFish setup script. In the unpacked glassfish directory, there are two .xml files: setup.xml and setup-cluster.xml. Most users will just want to use setup.xml here, but if you're interested in clustering several GlassFish instances across machine, you'll want to look into the clustered setup. I won't go into it here.

The unpacked glassfish dir also contains Apache's Ant build tool, so you don't need to download it. If you already have it available, your copy should work fine, and the chmod command below--which sets the provided Ant's bin scripts executable--would be unnecessary. If you're on Windows, the bin scripts are bat files, so they'll work fine as-is.

Two items to note: you should probably move the glassfish dir where you want it to live in production, and you should run the installer with the version of Java you'd like GlassFish to run under. Both can be changed later, but it's better to just get it right the first time.
~/work ➔ cd glassfish
~/work/glassfish ➔ chmod a+x lib/ant/bin/*
~/work/glassfish ➔ lib/ant/bin/ant -f setup.xml
Buildfile: setup.xml

all:
[mkdir] Created dir: /Users/headius/work/glassfish/bin

get.java.home:

setup.init:
...
jar-unpack:
[unpack200] Unpacking with Unpack200
[unpack200] Source File :/Users/headius/work/glassfish/lib/appserv-cmp.jar.pack.gz
[unpack200] Dest. File :/Users/headius/work/glassfish/lib/appserv-cmp.jar
[delete] Deleting: /Users/headius/work/glassfish/lib/appserv-cmp.jar.pack.gz
...
do.copy.unix:
[copy] Copying 1 file to /Users/headius/work/glassfish/config
[copy] Copying 1 file to /Users/headius/work/glassfish/bin
[copy] Copying 1 file to /Users/headius/work/glassfish/bin
...
create.domain:
[exec] Using port 4848 for Admin.
[exec] Using port 8080 for HTTP Instance.
[exec] Using port 7676 for JMS.
...
BUILD SUCCESSFUL
Total time: 29 seconds
~/work/glassfish ➔
4. Start up your GlassFish server. It's as simple as one command now.
~/work/glassfish ➔ bin/asadmin start-domain
Starting Domain domain1, please wait.
Log redirected to /Users/headius/work/glassfish/domains/domain1/logs/server.log.
Redirecting output to /Users/headius/work/glassfish/domains/domain1/logs/server.log
Domain domain1 is ready to receive client requests. Additional services are being started in background.
Domain [domain1] is running [Sun Java System Application Server 9.1_02 (build b04-fcs)] with its configuration and logs at: [/Users/headius/work/glassfish/domains].
Admin Console is available at [http://localhost:4848].
Use the same port [4848] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8080 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /__wstx-services ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://charles-nutters-computer.local:8686/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8181 4848 3700 3820 3920 8686 ].
Domain does not support application server clusters and other standalone instances.

~/work/glassfish ➔
Congratulations! You have installed GlassFish. Simple, eh?

A few tips for using your new server:
  • There's a web-based admin page at http://localhost:4848 where the admin login is admin/adminadmin by default. You'll want to change that password. Select "Application Server" on the left and then "Administrator Password" along the top.
  • Poke around the admin console to get a feel for the services provided. You won't need any of them for the rest of this walkthrough, but you might want to dabble some day. And if you want to set up a connection pool later on (which ActiveRecord-JDBC supports) this is where you'll do it.
  • Most folks will probably want to set up init scripts to ensure GlassFish is launched at server startup. That's outside the scope of this walkthrough, but it's pretty simple. I'll update this page (and it's equivalent on the JRuby Wiki) once I know more.
  • GlassFish works just fine as a standalone server, but many users will want to proxy it through Apache or another web server. Again, this is outside the scope of this walkthrough, but it should be as simple as configuring a virtual host or a set of matching URLs to hit the GlassFish server at port 8080 (which is the default port for web applications). For apps I'm running, however, I just use GlassFish.
Step 2: Package your App

This step is made super-trivial by Nick Sieger's Warbler. It includes JRuby itself and provides a simple set of commands to package up your app, add a packaging config file, and more. In this case, I'll just be packaging up a simple Rails app.

Note that Warbler works just fine under non-JRuby Ruby implementations, since it's all Ruby code. But again, if you're deploying with JRuby, it's probably a good idea to test and build with JRuby as well.
~/work ➔ jruby -S gem install warbler
Successfully installed warbler-0.9.10
1 gem installed
Installing ri documentation for warbler-0.9.10...
Installing RDoc documentation for warbler-0.9.10...
~/work ➔ cd testapp
~/work/testapp ➔ ls .
README Rakefile app config db doc lib log public script test tmp vendor
~/work/testapp ➔ warble
jar cf testapp.war -C tmp/war .
~/work/testapp ➔
And that's essentially all there is to it. You will get a .war file containing your app, JRuby, Rails, and the Ruby standard library. This one file is now a deployable Rails applications, suitable for any app server, any OS, and any platform without a recompile. The target server doesn't even have to have JRuby or Rails installed.

Step 3: Deploy your Application

There's two ways you can deploy. You can either go to the Admin Console web page, select "Web Applications" from the "Applications" category on the left, and deploy the file there, or you can just use GlassFish's command-line interface. I will demonstrate the latter, and I'm providing the optional contextroot flag to deploy my app at the root context.
~/work/testapp ➔ ../glassfish/bin/asadmin deploy --contextroot / testapp.war
Command deploy executed successfully.
~/work/testapp ➔
That's it? Yep, that's it! If we now hit the server at port 8080, we can see the app is deployed.
Step 4: Tweaking

There's a few things you can do to tweak your deployment a bit. The first would be to generate a warble.rb config file and adjust settings to suit your application.
~/work/testapp ➔ warble config
~/work/testapp ➔ head config/warble.rb
# Warbler web application assembly configuration file
Warbler::Config.new do |config|
# Temporary directory where the application is staged
# config.staging_dir = "tmp/war"

# Application directories to be included in the webapp.
config.dirs = %w(app config lib log vendor tmp)
...
In this file you can set the min/max number of Rails instances you need, additional files and directories to include, additional gems and libraries to include, and so on. The file is heavily commented, so you should have no trouble figuring it out, but otherwise the Warbler page on the JRuby Wiki is probably your best source of information. And since a lot of people ask how many instances they should use, I'll provide a definitive answer: it depends. Try the defaults, and scale up or down as appropriate. Hopefully with Rails 2.2 this will no longer be needed, as is the case for Merb and friends.

The other tweak you'll probably want to look into is using the JDBC-based ActiveRecord adapters instead of the pure Ruby versions (or the C-based versions, if you're migrating from the C-based Ruby impls). This is generally pretty simple too. Install the JDBC adapter for your database, and tweak your database.yml. Here's the commands on my system:
~/work/testapp ➔ jruby -S gem install activerecord-jdbcmysql-adapter
Successfully installed jdbc-mysql-5.0.4
Successfully installed activerecord-jdbcmysql-adapter-0.8.2
2 gems installed
Installing ri documentation for jdbc-mysql-5.0.4...
Installing ri documentation for activerecord-jdbcmysql-adapter-0.8.2...
Installing RDoc documentation for jdbc-mysql-5.0.4...
Installing RDoc documentation for activerecord-jdbcmysql-adapter-0.8.2...
~/work/testapp ➔ vi config/database.yml
~/work/testapp ➔
And the modified database.yml file:
...
# This was changed from "adapter: mysql"
production:
adapter: jdbcmysql
encoding: utf8
...
Now repackage ("warble" command again), redeploy, and you're done!

Conclusion

Hopefully this walkthrough clears up some confusion around JRuby on Rails deployment to an app server. It's really a simple process, despite the not-so-simple history surrounding Enterprise Application Servers, and GlassFish almost makes it fun :)

20 comments:

noble said...

I can understand why you had to use GlassFish for an example (GlassFish is by SUN). But , out there in the wild the most commonly used containers are Tomcat & jetty . Will the same steps work for these containers ? or are they not supported ate all?

Ariejan de Vroom said...

Great write-up! However, the word "App Server" sounds like "I'll eat all your memory" to me.

What is the memory footprint of Glassfish and what's the recommended system to run on? I think this is one aspect of jruby/app servers that no one seems to be talking about.

Currently I'm running ±5 Rails apps on Apache+Passenger. I'm willing to give Glassfish a try, but does it run on a 512Mb VPS?

I hope to learn a lot more about jruby at RailsConfEurope. Are you going to be there?

mtkd said...

I Installed Netbeans 6.5 last week and tried the jruby and glassfish installed with it - but got in to some "bundle symbolic name" hell, also it didn't seem to find my Gems even through they were on the path listed.

I really want to give jruby a spin, I'll try and make some time for it next week.

Charles Oliver Nutter said...

noble: The same steps should work for any web container like Tomcat or Jetty. Any server that can deploy a .war file should work with warbler.

ariejan: On my system GlassFish + one dummy app running two instances is taking about 215MB. In general we tend to use less memory per instance than Mongrel but not yet less than Passenger, so getting five apps on GlassFish on a 512MB VPS might be difficult. There are other stripped-down options coming soon, however, like the GlassFish gem. Also, I will not be at RailsConf EU but Tom Enebo and Nick Sieger will.

Anonymous said...

'war'bler -- I get it.

Man I feel dumb.

stephen said...

Great information. This kind of goes along the line of ariejan's post.

It's pretty easy to understand how a mongrel cluster or passenger works, but how does JRuby + Glassfish work? Does it spawn threads as needed (passenger) or does it just run what you tell it to run (mongrel)?

I guess it just seems a bit walled off, but that's because I have no clue how Java web servers work.

Julien Ponge said...

You can also use the IzPack-based installer from http://izpack.org/showcase-glassfish-v2/ BTW :-)

Charles Oliver Nutter said...

stephen: Currently, Warbler (using jruby-rack) scales up and down Rails instances as needed. As far as threading, that's handled by the app server, and most servers use a pool of threads to handle incoming requests, parking requests until a thread comes available.

julien: Thanks for that links, I didn't know about those installers :)

Anonymous said...

I'm curious if you did anything to secure/harden the installation?

- Did you modify default settings/configuration or file system permissions in any way?

- Did you delete unnecessary files/configs/samples or anything like that?

Charles Oliver Nutter said...

anonymous: No, I left hardening as an exercise for the user. Those steps would not be particularly different from hardening any server, and certainly aren't Rails-specific, so they're somewhat out of scope for this walkthrough. I'd welcome a pointer to such instructions for GlassFish and other servers.

David said...

Thanks for the great writeup. JRuby is a fine technology, but just to put the "15 minutes" in perspective, porting a rather complex application with a number of external dependencies from ruby to jruby took about a week (which I thought was perfectly reasonable). Some of that time was learning curve, some was swapping in different gems, and the one issue I hadn't predicted was jdbc. We were using DB2, and the JDBC adapter has some different behavior than the native ibm_db gem.

Rene said...

I started up glassfish v2ur2 on a solaris container at joyent and the default memory heap params as listed in the admin console are: -XX:MaxPermSize=192m and -Xmx512m. The current memory consumption without any apps deployed is 171 MB.

I have a small rails app for development running on mongrel on the same container and it's only using 42 MB. I can only assume adding my rails app war to glassfish is going to increase memory consumption from 171 MB, not decrease it.

How can I justify using glassfish over mongrel in this case? Get rid of those default startup memory heap params? Thanks.

Ed Powell said...

Its great to finally have a step-by-step on how to get things running... but my concern is how to develop a Rails application for deployment through JRuby and Warbler. It seems a bit of a pain to have to warble the project every time I want to test it. Although, I suppose that I could just warble an empty project and then unpack the war...

rcoder said...

@Rene: The base footprint of a JRuby deployment is almost always going to be higher. However, you're likely to see real advantages once you start running more applications, especially if you're following "best practices" and running four or more Mongrels per application. Adding a new Ruby runtime to the pool within a single GlassFish instance is almost guaranteed to use less RAM than a full copy of Mongrel, since the JVM can share loaded classes and core library objects between threads.

I've also found that you can get the memory footprint of a JRuby Rails WAR deployment down considerably by using a lighter-weight servlet container like Jetty or Winstone. Of course, you lose all the nice management and administration features that GlassFish offers, and you can't host full-stack J2EE apps in the same container, but for simple web applications on resource-constrained servers, it can be worth it.

ckutler said...

This is for Ed.
An option is to do development and testing using WEBrick, Mongrel, or GlassFish V3 (the 6.5 version of NetBeans makes it easy to develop and test with GlassFish V3), and then use the WAR file for deployment.

rav said...

hey guys, just wondering if anyone knows where glassfish picks up localhost as the servername before it creates the domain. I am trying to use a different virtual servername but i want to change it before the domain is created (since i am trying to automate the process)
Any ideas???/

Martijn said...

Great guide.. got it running in a few hours with an application that's in the middle of it's development cycle and had never run on JRuby before. The biggest hurdle was working around gems with native extensions (byebye RMagick, hello ImageVoodoo).

I was looking for a sensible deployment method for this application and this looks pretty nice. I'm just wondering: Should I combine this with a webserver/proxy (e.g. nginx) for serving static files? I would guess so, but prehaps Glassfish is unexpectedly fast.

Rich Apodaca said...

Thanks for the clear and concise article.

I ran into a little problem with a Rails 2.2.2 test app:

org.jruby.rack.RackInitializationException: Please install the jdbcmysql adapter: `gem install activerecord-jdbcmysql-adapter` (no such file to load -- active_record/connection_adapters/jdbcmysql_adapter)

...

The solution was to include the activerecord-jdbcmysql-adapter gem in my config/warble.rb file (more info here).

danieltsadok said...

Should I try V3? It seems to be more Rails-friendly.

dipe said...

Sounds good "Zero to Production in 15 Minutes".

But I miss a lot of points. A few of them are:

Who is running your migrations? Where are the log-files, session files and so on? How do you track the recently deployed versions of your application as capistrano does? How do you control the jruby version running your application?

I love a capistrano based deployment. But because of many ugly assumption made by the application servers, doing this right is not easy. At least these things have to be resolved:

* Create en exploded directory from the war file, witch enables you to run rake tasks.

* Extracting the jruby from the war-file, so you don't need an extra jruby installation on your app server for running rake tasks.

* Create and maintain a shadow tree for the directories shared by successively deployed applications.

* Decoupling jruby version from warbler (seems already be done).