deployment

11 Things to Consider Before Deploying Your Rails Application


Cross Posted from http://vinsol.com/blog

At VinSol, we have been developing and deploying Rails applications for more than four years. During this period, we have identified some best practices that we prefer to follow while deploying rails application to production server.

Below is the checklist of these practices:

 

1. Ensure that NS records and MX records are changed if they need to be changed

Changing nameservers will point the domain to the hosting server, and changing MX records will redirect incoming mails to the mail server. As a very first step, we should make sure that name servers of the domain are set to be the correct one. Changing MX record is a must if our application is parsing incoming mails or we wants to use other mail services for e-mail exchange, for example Gmail.

 

2. Ensure some backup mechanism in place for both data as well as user uploaded content like images/documents etc.

Since production data is very critical, we must setup backup mechanism. It could be some type of scheduled task that takes periodic backup of all critical data, Or it could be some type of backup service provided by hosting company. When we talk about critical production data, it includes production DB, content generated by application users like images, documents, etc.

 

3. Ensure database indexes

We might have done development without having proper database indexes, but we should avoid going to production without them. Adding indexes might slow down insert queries a bit but it increases the performance of read queries. It applies when application in production has percentage of read operations much more than write operations.

 

4. Enable your slow query log

This is specific to MySQL. Enabling slow query log allows MySQL to log slow running queries to a file. And this log can be used to find queries that take a long time to execute and are therefore candidates for optimization.

 

5. Ensure exception capturing is in place

We might want to be notified when something bad happens to our application. There are several hosted services available who receive and track exceptions, for example Hoptoadapp.com, GetExceptional.com etc… Either we can choose one from these hosted services or we can use “exception notifier” plugin.

 

6. Ensure adding entries for cron/scheduled jobs

Most of the applications have some functionality/jobs that need to be run periodically, for example generating invoices, sending newsletters etc. In most cases these jobs are done by a rake task. We should make sure that we have added such jobs to cron or similar program.

 

7. Monitoring important processes

To ensure that our site is up 24×7 we need to ensure that all processes that our application needs are up. There can be many processes like MySQL, Mongrel, Apache etc.. These processes are very important as our application directly depends on them. For example if MySQL process get killed accidentally, our application would not be able to connect to MySQL and will start throwing exceptions.

We can choose any of the available monitoring tools like God, Monit, 24×7 etc…

 

8. Ensure confidential data filtering

We would never like to leak/share confidential information of our application users. We should make sure that none of the user’s confidential data like SSN, Credit card info, password are being written to log files. We might not have paid much attention on this while developing the application.

 

9. Rotate log files

Once our site is up and running, every single request write some text in log file. And hence size of the log file keeps on increasing. Larger log files can put us in trouble if we get it beyond certain size. Its difficult to manage these log files, as larger files need more memory to open and need more time to download. In one of the rescue project we did , the log file size was 3GB.

We would recommend having logrotate setup for the application.

 

10. Setup Asset Host

Setting up asset hosts can reduce loading time by 50% or more. We must setup asset hosts for our application. Once asset hosts are all set, our static files will be delivered via asset hosts for example asset1.hostname.com, asset2.hostname.com

 

11. Clearing up stale sessions

We should make sure we should not left any stale session on the server. If our application is using DB or file system as session store, we must add a schedule task to delete stale sessions.

These are some of the points we have identified from our past experience and we might be missing some. Feel free to always add them as comments, and I’ll keep this post updated.


Akhil is a senior software engineer working with Vinsol for last 5 years. He is an inhouse deployment ninja.

 

 

We also provide affordable rails deployment services.

 

 

Deploying rails application with pound as a Balancer

Now a days Apache + mod_proxy + mongrel_clusters, Lighttpd + Mongrel cluster and Nginx + mongrel cluster are well known for deploying rails applications.

You can also deploy your rails application with pound(a reverse proxy, load balancer and HTTPS front-end for Web server).

First you need to setup mongrel_clusters for your rails application by issuing ” mongrel_rails cluster::configure -e production -p 8000 -a 127.0.0.1 -N 3 -c ./ ” inside the rails application root directory(on client machine). This will create mongrel_cluster.yml in config directory. You can change parameters in this command, as -p 8000 specifies that mongrel instances will start up on port number starting 8000, -a 127.0.0.1 specifies that mongrel instances will listen to the localhost, -N specifies the number of mongrel instances and -c specifies the rails root directory.

Now you need to install pound(if not installed) by issuing following commands(as root):

  • cd /opt/src
  • wget http://www.apsis.ch/pound/Pound-2.3.2.tgz
  • tar xzpf Pound-2.1.6.tgz
  • cd Pound-2.1.6
  • ./configure
  • make
  • make install

This will install pound in /usr/local/sbin/pound. In order to proceed further we need to create pound.cfg(pound configuration file) in /etc/pound/pound.cfg . Below is the content of pound.cfg:


User "pound"
Group "pound"

ListenHTTP
  Address 0.0.0.0
  Port 80
  Service
    BackEnd
      Address 127.0.0.1
      Port 8000
    End
    BackEnd
      Address 127.0.0.1
      Port 8001
    End
    BackEnd
      Address 127.0.0.1
      Port 8002
    End
  End
End

Start mongrel cluster by issuing mongrel_rails cluster::start in you app root directory, start pound by /usr/local/sbin/pound -f /etc/pound/pound.cfg , now you are done. Pound is listening the port 80 and redirect all requests to mongrel instances running on 8000, 8001, 8002.

* Please Note that we have configured pound at port 80, if port 80 is being used by apache or any other application pound will not start. You need to stop any service using port 80, if it is apache then stop apache, change line ‘Listen 80′ to “Listen 8080″ and start apache.

In a specific case, when apache is running at some port (let say 8080), you may want to use apache to serve static content of your application, in order to reduce some load from mongrels. In that case use the following:


User "pound"
Group "pound"

ListenHTTP
  Address 0.0.0.0
  Port 80
  Service
      URL "/(images|stylesheets|flash|javascripts)/"
      BackEnd
          Address 127.0.0.1
          Port    8080
      End
      Session
          Type    BASIC
          TTL     300
      End
  End
  Service
    BackEnd
      Address 127.0.0.1
      Port 8000
    End
    BackEnd
      Address 127.0.0.1
      Port 8001
    End
    BackEnd
      Address 127.0.0.1
      Port 8002
    End
  End
End

This will redirect all requests for image, stylesheets, javascripts, flash to apache. Now we need to configure apache to serve those static content. Just add a virtualhost for that:

<virtualHost *:8080>
  ServerName example.com
  ServerAlias www.example.com
  DocumentRoot /var/www/html/example.com/public
  <directory "/var/www/html/example.com/public" >
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  </directory>

  RewriteEngine On

  # For static files it's good to avoid hitting your mongrel process
  # so let apache knows it should serve it directly
  # for a rails application it means, files in images / stylesheets / javascripts
  RewriteRule "^/(images|stylesheets|flash|javascripts)/?(.*)" "$0" [L]

</virtualHost>

Its all done. All requests for dynamic content at port 80 will be redirect to mongrel running at 8000, 8001, 8001 and requests for static content will be served by apache running at port 8080.

Stuck with capistrano

I was deploying my latest code to live site today using Capistrano, and suddenly I stuck. When I tried to deploy the code, it checked out the latest version on the server and while executing after_symlink task it rolled back. I was surprised because the deploy script was working fine earlier and I haven’t made any change in deploy.rb or deploy procedure. I tried many times but not succeed, and then I found the root of problem.

The problem was that I mistakenly created a file in releases directory and Capistrano uses ln -x1 command to create symbolic link to current release. And Capistrano links last output of ‘ln -x1′ as ‘current’, so in this case ‘current’ was linking to a file. This was doing all the mess, as ‘current’ was pointing to a file instead of latest release directory. I deleted that file from releases and then it worked fine…

Engine Yard: Best Hosting for rails applications

Today I deployed my rails application to Engine Yard server. They have great configuration, ultimate cluster architecture(physical clusters then logical clusters). They have separate servers for SSH, email, web, app etc… . Servers are managed by top architects, I should say they are expert in rails application deployment.

apache proxy balancer + mongrel clusters and deploying application with capistrano

So you want to setup production server with mongrel clusters and apache proxy balancer, also wants to use capistrano for deployment, huh. Take it easy, its very simple.

You need Apache 2.2 or later on your production server, and the following ruby gems on your both machine(server and local):

  • capistrano
  • mongrel
  • mongrel_cluster

I haven’t mentioned rails and rake gem as we are deploying a rails application so these gems are obvious.

Lets install above gems (if not installed) by issuing:

gem install –include-dependencies capistrano
gem install –include-dependencies mongrel
gem install –include-dependencies mongrel_cluster

Now make sure that the following modules are enabled (they are disabled by default) :

  • mod-rewrite
  • mod-proxy
  • mod-proxy-http
  • mod-proxy-balancer

to check if they are enabled issue ” /etc/init.d/apachectl -M ” on server, it will list all the enabled modules. For Debian systems all enabled modules are in /etc/apache2/mods-enabled directory and all available modules are in /etc/apache2/mods-available directory, to enable them issue ” a2enmod MOD_NAME “.

Now create the production database(on server) and update database.yml for production database settings.

After this configure mongrel by issuing ” mongrel_rails cluster::configure -e production -p 8000 -a 127.0.0.1 -N 2 -c ./ ” inside the rails application root directory(on client machine). This will create mongrel_cluster.yml in config directory. You can change parameters in this command, as -p 8000 specifies that mongrel instances will start up on port number starting 8000, -a 127.0.0.1 specifies that mongrel instances will listen to the localhost, -N specifies the number of mongrel instances and -c specifies the rails root directory.

Now its time to capistranize rails application, issue ” cap –apply-to ./ APP_NAME ” inside rails application root directory(on client machine), this will add two files(config/deploy.rb and lib/tasks/capistrano.rake) to the rails application. Edit deploy.rb according to your requirements. Also add the following code to deploy.rb :

task :restart, :roles => :app do
# stop mongrel clusters for previous release and start for current
run “cd #{previous_release} && mongrel_rails cluster::stop”
run “sleep 5″
run “cd #{current_release} && mongrel_rails cluster::start”
end

Now on your local machine issue these two commands in only once ” rake remote:setup ” and ” rake remote:cold_deploy “, when you issue ” rake remote:setup ” it will prompt for the server password and create necessary directories on the server and ” rake remote:cold_deploy ” will deploy your code to the server. Next time whenever you want to deploy to the server you just need to issue ” rake remote:deploy ” not ” rake remote:cold_deploy “.

Now we are just one step back, we need to configure apache proxy balancer for mongrel instances. Add the following code to the httpd.conf file:

BalancerMember http://127.0.0.1:8000
BalancerMember http://127.0.0.1:8001


ServerName myapp.com
DocumentRoot /rails_apps/app/public


Options FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

RewriteEngine On

RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ /system/maintenance.html [L]

RewriteRule ^/$ /index.html [QSA]
# Rewrite to check for Rails cached page
RewriteRule ^([^.]+)$ $1.html [QSA]

RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L]

you need to change the above code according to you requirement. Also you need to restart apache server by issuing ” /etc/init.d/apachectl restart “. Now you are done.

But we also need to add a script to start mongrel instances when the server restarts, otherwise whenever the server restart there will no mongrel instance running.

Just create a file named mongrel_clusters (you can choose any name) in /etc/init.d directory with the following code:

#!/bin/bash
#
# chkconfig: 345 94 16
# description: Startup script for mongrel
BASEDIR=/var/www/your_app
export HZ=100
export TERM=linux
export SHELL=/bin/bash
export HUSHLOGIN=FALSE
export USER=root
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
export MAIL=/var/mail/root
export _=/usr/bin/env
export PWD=/etc/init.d
export HOME=/root
export SHLVL=2
export LOGNAME=root

cd $BASEDIR
case “$1″ in
start)
echo “starting up mongrel in $BASEDIR”
mongrel_rails cluster::start
;;
stop)
echo “stopping mongrel”
mongrel_rails cluster::stop
;;
restart)
mongrel_rails cluser::stop
sleep 3
mongrel_rails cluster::start
;;
esac

You need to change the BASEDIR in the above code. Make this file executable by ” chmod +x /etc/init.d/mongrel_clusters
Issue these commands to add this script at system startup:

Debian: /usr/sbin/update-rc.d /etc/init.d/mongrel_clusters defaults
RedHat: /usr/sbin/chkconfig –add /etc/init.d/mongrel_clusters and /usr/sbin/chkconfig –level 2 /etc/init.d/mongrel_clusters on

Note: Use above instructions at your own risk, if your computer explodes its not my fault :-)

Configuring Multiple Rails Application with Lighttpd

I was using lighttpd web server for my Rails applications.
In development mode I was starting lighttpd server from the rails application directory by

lighttpd -D -f config/lighttpd.conf
But I was facing problem when I tried to run multiple rails applications in production mode. After some trials I succeded.

I inserted the following lines of code in my /etc/lighttpd/lighttpd.conf file

$HTTP["host"]== “domain.com” {

server.error-handler-404 = “/dispatch.fcgi”

server.document-root = “/home/railsapp/public/”

server.errorlog = “/home/railsapp/log/lighttpd.error.log”

accesslog.filename = “/home/railsapp/log/lighttpd.access.log”

url.rewrite = ( “^/$” => “index.html”, “^([^.]+)$” => “$1.html” )

fastcgi.server = ( “.fcgi” => ( “localhost” => (

“min-procs” => 1,

“max-procs” => 1,

“socket” => “/home/railsapp/tmp/sockets/fcgi.socket”,

“bin-path” => “/home/railsapp/public/dispatch.fcgi”,

“bin-environment” => ( “RAILS_ENV” => “production” )

) ) )

}

$HTTP["host"]== “another.domain.com” {

server.error-handler-404=”/dispatch.fcgi”

server.document-root = “/home/anotherrailsapp/sparitual/public/”

server.errorlog = “/home/anotherrailsapp/log/lighttpd.error.log”

accesslog.filename = “/home/anotherrailsapp/log/lighttpd.access.log”

url.rewrite = ( “^/$” => “index.html”, “^([^.]+)$” => “$1.html” )

compress.filetype = ( “text/plain”, “text/html”, “text/css”, “text/javascript” )

compress.cache-dir = “/home/anotherrailsapp/tmp/cache”

fastcgi.server = ( “.fcgi” => ( “localhost” => (

“min-procs” => 1,

“max-procs” => 1,

“socket” => “/home/anotherrailsapp/tmp/sockets/fcgi.socket”,

“bin-path” => “/home/anotherrailsapp/public/dispatch.fcgi”,

“bin-environment” => ( “RAILS_ENV” => “production” )

) ) )

}

Then started the lighttpd server by

lighttpd -D -f /etc/lighttpd/lighttpd.conf

And it worked…