Buy @ Amazon

Search This Blog

March 31, 2011

Consolidating DB migration in Rails

In my current project we used to maintain a good number of rails migration scripts until, one guest developer stepped into the team for a short while and shared his knowledge on consolidating Rails DB migrations which I intend to blog about in this post.

So what is consolidating DB migrations all about? So if you have the DB migrations say, 1_aaa.rb, 2_bbb.rb and 3_ccc.rb that has gone into your production release 1.00, post your successful release (or a day or two after that), you can consolidate the scripts to something, say 3_consolidated_migrations.rb, wherein this new script essentially borrows the id (here it's 3) from the last run migration in production release and the contents being the same as schema.rb

So what are the implications of this? Well you need to keep in mind of the following:

  1. Without this approach being adopted, we can just about run the DB migrations on DB dump irrespective of its snapshot date. To put it other way, adopting this practice implies that you'll necessarily have to get the snapshot of Production DB that is no behind the release - say, if you have made released v3, then the DB snapshot has to be after release v3 is made successful. 
  2. If you had written data migration scripts, say from a table1.column1 to table2.column2, these would be lost. However, is there a point of having them in the scripts beyond successful release to production? In most of the occassions, your answer will be NO. However, if you have a compelling reason to have such data migration scripts in Rails DB migrations, then you may not choose DB consolidation.
  3. This one is very important to reckon with. If your DB migrations have scripts related to DB constraints - primary and foreign key, then all these will be lost with DB Consolidation, as we don't have rails api that is DB agnostic. As a work around you may maintain these scripts in separate DB specific .sql file - something like oracle_db_constraints.sql. In our project, whenever we had to introduce this kind of constraints, we had it both in DB migrations script specific for that release and add the same to the list of SQL(s) in the .sql file(s) that we maintain separately.
  4. Some Rails team adopt the practice of including scripts for seed data insertion, in the db  migrations scripts. This will be lost with DB consolidation, which in my opinion is a good one because it then forces the team to have a seperate file for this purpose (remember 'Seperation of Concerns' here). The other approach is to exclude this specific migration file from DB consolidation code based on this file pattern, which in my humble opinion is not bad thing.

March 30, 2011

tar command (with compress option)

Reading through the Unix's man or manual for a command description may be a little pain but with repeated practice you get used to it. In this blog I mix the manual's content and add my own example which might be of frequent use in general.


tar [-] A --catenate --concatenate |
           c --create |
           d --diff --compare |
              --delete |
           r --append |
           t --list |
             --test-label |
           u --update |
           x --extract
              --get [options] [pathname ...]


Tar stores and extracts files from a tape or disk archive. The first argument to tar should be a function; either one of the letters Acdrtux, or one of the long function names.

Some options take a parameter; with the single-letter form these must be given as separate arguments. With the long form, they may be given by appending = value to the option.


-A, --catenate, --concatenate
     append tar files to an archive
-c, --create
     create a new archive
-d, --diff, --compare
     find differences between archive and file system
     delete from the archive (not on mag tapes!)
-r, --append
     append files to the end of an archive
-t, --list
     list the contents of an archive
     test the archive volume label and exit
-u, --update
     only append files newer than copy in archive
-x, --extract, --get
     extract files from an archive


-a, --auto-compress
     use archive suffix to determine the compression program
     add given FILE to the archive (useful if its name starts with a dash)
     preserve access times on dumped files, either by restoring the times
     do not use archive suffix to determine the compression program
-C, --directory DIR
     change to directory DIR
     display progress messages every NUMBERth record (default 10)
     exclude version control system directories
-f, --file ARCHIVE
     use archive file or device ARCHIVE
     archive file is local even if it has a colon
-h, --dereference
     follow symlinks; archive and dump the files they point to
     follow hard links; archive and dump the files they refer to
     ignore case
     case sensitive matching (default)
-k, --keep-old-files
     don't replace existing files when extracting
     don't replace existing files that are newer than their archive copies
-X, --exclude-from FILE
     exclude patterns listed in FILE
-z, --gzip, --gunzip --ungzip
     filter the archive through gzip
-Z, --compress, --uncompress
     filter the archive through compress


Create .tar archive from files foo and bar
     $ tar -cf archive.tar foo bar
List all files in .tar archive verbosely
     $ tar -tvf archive.tar
Extract all files from .tar archive
     $ tar -xf archive.tar
Create zipped .tar archive from /home/karthik/myprojects
     $ tar -zcvf myprojects.tar.gz /home/karthik/myprojects
Unzip and Restore an archive myprojects.tar.gz to say current directory
     $ tar -zxvf myprojects.tar.gz
Unzip and Restore an archive myprojects.tar.gz to an existing directory say /temp
     $ tar -zxvf myprojects.tar.gz /temp