How to deploy code from git on Linux

Posted by paul on 2014.07.19

Deploy code from Git repo to Linux using a shell script

2014.07.19 Sat

In this blog I will explain how to deploy code from GIT Repo to either QA and Production environments on Linux web servers, with a simple 2-step procedure. It will be fast, simple and error free.

To deploy code from my GIT repo to a Linux web server in the past, 1) I would add a tag (ex. ver.0.11) in GIT repo, 2) EDIT the script on Linux server to match the ver, and 3) run the script which GIT pulls with code tagged with (ex. ver.0.11) on the Linux server. However I did not like that I had to edit the script to match new GIT tag on Linux every time I was deploying code. I got rid of this manual step by using Typinator on Mac. And I also get a good history of code deployed over time. If something went wrong, I can edit the script to match date/time of earlier deployment and rerun the script to revert to a previous version.

Assumptions

  1. My PC is a Mac and it has the app Typinator.
  2. Using Typinator with a few simple configs, you can quickly type in something like dd.h (can be configured to anything you like) which is then converted to something similar to 20140719.11 (it shows year - month - day and hour in 24hr format). It will save a few seconds of your time every time you use it and the time saving adds up. It is a great tool to have on a Mac. Make sure Typinator pads with a zero when necessary. For July 9th, 2014, you want 20140709.11, NOT 201479.11.
  3. On my Linux server, web site files are kept in /data/www/.
  4. For my simple small site, 2 separate folders are used to keep QA and production version of coders.com separated. QA version is kept in /data/www/coders_qa/. Production version is kept in /data/www/coders/.
  5. I have DNS configured so that I can reach QA version (/data/www/coders_qa/) by gong to http://qa.coders.com. To reach the production version, I would go to http://coders.com (/data/www/coders/).
  6. I can ssh into the Linux server and run shell scripts.
  7. The Linux server needs to use Local Time zone, essentially to match the timezone your PC is in.

Procedure summary

  1. I add a tag (ex.: qa.20140719.11 OR prod.20140719.11) in my codebase when ready to either test in QA environment or publish to production environment.
  2. I run a script on my remote Linux server which pulls code that's tagged (ex.: qa.20140719.11 OR prod.20140719.11) and copies files to appropriate folder.

Prepare Linux server

You want to set up the Linux server to use Local Time zone, essentially to match the timezone your PC is in.

QA ready code deploy: On Linux server, add following script to deploy QA ready code. Name it coders-qa-deploy.sh

#!/bin/bash
#This script is used to deploy qa ready codes to http://qa.coders.com.
############
# Variables you MUST check/update.
############
# GIT related variables
REPO="coders"    #aka website
TARGET="coders_qa"   # QA ready code
CODES="/root/codes"

# Specify the number of set of older codes you want to keep. Used to keep down folder size.
# Insurance policy just in case something goes wrong and you need to recover code from previous
# deployment. Probably not necessary but better to be safe.
FOLDERS_TO_KEEP=5

# Array to hold list of folders containing old code
# I keep my Apache html files in /data/www/
OLD_FOLDER=/data/www/$TARGET-old
OLD_FOLDER_ARRAY=($OLD_FOLDER/*)

############
# Variables you do not need to update.
############
TAGDATE=$(date +"%Y%m%d.%H")
TAG="qa.$TAGDATE"
# Date/time stamp
NOW=$(date +"%Y-%m-%d_%H-%M")
NOWSTAT=$NOW
# This is necessary because of the way 'tail -n +6' works. Nothing to change here.
FOLDERS_VALUE=$(($FOLDERS_TO_KEEP + 1))
 
############
# Script
############
# Get files from GIT Repo.

mkdir $CODES > /dev/null 2>&1
cd $CODES
rm -rf $CODES/$REPO
git clone [email protected]:pauljchu/$REPO.git
cd $CODES/$REPO/
 
# Git switches to a tag.
# It probably would be tagged as "qa.20140719.11"
git checkout $TAG
 
# Move old $REPO site to $TARGET-old
mkdir -p /data/www/$TARGET-old/$NOWSTAT/ > /dev/null 2>&1
mv /data/www/$TARGET/* /data/www/$TARGET-old/$NOWSTAT/ > /dev/null 2>&1
 
# Delete old code from previous deployment and keep only newer 5 versions.
for i in ${OLD_FOLDER_ARRAY[@]}
do
    /bin/ls -dt $OLD_FOLDER/* | /usr/bin/tail -n +$FOLDERS_VALUE | /usr/bin/xargs /bin/rm -rf  #keep only latest 3 sets and delete older ones
done
echo "Only  $FOLDERS_TO_KEEP  newer subfolders are now in $OLD_FOLDER/."
 
#Copy in new files into Apache home directory and set file permissions.
mkdir /data/www/$TARGET/ > /dev/null 2>&1
rsync -a --exclude=.DS_Store --exclude=.git --exclude=.gitignore --exclude=.idea --exclude=.name $CODES/$REPO/* /data/www/$TARGET/ > /dev/null 2>&1
cd /data/www/$TARGET/
chown -R apache:apache /data/www/$TARGET/
chmod -R 750 /data/www/$TARGET/
exit 0

Production ready code deploy: On Linux server, add following script to deploy Production ready code. Name it coders-prod-deploy.sh. Only difference between coders-qa-deploy.sh and coders-prod.deploy.sh is the value of the variable 'TAG'.

#!/bin/bash
#This script is used to deploy qa ready codes to http://qa.coders.com.
############
# Variables you MUST check/update.
############
# GIT related variables
REPO="coders"    #aka website
TARGET="coders"   # Production ready code
CODES="/root/codes"

# Specify the number of set of older codes you want to keep. Used to keep down folder size.
# Insurance policy just in case something goes wrong and you need to recover code from previous
# deployment. Probably not necessary but better to be safe.
FOLDERS_TO_KEEP=5

# Array to hold list of folders containing old code
# I keep my Apache html files in /data/www/
OLD_FOLDER=/data/www/$TARGET-old
OLD_FOLDER_ARRAY=($OLD_FOLDER/*)

############
# Variables you do not need to update.
############
TAGDATE=$(date +"%Y%m%d.%H")
TAG="prod.$TAGDATE"  
# Date/time stamp
NOW=$(date +"%Y-%m-%d_%H-%M")
NOWSTAT=$NOW
# This is necessary because of the way 'tail -n +6' works. Nothing to change here.
FOLDERS_VALUE=$(($FOLDERS_TO_KEEP + 1))
 
############
# Script
############
# Get files from GIT Repo.
mkdir $CODES > /dev/null 2>&1
cd $CODES
rm -rf $CODES/$REPO
git clone [email protected]:pauljchu/$REPO.git
cd $CODES/$REPO/
 
# Git switches to a tag.
# It probably would be tagged as "qa.20140719.11"
git checkout $TAG
 
# Move old $REPO site to $TARGET-old
mkdir -p /data/www/$TARGET-old/$NOWSTAT/ > /dev/null 2>&1
mv /data/www/$TARGET/* /data/www/$TARGET-old/$NOWSTAT/ > /dev/null 2>&1
 
# Delete old code from previous deployment and keep only newer 5 versions.
for i in ${OLD_FOLDER_ARRAY[@]}
do
    /bin/ls -dt $OLD_FOLDER/* | /usr/bin/tail -n +$FOLDERS_VALUE | /usr/bin/xargs /bin/rm -rf  #keep only latest 3 sets and delete older ones
done
echo "Only  $FOLDERS_TO_KEEP  newer subfolders are now in $OLD_FOLDER/."
 
#Copy in new files into Apache home directory and set file permissions.
mkdir /data/www/$TARGET/ > /dev/null 2>&1
rsync -a --exclude=.DS_Store --exclude=.git --exclude=.gitignore --exclude=.idea --exclude=.name $CODES/$REPO/* /data/www/$TARGET/ > /dev/null 2>&1
cd /data/www/$TARGET/
chown -R apache:apache /data/www/$TARGET/
chmod -R 750 /data/www/$TARGET/
exit 0

Prepare on Mac

Set up Typinator and set it up so that if you type in dd.h (or any other combo you like), it will be converted into 20140719.11 (year-month-day.24hour). You obviously could simply type in year-month-day.24hour in manually. But I find Typinator very useful.

Deploy to QA environment

Let's say you are ready to deploy code to QA environment.

  1. On your PC, tag your code. When naming the code, type in qa.dd.h which will be converted to something like qa.20140719.11.
  2. Immediately after, on your Linux machine run coders-qa-deploy.sh.
  3. In coders-qa-deploy.sh the name of the TAG is automatically determined to be: qa.20140719.11.
  4. The script coder-qa-deploy.sh will finish deploying the code to your QA environment.

Deploy to Production environment

Let's say you are ready to publish code to Production environment.

  1. On your PC, tag your code. When naming the code, type in prod.dd.h which will be converted to something like prod.20140719.11.
  2. Immediately after, on your Linux machine run coders-prod-deploy.sh.
  3. In coders-prod-deploy.sh the name of the TAG is automatically determined to be: prod.20140719.11.
  4. The script coder-prod-deploy.sh will finish deploying the code to your Production environment.

Revert to older version of code when necessary

Let's say something went wrong and you need to return to a previous version of your code, maybe even a few versions back. To do so, just look through your GIT repo's tags and look for one with the date/time stamp you want to revert to. Once you decide on one, edit coders-prod-deploy.sh and change value of TAG to the date/time of the TAG name you want to revert to. In this example, I am assuming you want to return to the code tagged on June 1st, 2014 sometime between noon and 12:59pm.

BEFORE

TAG="prod.$TAGDATE"

AFTER editing to rever to the specific version deployed on June 1st, 2014

# TAG="prod.$TAGDATE"
TAG="prod.20140601.12"

With the change done, on Linux run coders-prod-deploy.sh and your website will be reverted to the version that was deployed on June 1st, 2014, between noon and 1PM.

Once you are ready to go back to releasing latest fixed code, you just need to edit coders-prod-deploy.sh as shown below and rerun the script again.

TAG="prod.$TAGDATE"
# TAG="prod.20140601.12"

That is it!