Difficulty with pushing website code to web server, and what to do about it.Let's say you opened an account with a VPS provider like Linode or AWS and started hosting your own website. Not WordPress but a plain HTML/CSS/JS website to teach yourself coding. But in rush to publish your website, you din't automate deployment process but do the following to deploy your website.
- Apache uses /var/www/html/ as the html home directory of your site.
- You don't have a separate dev environment to preview your website as you code.
- To deploy code, you ftp up files, log into the web server, and manually copy files over.
- To do away with ftp/move-files to publish, you may even edit code directly on the web server.
All of this is very inefficient and takes your attention away from working on your code.
I struggled with improving this process and think I have come up with an efficient deployment procedure. This deployment procedure does not use git or Jenkins or complicated scripts. Git/Jenkins are required for most professional shops and would add immediate boost to your resume, but for small personal website, they may be overkill. And even if you want to use Git/Jenkins, going through this article will still help you learn about ways to improve code deployment process. With this process, multiple steps required for deploying your code will be turned into 2 steps, 2 very quick steps.
Basically I set up 2 separate websites (dev.yoursite.com and yoursite.com). While developing, I use dev.yoursite.com to preview. When ready to publish the website, I copy the files to yoursite.com.
- PC running Mac OS or Linux.
- Linux server running Apache 2.x (CentOS 6.x in this article)
- VirtualHost in Apache
- Single line of command in Mac OS's Terminal, rsync
- Single line of command in web server
- Add 1 A/AAAA DNS record for your website (ex: dev.yoursite.com)
- Basic file/folder manipulation on Linux via CLI
What's not discussed
- Dealing with WordPresss or other CMS
- Using Windows as web server
iTermIf you are still using Terminal on your Mac, download and install iTerm ver 2.x. iTerm ver 3.x was released recently but I recommend 2.x. Once iTerm ver 2.x is installed, open an iTerm window. Inside of iTerm window, right click and choose "Split Pane Vertically". Use left pane for working on your local mac. Use right pane for working on your web server. This will minimize switching between windows.
Back up your current website
I'm assuming you are already hosting a website on your own VPS node running Linux web server. If so, using right pane of iTerm, ssh log in to your web server and copy /var/www/html/ to /var/www/html/yoursite_p/.
Also expect a short downtime as we move HTML files from /var/www/html/ to /var/www/html/yoursite_p/ and configure VirtualHost.
How 2 environments are structured
You first need to update your DNS records so that you have dev.yoursite.com working. The DNS propagation can take a few minutes to a few hours. You can always use updated HOSTS file to proceed with the tutorial until DNS propagation is completed. Ping both yoursite.com and dev.yoursite.com. Check you get same IP.
Set up VirtualHost by creating Apache .conf files
You will set up 2 VirtualHost config files so 1 IP on your web server can host 2 separate websites: yoursite.com AND dev.yoursite.com.
Create 2 .conf files./etc/httpd/conf.d/yoursite.com_dev.conf
1 <VirtualHost *:80> 2 ServerAdmin [email protected] 3 ServerName dev.yoursite.com 4 ServerAlias dev.yoursite.com 5 DocumentRoot /var/www/html/yoursite_d 6 ErrorLog /var/log/httpd/error_d_yoursite_co_log 7 CustomLog /var/log/httpd/access_d_yoursite_co_log combined 8 ErrorDocument 404 /notfound.html/etc/httpd/conf.d/yoursite.com_production.conf
1 <VirtualHost *:80> 2 ServerAdmin [email protected] 3 ServerName yoursite.com 4 ServerAlias yoursite.com 5 DocumentRoot /var/www/html/yoursite_p 6 ErrorLog /var/log/httpd/error_p_yoursite_co_log 7 CustomLog /var/log/httpd/access_p_yoursite_co_log combined 8 ErrorDocument 404 /notfound.html
Create HTML directories and move files around
If you have your current production website in /var/www/html/, move those files to /var/www/html/yoursite_production/ temporarily.
- Create following 2 directories to hold HTML files.
- Production site: Move back your current website files from /var/www/html/yoursite_production/ to /var/www/html/yoursite_p/. Check the permissions are good.
- Dev site: Create /var/www/html/yoursite_d/index.html and add just add following line.
1 QA site
Restart Apache on web server to check Apache config change is working. You should be able to hit http://yoursite.com and see your website. When you browse to http://dev.yoursite.com, you should see
1 QA site
Rsync up files to deploy to Dev environment
With DNS record updated and VirtualHost set up, you will rsync up files from your Mac to the web server. When copy/moving large amount of files or a large file, always use rsync instead of cp/scp/mv commands. Here's the command (which you should modify first to match your environment) you will run on your Mac (left pane in iTerm).
1 rsync -ave ssh --exclude=".DS_Store" ~/codes/yoursite [email protected]:~/devsite
Before running the command, let's dissect the rsync command.
- You have already configured your SSH public key on your webserver and you can ssh into your webserver without typing in password. This is not a requirement but I highly recommend you set this up. Set it up once and avoid typing in your password 100 times.
- We use --exclude option to ensure .DS_Store file (hidden file created by Mac OS) is not rsynced up to the web server. I exclude other folders, but for clarity I only exclude .DS_Store here.
- In this example the html files for your website are kept on your Mac in following folder. Change it to match your environment.
- The rsync uploads files to user account jdoe's directory on the web server. Change the login name to match yours.
- The rsync destination directory, ~/devsite/, is actually /home/jdoe/devsite/. First, create the directory 'devsite' on your web server. Note we are not rsyncing up the files directly to /var/www/yoursite_d/ or /var/www/yoursite_p/.
- In the left side of your iTerm, run following command. Modify the command to match your environment and run it to rsync up the files. The first time you run it, it may take awhile. However subsequent rsync operations will be very quick.
- Go ahead and run rsync command on your Mac.
5 rsync -ave ssh --exclude=".DS_Store" ~/codes/yoursite [email protected]:~/devsite
Deploy dev files
Once the files are rsynced up to the web server, deploy the html files you are actively developing. You need to run some commands (in the right pane of iTerm) on the web server to copy files in and set up permissions. Note that although the command appears to be 3 lines long, it's actually just a long command broken up into 3 lines using \ at the end of 1st and 2nd line. Copy and paste all 3 lines with 1 copy/paste
1 $ tar -czf /tmp/html.tar.gz /data/www/yoursite_d && rm -rf /data/www/yoursite_d/*" \ 2 rsync -av ~/devsite/yoursite/* /data/www/yoursite_d/ \ 3 chmod 750 -R /data/www/yoursite_d/ && chown -R apache: /data/www/yoursite_d/
- Line 1: Tars up /data/www/yoursite_d/ and copies the tar to /tmp/html.tar.gz, for backup. At this point, /data/www/yoursite_d/ has no files. But after this initial deploy, it will have files and it's always a good idea to backup files.
- Line 2: Copy files from /home/jdoe/devsite/yoursite/ to /data/www/yoursite_d/.
- Line 3: Set up file permissions.
Test dev site dev.yoursite.com.
Test browsing http://dev.yoursite.com and you should see the files you just uploaded.
Repeat deploying Dev website.
Now let's test updating a file on your Mac and go through deploying the files to dev.yoursite.com. What used to take multiple steps (ie. ftp files or waiting for files to be uploaded or setting permissions) is now just 2 actions.
- On your Mac, change something in a file in ~/codes/yoursite/. For example, add some text in ~/codes/yoursite/index.html.
- In the left pane of iTerm, use UP arrow to show the previous command you ran to upload the files and run it. In this case it would be something like this: *update to match your environment
- You will note that the file transfer upto your server is much faster now.
- Next, switch to the right pane of iTerm to work on your web server.
- Use UP arrow to show the previous command you used and run it again. In this case it would be something like this : *update to match your environment
- Test browsing http://dev.yoursite.com and you should see the change you just made in index.html is now present on the web server. The deployment step for dev environment literally just took seconds to complete, with 2 keyboard commands.
1 rsync -ave ssh --exclude=".DS_Store" ~/codes/yoursite [email protected]:~/devsite
1 tar -czf /tmp/html.tar.gz /data/www/yoursite_d && rm -rf /data/www/yoursite_d/*" \ 2 rsync -av ~/devsite/yoursite/* /data/www/yoursite_d/ \ 3 chmod 750 -R /data/www/yoursite_d/ && chown -R apache: /data/www/yoursite_d/\
Deploy code to production environment
Once you are ready to deploy latest version of you website to production environment, on web server you can use a slightly modified version of the command used for deploying dev environment. The modified command would look like this:
1 $ tar -czf /tmp/html_p.tar.gz /data/www/yoursite_p && rm -rf /data/www/yoursite_p/*" \ 2 rsync -av /data/www/yoursite_d/* /data/www/yoursite_p/ \ 3 chmod 750 -R /data/www/yoursite_p/ && chown -R apache: /data/www/yoursite_p/
- Line 1: It tars /data/www/yoursite_p to /tmp/html_p.tar.gz to back it up. And then clears out content of /data/www/yoursite_p/ (where you production version of your website).
- Line 2: Copy files from /data/www/yoursite_d/ (where you keep dev version) to /data/www/yoursite_p/.
- Line 3: Set permissions on files in /data/www/yoursite_p/. Permissions from /data/www/yoursite_d/ should work, but I reapply permissions just to make sure.
- Check yoursite.com and you will note your code has been deployed successfully.
This deploy procedure works, improvement can be made. It will work even better if you used git. That will be my next article.