Entries tagged as geek
Sticky Postings
Topbit Technicals - what to expect
Clearly, the world needs another blog, and especially one by an arrogant opinionated developer that thinks he knows better than most. This is that blog
While I’ve had my LJ for a while, that has never been the right place for some highly technical posts, so, frankly years late, I’ve decided to start this up. This will be the home for my technical discussions, suggestions, links to some very interesting posts I’ve seen elsewhere (hopefully good), and more than the odd rant, about other developers, technologies and anything else that’s vaguely related.
If it involves the flowing of electrons, it’s pretty much fair game.
Sunday, March 9. 2008
svn checkouts vs exports for live versions
I've just read http://www.svn-checkout.co.uk/2008/01/19/how-to-release-new-versions-of-websites/ via http://www.lornajane.net/posts/2008/SVN-Deployment-and-a-New-Site and while I consider revision control an essential tool (my current job is the only one in the last five years where I didn't have to install my own RCS, but in the year I've been here so far, I've added nearly 1000 revisions), I somewhat disagree on the idea they suggest.
That first link, 'how to release new versions of websites', suggests checking out a version of the site as a working copy, (it's certainly something I've done before now), but then it goes on to use the 'svn switch' capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I'd run a server on Windows - not even for testing). There is however some trouble with revision switching - especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits - the new code goes into the repository all at once, or not, it's harder to do the all at once part on a non-transactional file-system - such as a webserver. The bigger problem is that rolling back will also take time - and in an emergency, the time taken to do something is crucial.
Here's what I do - Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I'll update run the script below with the specific revision number to export (and a date/time, but that's just for easy reference). When it runs, it also symlinks the given version as 'dev' - which is part of the path to the site 'dev.example.com', wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with 'rm dev && ln -s 1234.20080102 dev'). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it's just as easy to put a similar 'live' symlink into place with a new link.
If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine's own hostname (more useful for CLI scripts, generally run from cron).
There are some downsides - with a complete checkout of a 40-some megabyte website (it's mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it's a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster - and the images aren't being served from Apache, but a dedicated image webserver, that's not a significant issue - and even on Apache we don't have Etags enabled (Yslow from Yahoo also suggests that).
That first link, 'how to release new versions of websites', suggests checking out a version of the site as a working copy, (it's certainly something I've done before now), but then it goes on to use the 'svn switch' capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I'd run a server on Windows - not even for testing). There is however some trouble with revision switching - especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits - the new code goes into the repository all at once, or not, it's harder to do the all at once part on a non-transactional file-system - such as a webserver. The bigger problem is that rolling back will also take time - and in an emergency, the time taken to do something is crucial.
Here's what I do - Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I'll update run the script below with the specific revision number to export (and a date/time, but that's just for easy reference). When it runs, it also symlinks the given version as 'dev' - which is part of the path to the site 'dev.example.com', wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with 'rm dev && ln -s 1234.20080102 dev'). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it's just as easy to put a similar 'live' symlink into place with a new link.
If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine's own hostname (more useful for CLI scripts, generally run from cron).
There are some downsides - with a complete checkout of a 40-some megabyte website (it's mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it's a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster - and the images aren't being served from Apache, but a dedicated image webserver, that's not a significant issue - and even on Apache we don't have Etags enabled (Yslow from Yahoo also suggests that).
#!/bin/sh
REV=1914
REVDIR=$REV.20080214.1914
REPOSITORY=svn+ssh://username@svn.example.com/var/svn/SITEURL/trunk
IMAGES=$REVDIR/htdocs/i
STATIC1=$REVDIR/example.co.uk
svn export --revision $REV $REPOSITORY $REVDIR
mkdir -p $REVDIR/tmp/templates_c
chown -R www-data: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/htdocs/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/htdocs/cache/ $IMAGES $STATIC1
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php
# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x
ls -l $IMAGES/* | grep -- "-x"
# make the latest version available at http://dev.example.com
rm dev && ln -s $REVDIR dev
REV=1914
REVDIR=$REV.20080214.1914
REPOSITORY=svn+ssh://username@svn.example.com/var/svn/SITEURL/trunk
IMAGES=$REVDIR/htdocs/i
STATIC1=$REVDIR/example.co.uk
svn export --revision $REV $REPOSITORY $REVDIR
mkdir -p $REVDIR/tmp/templates_c
chown -R www-data: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/htdocs/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/htdocs/cache/ $IMAGES $STATIC1
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php
# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x
ls -l $IMAGES/* | grep -- "-x"
# make the latest version available at http://dev.example.com
rm dev && ln -s $REVDIR dev

Comments
Mon, 24.03.2008 19:30
Sun, 16.03.2008 23:45
Sun, 16.03.2008 21:31
Although example shown was usi ng Windows (my desktop) there is no reason why this can’t be used on Linux or as we [...]