Entries tagged as linux
Sunday, March 16. 2008
Always have up to date documentation, part #1
As I mentioned in my second post, ZCE prep – and dumb tests – about open book tests (like Brainbench), having a copy of all the relevant documentation can be incredibly useful, if only from a speed issue. Knowing you can just open a new tab and type a few words to get the information on a function, or concept from the manual takes away so many problems.
I mentioned there that I have a local copy of the main PHP manual – and I wanted to tell you how I keep it, and a couple of other manuals up to date, as well as other documentation.
Sunday, March 9. 2008
svn checkouts vs exports for live versions
I've just read http://www.svn-checkout.co.uk/2008/01/19/how-to-release-new-versions-of-websites/ via http://www.lornajane.net/posts/2008/SVN-Deployment-and-a-New-Site and while I consider revision control an essential tool (my current job is the only one in the last five years where I didn't have to install my own RCS, but in the year I've been here so far, I've added nearly 1000 revisions), I somewhat disagree on the idea they suggest.
That first link, 'how to release new versions of websites', suggests checking out a version of the site as a working copy, (it's certainly something I've done before now), but then it goes on to use the 'svn switch' capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I'd run a server on Windows - not even for testing). There is however some trouble with revision switching - especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits - the new code goes into the repository all at once, or not, it's harder to do the all at once part on a non-transactional file-system - such as a webserver. The bigger problem is that rolling back will also take time - and in an emergency, the time taken to do something is crucial.
Here's what I do - Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I'll update run the script below with the specific revision number to export (and a date/time, but that's just for easy reference). When it runs, it also symlinks the given version as 'dev' - which is part of the path to the site 'dev.example.com', wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with 'rm dev && ln -s 1234.20080102 dev'). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it's just as easy to put a similar 'live' symlink into place with a new link.
If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine's own hostname (more useful for CLI scripts, generally run from cron).
There are some downsides - with a complete checkout of a 40-some megabyte website (it's mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it's a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster - and the images aren't being served from Apache, but a dedicated image webserver, that's not a significant issue - and even on Apache we don't have Etags enabled (Yslow from Yahoo also suggests that).
That first link, 'how to release new versions of websites', suggests checking out a version of the site as a working copy, (it's certainly something I've done before now), but then it goes on to use the 'svn switch' capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I'd run a server on Windows - not even for testing). There is however some trouble with revision switching - especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits - the new code goes into the repository all at once, or not, it's harder to do the all at once part on a non-transactional file-system - such as a webserver. The bigger problem is that rolling back will also take time - and in an emergency, the time taken to do something is crucial.
Here's what I do - Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I'll update run the script below with the specific revision number to export (and a date/time, but that's just for easy reference). When it runs, it also symlinks the given version as 'dev' - which is part of the path to the site 'dev.example.com', wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with 'rm dev && ln -s 1234.20080102 dev'). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it's just as easy to put a similar 'live' symlink into place with a new link.
If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine's own hostname (more useful for CLI scripts, generally run from cron).
There are some downsides - with a complete checkout of a 40-some megabyte website (it's mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it's a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster - and the images aren't being served from Apache, but a dedicated image webserver, that's not a significant issue - and even on Apache we don't have Etags enabled (Yslow from Yahoo also suggests that).
#!/bin/sh
REV=1914
REVDIR=$REV.20080214.1914
REPOSITORY=svn+ssh://username@svn.example.com/var/svn/SITEURL/trunk
IMAGES=$REVDIR/htdocs/i
STATIC1=$REVDIR/example.co.uk
svn export --revision $REV $REPOSITORY $REVDIR
mkdir -p $REVDIR/tmp/templates_c
chown -R www-data: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/htdocs/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/htdocs/cache/ $IMAGES $STATIC1
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php
# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x
ls -l $IMAGES/* | grep -- "-x"
# make the latest version available at http://dev.example.com
rm dev && ln -s $REVDIR dev
REV=1914
REVDIR=$REV.20080214.1914
REPOSITORY=svn+ssh://username@svn.example.com/var/svn/SITEURL/trunk
IMAGES=$REVDIR/htdocs/i
STATIC1=$REVDIR/example.co.uk
svn export --revision $REV $REPOSITORY $REVDIR
mkdir -p $REVDIR/tmp/templates_c
chown -R www-data: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/htdocs/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/htdocs/cache/ $IMAGES $STATIC1
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php
# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x
ls -l $IMAGES/* | grep -- "-x"
# make the latest version available at http://dev.example.com
rm dev && ln -s $REVDIR dev

Comments
Wed, 28.10.2009 20:56
Multiple workers running is no t a problem – Beanstalkd will keep the jobs separate even if multiple ones are reser [...]
Wed, 28.10.2009 19:47
Is there any way to run more t he 1 worker (bash script) at a time?
Fri, 26.06.2009 01:13
Hi, What kind of beanstalk client library do you use or h ave you written your own? Is i t in PHP space or a C ex [...]
Mon, 22.06.2009 22:28
Hi, Good post. I am also pl aying with beanstakd and I am waiting for the rest of your p osts. Greetings, Alf [...]
Mon, 24.03.2008 19:30
Sun, 16.03.2008 23:45
Sun, 16.03.2008 21:31
Although example shown was usi ng Windows (my desktop) there is no reason why this can’t be used on Linux or as we [...]