Topbit Technicals - php

Doing the work elsewhere - Adding a job to the queue

topbit+s9y@gmail.com (Topbit) — Tue, 06 Oct 2009 23:30:26 +0100

I've previously shown you why you may want to put some tasks through a queuing system, what sort of jobs you could define, plus how to keep a worker process running for as long as you would like (but still be mindful of problems that happen).

In this post, I'll show you how to put the messages into the queue, and we'll also make a start on reading them back out.

Continue reading "Doing the work elsewhere - Adding a job to the queue"

Doing the work elsewhere - Sidebar - running the worker

topbit+s9y@gmail.com (Topbit) — Tue, 23 Jun 2009 22:19:33 +0100

I'm taking a slight diversion now, to show you how the main worker processor runs. There are two parts to it - the actual worker, written in PHP, and the script that keeps running it.
Continue reading "Doing the work elsewhere - Sidebar - running the worker"

Doing the work elsewhere - Asynchronous Message Queues

topbit+s9y@gmail.com (Topbit) — Wed, 10 Jun 2009 16:43:29 +0100

The use of Beanstalkd as a queueing system

What is an asynchronous queue

The classic wikipedia quote (Message queue)

In computer science, message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content. Group communication systems provide similar kinds of functionality.

So one part of a system puts a message into a queue for another part to read from, and then act upon. The asynchronous nature means that each side is otherwise independent from the other, and does not wait for a response. That independence is an important part of the nature of the system though – and we’ll see later how some of the more advanced functionality for our software of choice here can give some extraordinary flexibility to what can be done.

Continue reading "Doing the work elsewhere - Asynchronous Message Queues"

(me instanceOf ZCE) === true

topbit+s9y@gmail.com (Topbit) — Sat, 06 Jun 2009 12:50:00 +0100

Phew. That would have been embarrassing if I’d not passed my ZCE on Thursday afternoon.

The rant on how useful it actually is though, will be coming early next week.

I laugh at your ZCE exam prep tests #2

topbit+s9y@gmail.com (Topbit) — Sat, 30 May 2009 15:20:23 +0100

Back at the PHP London Conference at the end of February, iBuildings was offering a little test, with prize for people that could do well answering the sort of questions that are on the ZCE exam. Never one to turn down something for free, I took ten minutes to answer the eight questions. A few weeks later, I get an email from them/Zend to say I’d won the chance to take an exam – ZCE, or ZFE (Zend Framework). Although I use ZF, I don’t know it well enough to begin to pass any exam, so as I’ve still not had the chance to take it, I figured, why not take it on their dime?

About 14 months ago, I’d bought 5 tries on the PHPArch-based ‘Vulcan’ test prep exam. Today, I’ve come back to it, and gone through it again. Like last time, the test (practice and real) is scheduled to take up to 90 minutes, but I had whipped through them all in 45 minutes, I have finished the 70 questions.

I’m amused by the fact the only part of this I failed was ‘Basic Language’. The first time around it was design patterns. Either way, now I’ve got some time, I’m going to schedule the test for quite possibly later this week and see about getting the paperwork for it.

It’s also still 7 ‘EXCELLENT’s, and a fail – just in different places :-)

Category	Grade
XML & Web Services	PASS
Arrays	PASS
Web Features	EXCELLENT
Basic Language	FAIL
Streams and Network Programming	PASS
Database Access	PASS
String Manipulation and Regular Expressions	EXCELLENT
PHP 4/5 differences	EXCELLENT
Security	EXCELLENT
OOP	EXCELLENT
Functions	EXCELLENT
Design	EXCELLENT

Overall : EXCELLENT

Elephpants on parade

topbit+s9y@gmail.com (Topbit) — Thu, 18 Sep 2008 17:49:48 +0100

After my little trip out yesterday to Google Dev Day 2008 (London) at Wembley – I thought I’d post some pics I took of the day, or more precisely, of what my Elephpant got up to. You can see them on the Elephpant at GoogleDevDay08 Flickr set page.

Also, some link love to to http://www.elephpantworldtour.com/ for the idea of taking the cute little blue guy along.

Riddled me that

topbit+s9y@gmail.com (Topbit) — Thu, 24 Apr 2008 20:56:18 +0100

Well go figure. I’ve just won $50 (Canadian, that’s about $3000 USD by now) of books and ‘stuff’ from PHP Arch, care of it’s publisher, Marco Tabini’s, blog

He’d put a little puzzle up last night, some long numbers, and a few short. I recognised them as almost ISBNs – it wasn’t hard to figure them as having dropped a zero from the front, making them “php|architect’s Guide to Programming with Zend Framework” and “php|architect’s Zend PHP 5 Certification Study Guide, 2nd Edition”. From there, guessing the other numbers were page, line and word counts was easy.

So, what should I buy? I’ve already got a subscription to the magazine – PDF edition (it’s so much easier to ship bits over the atlantic…).

Always have up to date documentation, part #2

topbit+s9y@gmail.com (Topbit) — Mon, 24 Mar 2008 15:38:28 +0000

see my previous post on the topic, #1.

My last post ended up more as a how-to than what-to. This time, I’ll say why you should have local copies of the documentation for most of the tools you use. I’ll also tell you the sort of things I always have handy as well.

Getting a local copy of php.net – and getting installed as an apache vhost and updated (probably weekly) is some effort, but well worth it. I’ve said it before, but PHP.net is the best language reference site that I’ve seen. It’s kept up to date (sometimes ahead of the code releases in fact) and while the notes that are added to it can sometimes confuse, as much as help, when they do help, they will really make the difference.

I don’t tend to buy many PHP books, because what can they do besides re-iterate what is is already there?

The most important thing to bear in mind though is not to just have the documentation there to read – you have to know what is available. Projects like PHP, the Zend Framework and PHPUnit have a lot of parts – and knowing that they have things – even if you don’t know how they work right now, can save you days or weeks of effort.

It’s for that reason that you need to at least scan over all the the docs you have – and indeed for all the libraries and tools that you use. Even I don’t read everything and expect to remember it all – but I remember enough to recognise that a paticular tool might have something to help – maybe PHP has something to search the values in an array (http://php.net/array-search), or can use Oracle, or Ldap, or Memcached, or that Zend Framework can let you easily loop over maildirs (or an mbox) to get each mail from within it. If you don’t read the manual – at least skimming over it, you would never know that functionality exists, and you inevitably end up reimplementing other people’s already debugged code. That’s a waste of your time.

So, take an hour now, and assemble a directory to put these docs into, and read through them – not everything, but at least look at headers of every section, just to get an idea of what is available and maybe go back and read up some more on things that may be useful to you. If something isn’t so interesting to you now, do bear in mind, your next project, or job, might change that.

Above all, keep learning. Never stop.

Continue reading "Always have up to date documentation, part #2"

Know thy tools first of all

topbit+s9y@gmail.com (Topbit) — Mon, 17 Mar 2008 20:09:22 +0000

Just a quick tip here, and I’ll expand on it below the cut.

When you have a library, like PEAR or Zend Framework – or ven just the whole PHP language library – it’s absolutely vital you know what it can do.

What you don’t know can cost you weeks of effort and pain. I found this out (again) today, but it’s not my pain – it’s an employee who was too busy deciding that the Zend Framework wasn’t suitable for a simple cron-script task, he has spent most of the last few weeks duplicating something that is not as good as what I could write – with ZF – in about an hour.

Continue reading "Know thy tools first of all"

Always have up to date documentation, part #1

topbit+s9y@gmail.com (Topbit) — Sun, 16 Mar 2008 23:41:00 +0000

As I mentioned in my second post, ZCE prep – and dumb tests – about open book tests (like Brainbench), having a copy of all the relevant documentation can be incredibly useful, if only from a speed issue. Knowing you can just open a new tab and type a few words to get the information on a function, or concept from the manual takes away so many problems.

I mentioned there that I have a local copy of the main PHP manual – and I wanted to tell you how I keep it, and a couple of other manuals up to date, as well as other documentation.

Continue reading "Always have up to date documentation, part #1"

ZCE prep - practice test #1

topbit+s9y@gmail.com (Topbit) — Thu, 13 Mar 2008 21:33:00 +0000

Well, I’ve just completed the PHP Arch ‘Vulcan’ practice test – the first of up to five such practice tests I’ve purchased. I have quite deliberately not gone through what study materials I have on hand before I took this test (I wanted to get a baseline), but none the less got an ‘EXCELLENT’ final score, and the same ‘Excellent’ on seven of the twelve sections the pre-test is broken down into. ‘Pass’ on four others, and just one ‘Fail’ on the design (patterns) section. If the real test worked much the same – and with a composite score, rather than having to pass all sections – I doubt I would have a problem to have gotten a passing grade.

Although the test (practice and real) is scheduled to take up to 90 minutes, after 45, I have finished the 70 questions. I’d set about half-dozen to review, but I don’t think I changed any of those answers, and so after a little more than 50 minutes – I called for the results.

It was much as I expected, with just some occasional requirements to know some parameter orders and specific use (the kinda thing where you test it, and if you didn’t get it right first time, I would trivially look it up to check, for functions like sub-string searching). Also several questions on XML handling, which I muddled along with.

Streams, strings and web-features are something I will have to look at more carefully for next time, but for me the big one is design patterns – it was my only failing section.

All in all, I’ve very happy with this evening’s events and the (practice) results.

Category	Grade
XML & Web Services	EXCELLENT
Arrays	EXCELLENT
Web Features	PASS
Basic Language	EXCELLENT
Streams and Network Programming	PASS
Database Access	EXCELLENT
String Manipulation and Regular Expressions	PASS
PHP 4/5 differences	EXCELLENT
Security	EXCELLENT
OOP	EXCELLENT
Functions	PASS
Design	FAIL

A useful idea for helping to enforce PHP code standards

topbit+s9y@gmail.com (Topbit) — Tue, 11 Mar 2008 17:16:00 +0000

extending PHP_CodeSniffer by Raphael Stolt shows how to quite easily add to a tool that will report what parts of your PHP source needs a clean-up, from the built in ‘sniffs’ for coding standards, and now adding to that for some slightly more opinionated choices on the maximum number of lines per function, or functions per class.

This could be a start of a whole collection of additional classes for additional checks.

The only challenge I see at the moment (and I don’t think it’s a big one, though I’ve not tried it, so it might not even exist, I’ve just not had the opportunity to look yet) is having to put code into the PEAR directory path, since that is where PHP_CodeSniffer is looking for the base library. I do see that they use long PEAR class-names, so it may just a matter of having the phpcs tool look elsewhere for the base coding style class.

1234.20080102 dev’). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it’s just as easy to put a similar ‘live’ symlink into place with a new link.

If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine’s own hostname (more useful for CLI scripts, generally run from cron).

There are some downsides – with a complete checkout of a 40-some megabyte website (it’s mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it’s a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster – and the images aren’t being served from Apache, but a dedicated image webserver, that’s not a significant issue – and even on Apache we don’t have Etags enabled (Yslow from Yahoo also suggests that).

[edit: 18:06]
Ahah, some investigation – and copious use of echos in the phpcs script, and /usr/share/php/PHP/CodeSniffer.php class:

Creating your own standards, and using them from anywhere on the filesystem – not having to link from inside the PEAR/PHP/CodeSniffer directory:

The coding standard class with your extension:

The class is called PHP_CodeSniffer_Standards_Example_ExampleCodingStandard – the two ‘Example’s being the name of the coding standard. The file is called ‘Example/ExampleCodingStandard.php’

<?php
// in the …./Example/ directory
require_once ‘PHP/CodeSniffer/Standards/CodingStandard.php’;

class PHP_CodeSniffer_Standards_Example_ExampleCodingStandard extends PHP_CodeSniffer_Standards_CodingStandard
{
function getIncludedSniffs()
{
return array(
‘PEAR’, // the main PEAR standards
// Add our other standards, all from a subdir
dirname(FILE).‘/Standards’,
);
}

function getExcludedSniffs()
{
return array();
}

}

And in the subdir: .../Example/Standards/
– Raphael’s ToManyMethodsSniff.php and MethodLengthSniff.php files “from his post”:http://raphaelstolt.blogspot.com/2008/03/sniffing-refactoring-needs.html

To call it:

# we have to give the full path to the file called Example/ExampleCodingStandard.php
# $(pwd)/ is just Bash-ish for outputting the current directory name
phpcs —standard=$(pwd)/Example /path/to/file/to/check/ | less

ZCE prep - and dumb tests

topbit+s9y@gmail.com (Topbit) — Mon, 10 Mar 2008 21:55:00 +0000

This week I’m going to take the first of my PHPArch.com’s ZCE prep test – then I’ll read the book and see they they expect me to know.

Going for the Zend Certification is something I’ve been thinking of doing for a couple of years, and especially now that it covers PHP5 – and increasingly good practices and security topics. It’s not that I need to get the ZCE, I’d go for it , for the intellectual challenge if nothing else. It’s also the closest thing I would have to a professional qualification since I completed a HNC computer studies in 1992 – and that was just 1/day week day release over the course of a couple of years.

Of course, it’s not the first PHP test I’ve taken – last year, just before the PHP London 2007 conference, Allegis had come along to the PHP groups’s early-February meeting, to plug their services (and they got business from it, one guy interview the following day, a Friday, and started work on the Monday) – but they offered to have anyone that wanted to do the Brainbench test, paid for, by Allegis.

At the time, I had just started a couple of days before at a job near Covent Garden, but then left it after a week for a better gig (where I still am now, some 13 months on) – but I took Allegis up on the offer, and finally got the results at the conference. I never did get a copy of the exact numbers, but I was told the headlines, so these may not be exact, but they are certainly in the ballpark.

Time: 28 minutes (apparently this is very good)
Score 4.73 (out of 5.0)
Better than 98% of other test-takers.

I’m told that the harder the questions you answer, the harder the next questions get – so getting from 4.0 to 5.0 is a lot harder than getting from 3.0 to 4.0 – if I’m wrong about that, then please let me know.

The thing is, the Brainbench tests are open-book – they pretty much have to be, you take them at home, though they are against the clock. I can certainly appreciate the logic of it – after all, which serious developer doesn’t have an internet connection and a quick bookmark to http://php.net (and I’ve look at the other language sites – php.net is by far the best), or at least a copy of the documentation around (.CHM file or just a bunch of HTML pages – or, like I do, a weekly rsynced copy of the php.net manual!). Just as well I did, the tests I’ve seen always throw in some pointless questions like how to use LDAP, or how to connect to an Oracle database. I’ve never used either of them, so I don’t bother to learn them – but if I did need them, I’d figure it out in a few minutes reading – or, more likely, I’d have some kind of library, like the Zend Framework which did the hard work for me – plus, I’d only end up writing that kind of code once anyway before I threw it into a function and forgot the minutiae.

It saddens me when people are too dumb to do well on such a test though – how hard is it to read the manual, at least well enough to know where to refer to for more advice?

The ZCE is a closed book exam – or, as I call it (for all the reasons the brain-bench is open-book) – unrealistic. If I can’t recall whether the $haystack or $needle come first in in array or string search – it’s but a moment to look it up.

Even though I’ve listed my scores above, I don’t bother to promote myself with them on my CV – indeed Allegis is the only company (recruiter or not) that know them – they did pay for it after all. Because I can get those kind of scores with what I consider so little effort (about half-an-hour’s worth in fact), then either the test is bad, or 98% of the other people that gave taken that test are. Frankly, I’ve got to think it’s mostly the latter.

Keep reading my posts, and I’ll tell you want you need to do to ace your tests – and not look a fool when it comes to developing something I might set you.

svn checkouts vs exports for live versions

topbit+s9y@gmail.com (Topbit) — Sun, 09 Mar 2008 19:45:30 +0000

I've just read http://www.svn-checkout.co.uk/2008/01/19/how-to-release-new-versions-of-websites/ via http://www.lornajane.net/posts/2008/SVN-Deployment-and-a-New-Site and while I consider revision control an essential tool (my current job is the only one in the last five years where I didn't have to install my own RCS, but in the year I've been here so far, I've added nearly 1000 revisions), I somewhat disagree on the idea they suggest.

That first link, 'how to release new versions of websites', suggests checking out a version of the site as a working copy, (it's certainly something I've done before now), but then it goes on to use the 'svn switch' capability to move between versions (of course, they are also doing in in TortoiseSvn, and there for the live web-server is likely to be running Windows, no way I'd run a server on Windows - not even for testing). There is however some trouble with revision switching - especially on a busy site that has to keep running even while the new version is being put into place. While SVN does atomic commits - the new code goes into the repository all at once, or not, it's harder to do the all at once part on a non-transactional file-system - such as a webserver. The bigger problem is that rolling back will also take time - and in an emergency, the time taken to do something is crucial.

Here's what I do - Whenever I want a new version to go live, which might be from every couple of days to as often as a couple of times per day, I'll update run the script below with the specific revision number to export (and a date/time, but that's just for easy reference). When it runs, it also symlinks the given version as 'dev' - which is part of the path to the site 'dev.example.com', wham, instant new version, and I can trivially delete the symlink and on the same command line symlink the older version (with 'rm dev && ln -s 1234.20080102 dev'). After a quick wander around the newly checked out version, maybe run a set of unit tests, just for security, it's just as easy to put a similar 'live' symlink into place with a new link.

If there is ever a problem, rolling back to a previous live version is just as easy. Other configuration changes are made within the code on the apache servername, or the machine's own hostname (more useful for CLI scripts, generally run from cron).

There are some downsides - with a complete checkout of a 40-some megabyte website (it's mostly the Zend Framework and other libraries from which I use a number of files, though rarely all), it takes a little while (not too long, it's a gigabit link between the repository and the main webserver) and there are also some potential caching issues (Etags are usually based on the file inodes), but as we plan to move to a multi-machine cluster - and the images aren't being served from Apache, but a dedicated image webserver, that's not a significant issue - and even on Apache we don't have Etags enabled (Yslow from Yahoo also suggests that).

#!/bin/sh

REV=1914
REVDIR=$REV.20080214.1914

REPOSITORY=svn+ssh://username@svn.example.com/var/svn/SITEURL/trunk
IMAGES=$REVDIR/htdocs/i
STATIC1=$REVDIR/example.co.uk

svn export --revision $REV $REPOSITORY $REVDIR

mkdir -p $REVDIR/tmp/templates_c
chown -R www-data: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/htdocs/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/htdocs/cache/ $IMAGES $STATIC1
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php

# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x

ls -l $IMAGES/* | grep -- "-x"

# make the latest version available at http://dev.example.com
rm dev && ln -s $REVDIR dev