elliot's blog

Too much technology

There's been too much technology on here recently, and not enough frivolity. Ironically, this has been one of my least busy periods for a while, and I've had about 6 weeks of free evenings. So my main activities have been reading and watching DVDs. So why the heck not enjoy a list of my recent entertainments?

DVDs (and quality drama):

  • Heroes. Everyone says it, but it is bloody ace.
  • Battlestar Galactica. Don't laugh. I mean the new series, not the ropey old one. I'm half way through series one, and think it is possibly the best science fiction series I've ever seen. I particularly like the idea of a religious android.
  • The West Wing. I'm a latecomer to this one, but can see what the fuss is about. Fantastic, zippy writing, great characters, moral dilemmas, humour, what more could you want?
  • House of Cards. The BBC series from the early 90s, starring Ian Richardson. Marvellous, and Ian Richardson is completely convincing, horrible yet somehow sympathetic. Get it.
  • Firefly. The short-lived science fiction series by Josh Whedon, of Buffy fame. Really good. Funny, interesting, quirky, good characters. Shame they didn't make more.

As for books:

  • The Cogwheel Brain by Doron Swade is a great biography of Charles Babbage, written by one of the blokes who recently built a working difference engine. Vivid, thorough, and does a good job of placing Babbage (and Ada Lovelace - who apparently wasn't really the first programmer) in the context of computing history.
  • Rendezvous with Rama by good old Arthur C. Clarke was one of the first science fiction books I ever read, when I was about 10. It blew me away when I was that age. I got a new copy cheap recently, and have started reading it again in a fit of nostalgia. It's grand-scale hard sf, brilliantly done.
  • The Brief and Frightening Reign of Phil by George Saunders is one of the best books I've read for ages. I'm convinced this guy is one of the funniest and potentially most significant writers of our time. I reckon his books will eventually gain the reputation of a Pynchon or Kafka, in terms of how well he satirises and captures the spirit of his age. Not heavy like those writers, which I suppose reflects the culture which produced him, but utterly brilliant, laugh-out-loud hilarious, and piercingly perceptive.


One of my favourite recent finds in the music sphere is Planningtorock. I think this is a performance artist originally from the UK. She recently released an album called Have It All, which I bought through eMusic. It is a really unusual, exotic, hilarious, deranged, baroque album, reminding me of The Residents, Aphex Twin, Kate Bush, Lydia Lunch, Moloko, Kurt Weill, Goldfrapp, and Bjork, often all at the same time in one song. It's a while since I've heard something so outrageously inventive, especially in how the vocals are layered and manipulated. Well worth a listen.

Here are some of her videos on YouTube:

Thank you, GParted

I bought a new Maxtor Personal Storage 3200 (external USB drive with 400Gb capacity) this week, so I could make better backups. It came it at £60 from eBuyer, which I thought seemed reasonable.

When I first plugged it in, Ubuntu Linux recognised it OK, but wouldn't let me write to it, even as root. On inspecting the output of dmesg, it turned out it had been formatted using NTFS, and mounted as read-only. Not much use for backups.

I decided to delete the existing partitions and create new ones, using fdisk and parted, two command-line tools which I know a bit. However, while I could create the partitions, there seemed to be a problem with the partition table, and fdisk kept spitting out an error message which looked like this:

Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

This seemed to render the drive unusable: it wouldn't mount; then it mounted but was unreadable. I tried different filesystems, but couldn't get anything to work. I googled around, and apparently this is something to do with how NTFS and the BIOS interact (see this article for details). I thought "There goes £60 of hardware". Though I then thought "Probably Windows can fix it, if necessary".

Just as I was starting to despair, mainly at the prospect of having to boot Windows, I decided to give GParted (Gnome Graphical Partition Editor) a go, to see if this could help. I haven't used this before, but I am bloody glad I did today. The interface is similar to PartitionMagic (if you're familiar with that), and it was really simple to delete my screwed partitions and create a new ext3 one on the external drive with it. Once I unplugged the USB cable then reattached, the drive was recognised and writable. Thanks GParted guys!

iPlayer - response to petition

I recently signed a petition asking that the BBC's iPlayer be made available for users on platforms other than Windows (I use Ubuntu Linux on my laptop). iPlayer is an initiative to make the last 7 days' TV available to computer users via an in-browser viewer; however, the beta version is only available for Windows, and until now the BBC have been cagey about whether it will make it to other platforms. (Though I note that they now mention on the iPlayer page that they intend to get it to work on Linux and Mac eventually.)

The government's response to the petition (see petition and response) seems positive, as it states:

The BBC Trust made it a condition of approval for the BBC's on-demand services that the iPlayer is available to users of a range of operating systems, and has given a commitment that it will ensure that the BBC meets this demand as soon as possible. They will measure the BBC's progress on this every six months and publish the findings.

Let's hope they keep to the promise so every licence payer can take advantage of iPlayer. Ideally, I'd like to see it open-sourced as well. I can't see any benefit to keeping the source closed, particularly as the BBC is publically-funded, and (I'd argue) has an obligation to return as much as it can to the public. They might also find that open sourcing would help get it ported to Linux far more quickly.

Patchworking and being a generalist

As I've mentioned previously, my current contract with OpenAdvantage runs out at the end of September. (The organisation will continue to exist, but at different premises and with a slightly different structure, by the way.) During my job hunting, I considered various ways of working, including becoming a contractor, working part time, or taking up a full time post; I've also been talking to my wife (Nicola) about which road I should go down, and had a big shake out of my ambitions and career goals.

One lesson I've learned from my interviews and job hunting is this: I am a generalist. I have moved across the whole spectrum of computing, from doing a Ph.D. in Artificial Intelligence, through to developing e-learning materials, administering virtual learning environments, to programming and web development, to broader "business analysis" and open source advocacy. Along the way I've picked up a lot of knowledge of many different areas of computing, from the business level right down to the code level, with bits of everything in-between. However, the only thing I am a real specialist in is the theory of abductive interpretation (my Ph.D. topic), which you don't see on many job descriptions.

My worry, then, was that I didn't have a path to follow, a career I was definitely looking for. I was unlikely to find another role like the one I have at OpenAdvantage, very flexible with a huge range of activities. Consequently, I've been applying for a variety of roles, from business analyst to programmer to lecturer to trainer. Does this mean I don't know what my "career" should be? Or that I am sufficiently generalised to be able to fit to several types of role? Maybe it's not me that needs to be more specialised, but the jobs which need to be more general.

The main thing I've realised is that I need flexibility, so that I can work around my family, rather than work against them. That has been one of my main guiding principles throughout.

This then got me thinking about how I, Nicola and our friends organise our time, and how they'd like to work. We know quite a few people with young families, and there is a definite shift to parents wanting to share child care responsibilities. I was talking with Nicola about this this morning and she came up with a brilliant definition for this:

patchworking: working in two, three or more jobs, most of them part-time, perhaps shared between two people, to make up a full salary.

Here are some examples of patchworkers we know:

  • A. and S.: A. works 4 days a week and has the children for one day; S. works a few days and evenings to make up the rest of their salary.
  • A. and N.: A. works on contracts 2.5 days of the week; N. works 5 half days; they share the childcare.
  • X. works as a photographer and writer; Y. has a near full-time job.
  • C. works for half of the week and makes up the rest of her income selling items on eBay.

Nicola and I work a bit like this: I have a full-time job, but Nicola works evenings or the occasional weekend to provide a bit more money for luxuries. I have also been working a few evenings where I feel like it, again for a bit of extra money.

I think patchworking is an interesting trend, and one I'd like to see supported by more companies. One thing I've noticed when applying for jobs is how few part-time, flexi-time or home-working opportunities there are available. I suppose this is partly for efficiency reasons: it's easier to manage one person full-time than two people part-time. Though I'm sure there might be benefits to splitting a role in two: having two heads might be better than one; you could specialise roles to a finer grain and make sure you get the right person for the right part of the job; you could commodotise certain parts of a role to a less-skilled employee; etc.. Perhaps some of you know of good references looking at the pros and cons of full-time vs. part-time staff?

There is the option of contracting, of course, and setting your own hours, but I am fairly risk-averse, and have shied from this for that reason. It would be nice to get some of the benefits of patchworking, but without having to take the risks associated with contracting.

In I.T., lack of support for patchworking seems particularly crazy: as open source projects have proven, there is no reason why software and I.T. projects can't be carried out by distributed teams who hardly ever meet and work at different times of day. I understand the need and efficacy of physical meetings to support particular phases of a project. But in many situations, the requirement to physically be in an office derives from habit rather than need.

Patchworking also points, perhaps, to the trend towards people having lots of short-term careers, rather than one over-arching one. You quite often hear about people changing career quite late into life, or moving between vastly different areas of work. Again, this is a trend which is likely to continue, and one I think businesses in the UK should take notice of and respond appropriately to.

Learning Java

While preparing for recent job interviews, I've become painfully aware that my Java knowledge, while OK, is a bit lacking. I've never formally been taught Java, and have picked it up from hacking and a few text books. Consequently, my understanding is somewhat fractured.

What I'd like to know, then, is whether anyone can suggest a good reference book or site to get me up to speed with how to do modern Java programming. What I need are examples of good practice, including how to properly design Java applications from a test-driven perspective. Examples of decent Java patterns at a lower level (error handling, scope and size of methods, the Java idiom) would also be useful. Any suggestions?

Brian Wood, this one's for you

A few people have said they'd like to see an official Last.fm Drupal module, hosted on CVS. I have been using it on my own site, but I have hacked Drupal itself so I can set a timeout for HTTP requests when using drupal_http_request(). My reason for doing this was because Last.fm seems to timeout quite often, and Drupal won't stay up on my hosting if a module times out. This is a bit unfortunate. My development on it has stalled, as it appears finding another job is itself a full-time job. I've also shifted from Drupal stuff to exploring the Zend Framework, as I want to find a decent way to develop PHP applications as properly as possible.

Brian Wood, who has written some patches and done some work on this himself, is being far more active than me in adding functionality to this module, so I've suggested he become its custodian. So attached below is the latest version of my Last.fm module, without the patch to drupal_http_request(). It's primarily here so Brian can see what I've been doing with it. Hopefully he'll be able to pull something together and release it officially.

By the way, the thing I'd like to see in Drupal is a generic way to fetch data from a service and transform it. So far, I've used/written code which does this for Flickr, Last.fm, AllConsuming and TextLinkAds in this site. Each time, the module is using different, replicated code. It would be nice to have a proper Drupal API for this kind of stuff, don't you think? Perhaps there is one already and I just don't know about it.

The Python Challenge

(...where "challenge" is the operative word.)

Update: In the end, after an hour of trying, I had to give in and find a clue for level 5. I missed an obvious clue, which I realised when I found this solution.

After a suggestion from SteveA in the comments to my last post, I've been working on the Python Challenge. I was quite pleased with myself for the first 4 levels, which I finished with a little bit of huffing and puffing (for example, on level 3, I was getting the right matches, but not doing the right thing with them to get the next URL). Level 5 is an utter pig. I've got so far as unpickling the data, but then I'm not 100% sure what to do with it next. It's a list of lists of tuples, which presumably map onto some data structure, but I'm not sure what. I will persist. It's pretty good fun, though I think it is perhaps overly cryptic. I think there should be slightly more clues about what to do.

External validation of programming skill

We've had some bad news here over the past week or so: OpenAdvantage will go under wraps at the end of September. No money is available for the project to continue, so, consequently, we're all job hunting. This has resulted in a lot of soul-searching for me, an urge to hone my skills, and a need to work out what I'm really capable of. I started looking around on the web for self-tests I could use to determine my skill level and find out where I'm rusty. Like many open source types, I can turn my hand to pretty much anything, but have few certificates to prove it. The training courses for Linux certification, MySQL, etc. are very expensive and time-consuming; I was looking for something cheap and quick, to provide a rough indicator of my skill level. Here are a few resources I found useful:

  • JavaBat is a series of free, online Java practice problems. They are pretty simple, but very useful for reminding me of some parts of Java I'd otherwise ignore.
  • oDesk is a marketplace for remote workers. You can put your profile up there, bid for jobs, search for work, build teams, post jobs etc., all from the comfort of your web browser. The reason I like it is that they have a series of free tests you can take, which you can use to get some proof of your skills. Given that there are around 18,000 contractors on there, a potential employer can get an idea of where your skills sit within that pool. For example, I scored 3.6 out of 5 on the Ruby on Rails test, putting me at rank 15 in the pool of 83 people who've taken the test (of which 49 passed). You can see my profile here.
  • BrainBench is another US company providing some online tests for free, and lots more if you pay. My public transcript demonstrates I'm a bit rusty on Java, but better than I thought on Apache (3.99 out of 5, in the top 14% of test takers). Perhaps I should be an Apache administrator... You can also add your BrainBench certifications to oDesk, which is pretty nifty.
  • Transcender provides a range of free Microsoft (gasp! shock! horror!) tests for download, to practice for the MCSE exams. I'd always been intrigued by this qualification, and wanted to find out how whether it was actually difficult, so I did a few of the tests. I tried the one on .NET (bearing in mind I've never written more than about 6 lines of it), and I managed 40% on a multiple-choice test. Even more astonishing was that I scored 60% on the SQL Server test, and have hardly used it (though I did do a SQL Server course about 6 years ago). Ironically, the harder tests seemed easier to me, because they are more generically about design than specifics of the applications or languages. The SQL Server one, for example, gives you a case study and asks you to build the application to fit the requirements, which I've done plenty of.
  • I also found out about the CIW (Certified Internet Webmaster) qualification, which I'd seen as requirements for a couple of job adverts. Anyone heard of this or have it? I've ordered a cheap second-hand book from Amazon because it might be worth reading up and taking the exam (for £90).

Even though these services haven't had widespread coverage or acceptance in the UK, they're better than nothing, and a quick way to brush up your skills and identify your strengths. It's occurred to me over the past couple of weeks that programmers need external validation, particularly those of us who work with open source. I'd like to see cheaper, more readily-available courses available for normal developers who work independently or in their spare time, to help fill this need.

If anyone knows of any other self-tests (e.g. where's a good place to sharpen my Python skills?), please add a comment.

Removing a password from a PDF on Linux

I just bought a PDF, legally, from a publisher's website. However, in their wisdom, they decided it would be a good idea to password protect all legally-purchased PDFs. This means that each time you open the PDF using Acrobat Reader, you have to remember and type in the password to read it. (Evince, the built-in PDF viewer in Linux, allows you to permanently save the password, but I tend to use Acrobat as it copes better with some PDFs.)

So, if you know the password for a PDF and want to remove it, you can use the command line tools pdftops and ps2pdf to free your PDF from its chains.

  • Install pdftops. On Ubuntu, you can do:
    apt-get install xpdf-utils
  • Install ps2pdf. I think this is already included with a default Ubuntu.
  • Convert the PDF to a postscript file, using the password:
    pdftops -upw <password> <file>.pdf
  • Convert the resulting postscript (which is now sans password) back to a PDF:
    ps2pdf <file>.ps

The only thing you lose are any PDF-specific features which don't translate to postscript, e.g. hyperlinks.

Remember, this only works if you know the password for the PDF: it doesn't break the PDF password for you.

Syndicate content