elliot's blog

Evaluating open source software

There's a lot of OSS. How do I make choices about what's good and what's bad? Here's some things which put me off some OSS products.

  • Crippled deployment
    I dislike OSS products which don't provide an easy build process. They are often provided as source only and extremely hard to configure, requiring all kinds of dependencies which aren't specified. There are also often lots of configuration options which aren't documented. This isn't necessary. It is often eminently possible to produce RPMs or the like for these packages, with little fuss or complexity, but firms hold onto these and use them as a unique selling point. (For example, TotalRekall is a company which does this.)
  • Glossy websites
    Call me a bigot, but I'm not keen on OSS with a flashy website. This makes me suspicious. There's nothing wrong with a nice website, but I am wary if a company looks like they have spent a ton of money on it. It kind of suggests they've got their priorities wrong. (See Typo3.com.)
  • Not eating your own dogfood
    Say I go to a site to read about a CMS designed for corporate sites, and the company producing the CMS doesn't use it. This suggests the CMS didn't meet their needs. What's the chance it will meet mine? Similarly for companies selling OSS services who run their website using ASP and IIS: it suggests to me that their heart isn't in OSS, if they don't even use it in the areas where LAMP is quite obviously the correct solution.
  • Hiding the treasure
    Some companies produce dual-licensed OSS, then hide the OSS version. This is OK to an extent; but when the OSS version is only available from a single link at the bottom of a page buried in the site somewhere, you get the idea they don't want you to find it. This is deceitful. Why be scared about someone using the OSS version? I'm more likely to want to pay a company who makes it easy to find and use a proper trial version which is easy to install and understand.
  • Poor examples
    If assessing products for website development (CMSs, for example), it is easy to find example websites based on the CMS. This can immediately give some idea of the performance of the system (though only if averaged across all the example sites: poor performance on one site may be down to local configuration, but poor performance across many sites is more likely due to the system under analysis). It can also give an idea of its flexibility (though this is like trying to deduce the genotype from the phenotype, and may be a bit misleading): do all the sites look the same (in which case it's likely the templates are limited in scope)?
  • Poor interoperability
    Good open source projects will often reuse the work of other projects. This is a good indicator for me of how standards-compliant and open the APIs of the product are. If a product stands in isolation from everything else, and has reinvented several wheels (e.g. has its own web server specific to the product), it is likely that this was done either through a desire to hack for the sake of it, because the existing products wouldn't work with this product, or because the developers on the project didn't do sufficient work or have sufficient understanding to look for existing solutions.
  • Lack of extensibility
    Some open source products are very badly designed. This makes them difficult to customise and extend, if you want to do this (which is, after all, part of the reason for choosing open source).
    A good indicator of a well-architected piece of software is the availability of an API for plugins. This shows that the product has been designed to fit with other products from the get-go; it also gives an indication of how easy it would be to extend the product myself. Look at major open source projects like Ant and Eclipse: part of their popularity stems from their extensibility.
    Another indicator is the availability of templating or mapping facilities: these are generalised features which make adding your own extensions much easier. For example, the Castor data binding framework for Java provides facilities which allow you to specify how fields in a class are mapped to XML elements and/or attributes. The mapping itself is defined in XML. No pre-defined mapping style is mandated.
  • Lack of development activity
    A project with few developers, no activity on mailing lists, and no recent releases, is likely to be moribund. I am not prepared to commit myself to a project which hasn't seen life for several months: it is unlikely to be patched if security issues arise, and unlikely to improve if it has any missing features.
    If a product lacks features, but has a lively development community, I may well track its progress until it supports the features I need. For example, about 18 months ago I looked at the eXist XML-native database project and tried it out. It didn't support XUpdate, which made it unsuitable for what I was trying to do. But because the development seemed active and up to date, I kept an eye on it. Now, XUpdate support is included, and I am considering using it for a new project.
Syndicate content