elliot's blog

Stuff I've done - the howtos list

I maintain a howtos folder where I put text files about things I've managed to do, or failed to do, on my computer. I thought it would be fun to turn the list of files into a human-readable list to give an idea of what I've done over the last 20 years or so of working with computers. It might even be a useful thing for recruiters to look at, so they know I'm telling the truth when I say I can probably cope with most programming/computing tasks. (Not that I'm looking for a new job right now.)

aapt compile
agata report
alfresco
ant without java
archlinux rpi
asp.net on linux
atom text editor personalisation
audio file fixing
audio file trim
bash commands
bazaar ng
bind install
cmp jboss
cobol compilers
config dell laptop
cron jobs
db clustering
dbg install
ddwrt on tplink TL-WR740N-v4
dns servers
dosemu on ubuntu breezy
dvd ripping and video encoding
eclipse
elearning
fastcgi ruby
fedora 11 audio setup
fedora personal setup
firefox plugins
first router netgear 2004
flex2
git replay commits
gnome shell extensions
grip config
haxe install
horde and imp
icoya
importing outlook
install breezy
install edgy
install fedora
iptables
java install fedora
java unit testing
jboss
joomla ecommerce install
jruby rails
kino
lamp hardening
lighttpd
live cd ubuntu
ltsp hw requirements
mail by telnet
mail server postfix dovecot and fetchmail
mambo install
merb and datamapper
mongrel clustering
moodle install
moodle scalability
mplayer install win32 codecs on fedora
nas fun plug
openbsd on virtualbox
openbsd on vmware
open dns servers
openreports
opensurveypilot
oscommerce overview
phpbb db structure
php compilation
php with iis
playonlinux install
plone
quakejs build
rasbian rpi
raspberry pi setup
realplayer ripping
report generators
ruby intro
ruby on rails from source
ruby on rails migrations
ruby on rails tutorial
ruby on rails with xampp and fastcgi on ubuntu hoary
running dos games
running dos on linux
soft phones
spamassassin and postfix
squid minimal setup
squirrelmail
ssl self signed certificate
subversion
supercollider
surveys and data analysis
svn apache trac tortoisesvn eclipse
svn on site5
symfony2
tinyerp
tomcat install
tortoisesvn
trac
umbraco on docker
vim commands
webmin install
windows on linux
xampp addon
xdebug xampp
zaurus usb connect
zencart and authorize.net

Disabling "recently used" in MATE desktop file chooser

I've recently rebuilt an old laptop to use Fedora 24, installing the MATE desktop rather than using GNOME 3 (I really can't get to grips with it, and want my desktop to look like Windows circa 1996).

This all works really nicely, but Pluma, the default text editor in MATE, insists on showing you the most-recently used files in its file chooser. It took me ages to figure out how to fix this, so I'm posting it here now I've worked it out, for future reference.

Change the settings for the file chooser to open at the user's current working directory by running this on the command line:

dconf write /org/gtk/settings/file-chooser/startup-mode \"cwd\"

If you want to permanently disable recent files altogether, edit (or create) the file ~/.config/gtk-3.0/settings.ini with this content:

[Settings]
gtk-recent-files-enabled=0

Comments off

I've finally given up on the idea of making my blog a "social" site, and have turned off comments. The amount of spam I was getting on every article was absolutely ridiculous and took about 2 hours to clean up every time I got round to doing it.

I've now cleaned up all the spam (apologies if I removed any legitimate comments that anyone actually cared about). From now on will be turning off comments for every blog post and article (not many, given that I rarely update my blog these days).

Anyone who really want to get in touch with me can track me down via @townxelliot on Twitter.

Books read 2014

I read 67 books in 2014:

Books read 2013

I read 36 books in 2013. Here's the list:

Experiences with Firefox OS and the ZTE Open

Exit Galaxy S2

I had a Galaxy S2 for a couple of years, which was a brilliant phone. It's Android, so you are spoilt for choice as far as apps go: my main use of the device was for multimedia, watching Netflix, iPlayer, stuff on my NAS drive (via BubbleUPnp, the best Android UPnP client, by the way), Sky Go (it's one of a handful of compatible devices), listening to music and radio shows. Using the excellent SwiftKey virtual keyboard, typing was fast and easy, so I also used it for personal email, Twitter, Facebook etc. The camera was great; so good that I'd stopped using our digital camera in favour of the S2.

But problems started a few months back. The microSD card somehow got corrupted, so I lost a load of photos. The USB port (or the system's detection of it) got flaky, so sometimes it continued to behave as if it was charging, even when unplugged. I couldn't reliably connect the USB to download files from it. I couldn't update the operating system, so more and more apps were becoming incompatible.

Enter ZTE Open

Eventually I realised I was going to need a new phone, contract-free (I use giffgaff). Because I like a challenge, and I work in open source, and some of my friends and old colleagues work on it, and because it uses HTML5 for its whole UI, I went for a SIM-free ZTE Open running Firefox OS (£60 on eBay).

On paper, Firefox OS sounds like a great concept: fully open source, most of the userland apps in HTML5, extensive web APIs for third party developers to interact with system services. However, my disappointment on first trying the phone was almost palpable. In fact, my first impressions were so very bad that I contemplated selling it on eBay within about an hour of unwrapping it. I was going to write a thorough review explaining why, but then this Phones Show video summed it up. As a phone, this device is barely adequate, for all the reasons explained there. I'll summarise the salient points:

  • The capacitive screen is so unresponsive, it feels like a resistive screen.
  • The camera has no flash and is only 3 mega-pixels. I could have worked this out beforehand, but it never occurred to me that a phone would be sold without a flash in 2013.
  • The onscreen keyboard is unpredictable, fiddly and generally a very poor experience. I end up having to correct mistakes so often that I've practically given up using it for anything other than text messaging. Even then, the experience is about as good as typing on hardware keys 10 years ago (remember that?).
  • The user experience when using some of the applications is just dreadful. For example, if you use the Twitter app to follow a link, the link opens in a browser and the Twitter app disappears (closes itself, I'm guessing). You end up having to restart the Twitter app and scroll back to the place you'd got to in the status updates list.
  • The app-store is full of hobby projects or very thin wrappers around mobile websites. The quality is generally quite poor.
  • Some mobile sites don't recognise the phone, or think it's Android, or are so slow they're unusable. So iPlayer, Amazon Cloud Reader (there's no Kindle app), and Evernote are inaccessible. I'm guessing there are lots more. Other sites like GMail look like they did 5 years ago, as you get the default "we're not really sure which phone you're using" interface.
  • Because there's no widgets API as far as I can tell, you can't easily shortcut to tasks you might need to do often (e.g. change screen brightness: to do this, you have to dig into the configuration app, which takes a click, a scroll, another click, and a slide; I can't bear auto-brightness, as the screen brightness fluctuates madly in the British weather).
  • Setting up the email app for an IMAP server took me forever, until I realised that it was because the SSL certificate the email server was using was registered to a different domain name. Once I ping'ed the IP address and got the "real" domain name, and used that instead of the alias, it worked. I've never had this issue with any other email app, on Android or Linux or Windows or Mac, which will either carry on regardless or give you a decent error message about why they're hesistating. No normal user is going to know how to sort this out (though admittedly they'll probably be using GMail or Yahoo).
  • Setting up the weather app to show me the weather for where I live, automatically when I open it, took forever. I ended up going to the mobile site (instead of using the Accuweather app, which couldn't seem to remember my location), finding the weather for where I live, and bookmarking that instead. And of course there's no widget (see above) so you can see it at a glance.
  • The contacts app has no integration with Google contacts, only Facebook. It can import contacts from a SIM, though in my case they are a complete mess as I was maintaining contacts in Google contacts.
  • Updates failed for me for about a week. I kept getting a notification that there was an update, and when I tried to download it, I got a generic "update failed" message.
  • The web API has some big gaps which prevent you from writing certain types of apps (e.g. anything using udp). This is a bit developer-specific, but it puts a leash on how useful the phone is even to developers.

Within a few days of purchasing it, I realised it just wasn't going to do for general day-to-day use. I actually forked out for a very cheap (£50) Android (Jelly Bean) tablet from Amazon (I think I should have spent a bit more money, as the web browsing on it is a bit crap, but games and multimedia work fine; the screen is only adequate; and the system clock drifts incessantly, which is irritating).

Having said all this, the good points of the ZTE Open are:

  • It was dirt cheap.
  • The phonecall and text functionality are acceptable (I am using it as my everyday phone).
  • It integrates with Google calendars.
  • It is light and compact.
  • The battery life is excellent (though probably because I don't use it much).
  • I like the data usage monitoring (useful if you're on a capped-broadband or pay-as-you-go SIM).
  • Wireless works fine. I haven't had any issues with it on my wifi, unlike the chap in that video review. Though it does randomly drop off wifi occasionally.
  • Firefox is a decent mobile browser.
  • It is an incredible achievement to put together an operating system for a mobile device and get it to market. I have only admiration for the team that managed it.

It's also worth bearing in mind the context for the device. It's not a "first world smart phone", more a "developing markets first smart phone". In that context, it's great. I imagine even having internet access on a phone would be a massive boon in some countries (I remember how exciting it was when I got my first hand-me-down HTC phone a few years ago). Data usage monitoring is key in countries where phone companies are still developing infrastructure, and contracts are likely to be capped.

Marconi

And, this isn't the end of the story from my point of view. Once I got over the initial disappointment of how incomplete, inadequate and substandard this device is as a phone, and ordered an Android tablet to take up the slack, the "pressure" was off the ZTE Open. I could lower my expectations because I didn't need it to be anything more than a phone. Any functionality on top of that was a bonus. With this new (more realistic) perspective, I could more easily see the possibilities. Here was a phone which I could develop apps for, using my existing skills and toolchain, without having to run cumbersome SDKs and emulators (I have tried Android development in the past, but it's not really my thing).

Recently, I've been downloading radio shows and podcasts, and listening to them while walking around, after dropping the kids off at school, picking them up, shopping etc. Firefox OS has a music player, which is OK though basic. But this app, like most other music apps, is focused on playing music: generally short pieces of audio that you might listen to in one sitting, from start to finish. By contrast, listening to radio shows is more like watching a film on Netflix or reading a book on a Kindle: you might listen to the first half hour, then switch it off, then go back to it later.

Most music apps don't cope with this use case, where you "pick up" an audio file for a while then set it down, so you can come back later. The main feature you need for this is to track progress through an audio file and persist it, so a user can return to the same point they reached earlier. (The way Netflix and Kindle do for movies and books respectively.) On top of this, I also wanted a music player that showed me the full title and album for a track without truncating it or scrolling it leisurely into view; group by album as the default (as the album for a podcast or radio series tends to be the important thing, rather than the artist). Other desirables features: a slider for scrubbing through a track, while being able to see where you are in it (the Firefox OS Music app isn't great for this); and skip buttons to jump over chunks at the start of an audio file (where they tend to put introductions and adverts). I also wanted standard music app features, like being able to quickly go back to the last thing I listened to. And I didn't want loads of other features that I rarely use or care about, like playlists and album cover art.

I started a project called Marconi, an audio player focused on audio rather than music. My first step was ripping some code out of the Firefox OS Music app for decoding the ID3 tags out of mp3 files (the beauty of open source, though it would be nice if audio tag reading was a service...). I then wrote a wrapper round the Firefox OS deviceStorage API, so I could read audio files from the phone's hard drive and get their metadata. Initially, I just had a very simple list view of the files as an HTML file.

Next, for testing, I installed and setup adb (Android Debug Bridge) and the Firefox OS simulator (as documented on the Mozilla developer network). Shortly, I got Marconi running in the simulator (once I'd recognised this bug was holding me up).

After that, it took me a while to figure out the steps I needed to be able to push stuff to the phone. For those of you who might be similarly struggling (it was the main barrier for me), on Fedora Linux the steps are:

  • Unplug your Firefox OS phone from your development machine.
  • On your development machine, close Firefox.
  • On your development machine, you need a rule so that udev will recognise your phone when plugged in via USB (at least, I think you do; I get so confused sometimes). Add a file /etc/udev/rules.d/60-fxos (owned by root:root, chmod 644) with this content:

    SUBSYSTEM=="usb",ATTR{idVendor}=="19d2",MODE="0666",GROUP="plugdev"
  • Restart udev, probably sudo udevadm control --reload-rules. It all gets a bit vague here, but I just checked and this seems to work.
  • Kill any existing adb server instances.
  • Start adb as root. AS ROOT. You can probably do it as non-root, but will have to fiddle about with all sorts of permissions. I spent a while on this before giving up and resorting to root.
  • Run adb devices and ensure you see a line like "roamer2 device". If you get "Insufficient permissions", restart adb with your fingers crossed and/or a prayer on your lips and try again.
  • On your Firefox phone: make sure you have remote debugging enabled (Settings > Device information > More information > Developer > tick "Remote debugging"); and turn USB mass storage off (Settings > Media storage > untick "USB mass storage" so it is disabled).
  • Open Firefox, then the Firefox OS simulator.
  • Plug the phone into your development machine. Hopefully it will show up in the simulator tab. You should now be able to push apps to it.

One other thing which wasn't very clear from the docs: you will need a manifest file, called something like manifest.webapp (the .webapp is important) to be able to load your app into the simulator. It's explained in detail on the Mozilla developer site.

I continued developing the app, which uses Bootstrap 3 for most of the UI (I'm not 100% sold on it, but it's quite lightweight and looks pretty good), jQuery UI for the slider (it's the best HTML5 slider in my opinion), some media player code I wrote on a previous project (which wraps HTML5 <audio>), localStorage to persist progress data, and Stapes for event handling and the model layer. There are a couple of screenshots below.

Notes: the screen on the left is scrolled up a bit so you can see the progress bars for two files; the buttons are positioned so I can reach them with a thumb when holding the phone with one hand; the slider updates the progress text at the top as you slide, and you can slide it to reach right to the start or end of a track - difficult to do precisely with the default Firefox OS Music app; progress bars on the home screen update in real time as you're listening to a track; the top-right notes button takes you back to the "last played"/"currently playing" track.

Finally, after quite a bit of work, I have an app which does what I want, for now. It won't scale to thousands of files, only copes with mp3s, doesn't do playlists or album art; but it is optimised for me: it remembers what I listened to last, it tracks my progress through audio files, it shows me full album and track titles, and it has BIG UI elements and fonts which I can easily see and press while I'm walking around. Yes, it's fairly ugly, the proportions are all wrong, it's not perfect by any means, but it suits me very nicely.

The moral of this story is that while Firefox OS is terrible as a end-user phone in 2013 (well, about as good as the phone I had 10 years ago), it is great as a platform for experimenting with HTML5 web apps in a real mobile environment. I don't think of my ZTE Open as a phone so much; more as a custom media player which I wrote myself, with the phone functionality as a bonus. Which is a rewarding feeling to have.

Some git commands for amending commits and showing short logs

I noticed a friend of mine tweet about git commit --amend yesterday, and it occurred to me that some people probably haven't used git very much and are still learning the ins and outs. I thought I'd share some commands I find really useful.

The first thing to mention is that you can set up your own command aliases by editing a file called .gitconfig in your home directory. This is where I keep all my custom aliases. (I use them so often that I'm often lost when using git on a different system where my aliases aren't set up.) Here's an extract from my .gitconfig with some explanation of the aliases:

[user]
  # sets up your details so that commits carry the right user info;
  # I use my primary github email address for mail, so that
  # all my github commits are associated with the right account
  mail = <you primary github email address>
  name = <your real name>
[smtp]
[sendemail]
[color]
  # turns on the default terminal colouring for git output
  ui = auto
[alias]
  # A good resource on git aliases is https://git.wiki.kernel.org/index.php/Aliases

  # these are just shortcuts to full git commands
  st = status
  ci = commit
  br = branch
  co = checkout
  cp = cherry-pick

  # I use this one when I am working locally, miss some changes, then want to
  # add them onto the last commit I made; it stages all the files which have
  # changed (NB NOT new files or deleted files), merges those changes to HEAD,
  # and retains any existing comment from HEAD
  cim = commit -a --amend -C HEAD --reset-author

  # this is similar to the cim command, but instead of automatically adding any
  # changed files, it only merges staged files into the HEAD commit; this is useful
  # where you just want to add one or two files you've changed to the existing
  # HEAD commit, before making a new commit with the other changes
  fix = commit --amend -C HEAD

  # sl: short log showing author and changed files; useful for checking
  # what I've patched in each commit
  sl = log --format=format:'%h %s%n%an, %ar (%ai)' --name-status

  # author: Show all the commits a specified author has done
  # Usage: git author <email_address|name>
  author = "!sh -c 'git log --tags --source --oneline --author=\"$1\"' -"

[merge]
  tool = meld
  keepBackup = false

[push]
  default = current

Note that I have others, but these are the ones I use most often.

Next, to give a flavour of how you might use them, here's an example session making use of these commands.

First I init a git repo and add a README file:

[ell@lp shiny]$ git init .
Initialized empty Git repository in /tmp/shiny/.git/
[ell@lp shiny]$ vim README
[ell@lp shiny]$ git st
# On branch master
#
# Initial commit
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	README
nothing added to commit but untracked files present (use "git add" to track)
[ell@lp shiny]$ git add README
[ell@lp shiny]$ git ci -m "Add README"
[master (root-commit) a2584b9] Add README
 1 file changed, 1 insertion(+)
 create mode 100644 README

Now I edit the README, but want to merge those changes into the existing commit. So I use my cim command:

[ell@lp shiny]$ vim README 
[ell@lp shiny]$ git cim
[master f4be9e7] Add README
 1 file changed, 3 insertions(+)
 create mode 100644 README

This adds my changes to the existing commit, keeping the commit message I've already laboriously typed in.

Now I show a short log of what's happened so far. I prefer this log format as it shows just the changes files, a short form of the git commit idedntifier, the author, and the comment:

[ell@lp shiny]$ git sl
f4be9e7 Add README
Elliot Smith, 2 seconds ago (2013-07-18 10:35:12 +0100)
A       README

Now I add a new file, package.json, then make an edit to README:

[ell@lp shiny]$ vim package.json
[ell@lp shiny]$ vim README 
[ell@lp shiny]$ git st
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   README
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	package.json
no changes added to commit (use "git add" and/or "git commit -a")

I want to include my edit to README with the existing commit, then add a new commit for package.json. So I first stage the changes to README and use my fix command to merge them into the existing HEAD:

[ell@lp shiny]$ git add README
[ell@lp shiny]$ git fix
[master 71de8f9] Add README
 1 file changed, 5 insertions(+)
 create mode 100644 README
[ell@lp shiny]$ git sl
71de8f9 Add README
Elliot Smith, 32 seconds ago (2013-07-18 10:35:12 +0100)
A       README

Then I stage the changes to package.json and make a new commit for them:

[ell@lp shiny]$ git add package.json
[ell@lp shiny]$ git ci -m "Add package JSON"
[master b717cc8] Add package JSON
 1 file changed, 1 insertion(+)
 create mode 100644 package.json
[ell@lp shiny]$ git sl
b717cc8 Add package JSON
Elliot Smith, 2 seconds ago (2013-07-18 10:35:52 +0100)
A       package.json

71de8f9 Add README
Elliot Smith, 42 seconds ago (2013-07-18 10:35:12 +0100)
A       README
[ell@lp shiny]$ 

Notice that I almost automatically do git sl after each cim/fix/commit command, as it's so quick and easy to read the output.

Anyway, I might write some more if I feel so inclined another day. I make extensive use of interactive rebasing, for example, which is never really explained anywhere in simple terms, but which is massively useful (especially if you're making branches to turn into pull requests for other projects).

git can be a bit daunting at first, but after you've used it for a while I don't think you'd use any other version control system ever again, unless forced to.

grunt plugins for Tizen apps, creating zip files, and running mocha tests

The short version

I wrote some grunt plugins for managing Tizen application development and doing related stuff:

  • grunt-tizen: For pushing packages (or other files) to a Tizen device, installing/uninstalling them, and starting/stopping applications (via sdb).
  • grunt-zipup: For creating zip files with customisable, template-based filenames; as Tizen web apps are just zip files with a wgt suffix, it can also be used to create those.
  • grunt-mochaccino: For running mocha tests within a grunt build, using the mocha command line rather than the mocha API. This was developed for unit testing the above plugins, but can be used to run mocha tests for any project.

They are all available from github at the links above, or can be installed via npm (instructions in the READMEs).

They are all released under the Apache v2 license.

The long version

I've been working on Tizen-related applications (most notably some of the 01.org web apps) for a few months. (The Slider Puzzle I rewrote should work in very recent Firefox and Chrome.)

One aim was to get these applications to run on Tizen devices, partly as they are included as samples in the Tizen SDK. My colleagues have done some good work on the SDK, but, unfortunately, it doesn't yet support the Linux distribution I use (Fedora, currently version 17). So I had a requirement to install web apps on Tizen, but without the possibility of using the SDK.

Fortunately, one of the key tools for interacting with a Tizen device is sdb (Smart Development Bridge), which is a fork of the Android adb tool. It works over a network connection between the device and a developer machine, usually over a USB cable, enabling file transfer between the two and access to a shell on the device. sdb is very easy to build on Linux, so it wasn't long before I had a way to push web app packages to my device, get a command line on the same device, then use the Tizen package tools (wrt-installer and pkgcmd) to install the package, then running it in debug mode and remotely debugging it (via wrt-launcher).

As with all these things, I soon got bored with entering the same commands over and over, so I wrote some bash scripts to automate pushing the package, starting the shell, executing the command, and opening a debug session. One of my colleagues started using the same scripts, so I made them more portable and robust.

Alongside this, I was using the venerable grunt to minify my JavaScript/CSS/images etc. But this meant two build tools (bash scripts + grunt) in use at once, which would never do. So I started integrating the bash scripts with grunt, and gradually replacing calls to command line programs where possible with grunt plugins.

The result is the three plugins mentioned at the start of this post. They've been tested on Fedora Linux and Windows 7, and are hopefully fairly easy to use (I spent quite a bit of time on documentation).

Please try them, and, if you're so inclined, contribute via github (pull requests and issues).

Syndicate content