townx

This turned into a pretty long blog entry. To help you decide whether you want to stick it out, it's about:

• My experience with C development for Moblin
• JavaScript for desktop applications
• Gjs, GObject introspection, and GIR repository installation
• Running JavaScript Clutter applications using gjs
• Interesting movements in the JavaScript world

Background

I started work at Intel's Open Source Technology Centre 6 weeks ago (just realised I haven't written about that here yet), working on the Moblin SDK. Moblin is a Linux-based platform for mobile devices (netbooks, UMPCs, that sort of thing); my role in the project is working as part of the SDK team - basically trying to help third-party developers outside Intel develop software which runs on Moblin. So far, this has involved learning as much about the Moblin stack and ecosystem as possible, as well as helping organise and add to the content on Moblin-related websites. (It's challenging, exciting work and I'm thoroughly enjoying it so far.)

Currently, software development on Moblin is comparable to generic Linux development: languages like Python, Ruby, C++, Perl, Java, C# (via Mono) etc. are all available. However, if you want to make use of the full range of Moblin libraries (like our stuff for managing media metadata or interacting with social networking sites), and combine them with Clutter (our GUI toolkit), the best approach (at the moment) is to program in C. While many of the Moblin libraries have bindings to other languages (like Python and Perl), C gives you the best, most consistent access to all of them (as they're all written in C). You'll also find Moblin development easier if you're familiar with GNOME and glib, both of which are fairly C-centric.

My background isn't in low-level system programming. I have been learning C since starting at Intel, and have enjoyed it thoroughly, but there are several aspects of it I could live without:

• Memory management is my main bugbear. I really don't want to have to care about allocating memory for data structures, then cleaning up when I'm finished with them. Learning how to (still learning...) has definitely enhanced my understanding of how programs actually work under the bonnet, but dealing with memory directly is not, in general, fun. I'd rather the runtime dealt with it.
• autotools (aka the GNU Build System) are not straightforward, especially if you're used to build tools like Ant and Rake (or even Maven). I know they are extremely powerful and flexible, and ubiquitous for open source projects, but I defy anyone to tell me they are simple, easy to learn, or well documented. In fact, the autotools (automake, autoconf) actually wrap other tools (make) which are even more complicated. This is partly a consequence of the job they do (making it possible to compile software across different platforms), but means they are not approachable. See the diagram on the autools Wikipedia page to get an idea of how complicated they are.
• Pseudo object oriented programming (in C) isn't really as pleasant as full OOP (in a language like Ruby). GObject for C (part of GLib) gives you some of the facilities of full OOP, but (as far as I understand it) it's not possible to encapsulate data with the methods which act upon it. It also seems tremendously complicated. Even with GObject, you are still basically doing imperative programming, so encapsulation relies on namespacing and (kind of) casting objects to the right class before passing them to a function (as classes don't seem to know about their parent class, as such). Having said all this, one smart feature GObject does support is introspection, which makes it possible to automate binding other languages to a C library.

So, with my background, and with the goal of making Moblin development easy and attractive, I am keen to find alternatives to having to program in C all the time. Dynamic, interpreted languages (like Ruby or JavaScript) seem to me to be better suited to exploratory, iterative programming: the kind of programming most people do most of the time. This is a goal shared by the GObject Introspection project:

C is good for graphics, multimedia, and lower level systems work. However, writing complex software is difficult and error-prone without garbage collection. A managed runtime such as JavaScript, Python, Java, Lua, .NET, Scheme etc. makes a lot of sense for non-fast-path application logic such as configuration, layout, dialogs, etc. ... Thus, one of the major goals of the GObject introspection project is to be a convenient bridge between these two worlds, and allow you to choose the right tool for the job, rather than being limited inside one or the other. (from http://live.gnome.org/GObjectIntrospection)

GObject Introspection is already being used in GNOME development, to generate JavaScript bindings for the GNOME libraries: the next version of the GNOME desktop (in GNOME 3), GnomeShell, uses JavaScript extensively (calling Clutter, the Moblin graphics library, incidentally). Gjs is the name of the project producing the bindings, as well as being the home of the JavaScript engine itself (also called gjs, and based on Mozilla's implementation of JavaScript in C, SpiderMonkey).

JavaScript is an interesting language. I've used it in web applications for quite a long time (since about 2000), but never entertained the idea of using it for anything other than client-side web programming. Occasionally I'd come across server-side JavaScript (persevere), JavaScript implementations like Rhino, and even wrote some ASP code in JScript (as Microsoft called it) a few years ago. Frameworks like jQuery and the thorough integration of JavaScript with Rails meant that I've done more of it in recent months, but I never thought of using it as a general purpose language, or one for desktop development.

Looking for an alternative to C, and hearing from my colleagues about JavaScript, made me decide to get a feel for Gjs et al, as well as write something to run on Moblin. I decided to write a JavaScript application which made use of the Moblin graphics library, Clutter. Eventually, I ended up successfully writing a JavaScript implementation of tic-tac-toe, using Clutter for the user interface, and even including some simple animation. As an application, it is a bit scrappy (the code is attached), but as a learning aid, doing the project was very useful. It wasn't an easy journey, but it was an eventually satisfying one.

Using JavaScript for the not-web

I did my development work on a Fedora Core 11 laptop, with an Intel chipset.

I tried a variety of ways to install all the pieces I needed for JavaScript development with Gjs, GObject Introspection, GNOME libraries, and Moblin libraries. Even getting an idea of what all the pieces do is painful, and the documentation is pretty woeful. I ended up doing lots of source code grep-ing and googling.

In the end, it turns out that the following packages are available for Fedora Core:

• gobject-introspection: This provides tools and APIs for working with GIR XML files and typelib (binary versions of the GIR files): these are two formats for storing introspection data about GObject classes (basically, mapping the API of a C library into language-neutral descriptions of the APIs). The metadata for several common libraries (like GLib) is part of this package.
• gir-repository: This provides more introspection metadata about other common libraries (like GTK+ and DBus-GLib)
• gjs: This enables you to use the gobject-introspection data to access C libraries from JavaScript (i.e. it's the JavaScript bindings for GNOME libraries).

Now, unfortunately, the versions in the repositories for Fedora 11 are out of date: generating the Clutter bindings requires gobject-introspection-1.0, which isn't available. Plus, this code is so new that it's changing very rapidly, and you probably need the latest version from the source repository to get stuff to work.

To compile all these bits, you're going to need the basic build tools. On Fedora, that means this lot:

$ sudo yum install git flex bison gnome-common libgnome-devel libffi-devel \
    python-devel libtool automake autoconf make gtk-doc xulrunner-devel

(And maybe others: I lost track. But if you're trying to do this yourself, you'll probably know what to do if something's missing. Anyway, Josh is going to sort some packages out before long, hopefully. If you know a better way, let me know.)

Once you've got those installed, though, you should be able to build everything from source fairly easily.

As we're installing into /usr/local, first thing to do is make sure the compiler/linker can find your custom-compiled packages. In your ~/.bashrc file, add this line:

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig

This tells the pkg-config tool (used by autotools to locate libraries and headers) where your custom installations are. Then reload your bash settings with:

$ source ~/.bashrc

to make sure they're picked up. Any future bash sessions will use this setting. Now you're ready to build.

gobject-introspection:

$ git clone git://git.gnome.org/gobject-introspection
$ cd gobject-introspection
$ ./autogen.sh --prefix=/usr/local
$ make
$ sudo make install

gir-repository:

$ git clone git://git.gnome.org/gir-repository
$ cd gir-repository
$ ./autogen.sh --prefix=/usr/local
$ make
$ sudo make install

gjs:

$ git clone git://git.gnome.org/gjs
$ cd gjs
$ ./autogen.sh --prefix=/usr/local
$ make
$ sudo make install

You can check gjs is working by doing:

$ /usr/local/bin/gjs
gjs> 1+1
2
gjs>

It's a good idea to add /usr/local/bin to your path, again in ~/.bashrc:

export PATH=$PATH:/usr/local/bin

(and source ~/.bashrc to pick it up if necessary)

To compile Clutter itself, you'll also need these library headers:

$ sudo yum install glib2-devel mesa-libGL-devel libXext-devel \
    libXfixes-devel libXdamage-devel pango-devel cairo-devel

(Again, maybe others too.)

Once you've got those too, you can checkout Clutter using git, and build it with introspection turned on:

$ git clone git://git.clutter-project.org/clutter
$ cd clutter
$ ./autogen.sh --prefix=/usr/local --enable-introspection
$ make
$ sudo make install

The final step is one which doesn't appear to be documented anywhere, but which is utterly vital: without it, gjs can't find any of the introspection metadata, and won't be able to call any of the C libraries through their JavaScript bindings. Add this line to your ~/.bashrc file:

export GI_TYPELIB_PATH=$GI_TYPELIB_PATH:/usr/local/share/gir-1.0/

(Again, use source if necessary.)

Testing gjs with Clutter

To test the install, and make sure the Clutter JavaScript bindings are working, put this code into a file test.js:

const Clutter = imports.gi.Clutter;
Clutter.init (0, null);
let stage = Clutter.Stage.get_default ();
stage.title = "Test";
stage.set_color(new Clutter.Color( {red:150, blue:0, green:0, alpha:255} ));
stage.show ();
Clutter.main ();
stage.destroy ();

Run it from a command line with:

$ gjs test.js

You should see something like this:

If this works, you should be able to run my tic-tac-toe (semi-)implementation, linked at the end of this blog entry. To run it, copy the two files into a directory, remove the .txt suffixes, then get a command line in that directory and run:

$ gjs ox.js

There's no AI (so you'll have to play both sides), but notice the use of a scale animation as each move is written to the board. Also interesting is how you can use anonymous JavaScript functions to define callbacks for signals.

I've not experimented extensively, beyond calling a few bits of the Clutter and Netbook Toolkit APIs. If you want more examples, have a look at the GnomeShell tests and UI code. The mx tests are another source of sample code I used.

Another tip: if you're trying to work out what's available in the JavaScript bindings, there's unfortunately no nice way to do it at the moment. Your only recourse is to grep the XML GIR files, which are located in:

/usr/local/share/gir-1.0/

Reading these is a bit of a chore, and I also found it difficult to figure out what was in the API. For example, if you're trying to use a Clutter constant like CLUTTER_X_AXIS, the GIR file has this entry:

<namespace name="Clutter"
             version="1.0"
             shared-library="libclutter-glx-1.0.so.0"
             c:prefix="Clutter">

... loads of cruft ...

  <enumeration name="RotateAxis"
                  doc="Axis of a rotation."
                  version="0.4"
                  glib:type-name="ClutterRotateAxis"
                  glib:get-type="clutter_rotate_axis_get_type"
                  c:type="ClutterRotateAxis">

    <member name="x_axis"
               value="0"
               c:identifier="CLUTTER_X_AXIS"
               glib:nick="x-axis"/>

  </enumeration>

... loads more cruft ...

</namespace>

So, to reference the CLUTTER_X_AXIS constant, my first thought was look up the member element with a c:identifier corresponding to the constant I want; then use the path from the namespace, through the enumeration name, to the member name, i.e.:

Clutter.RotateAxis.x_axis

But in fact it's:

Clutter.RotateAxis.X_AXIS

(the name attribute, capitalised).

I also found that some methods and classes in the GIR for GLib didn't work (like GLib.printf). I'm sure there's a good reason why, it's just I couldn't work it out.

What's missing?

I spent quite a few hours figuring this stuff out, and stumbled a lot along the way. Partly because I'm not a C programmer, partly because I'm new to GNOME, I'm sure. But I think there are a few things which would have improved my experience:

• Decent documentation, particularly of the JavaScript APIs generated for C libraries like GLib - at the moment, working out what's going to be available in the JavaScript bindings is very tedious
• An easy installation process (see above) - largely though because I'm using libraries which are too bleeding edge, just because they're the only ones which worked well
• An intelligible module system, for modularising code - at the moment my application will only run from the directory where the files are, as I've yet to work out how to get the path to the directory containing the script programmatically; I'm not sure how you'd organise large groups of files spread over lots of directories
• A decent packaging system with dependency tracking, so that you can distribute JS apps and know they'll work
• Functions to make developers' lives easier, like a decent REPL (gjs doesn't even have a print function - there is a patch to gjs which will give you that - see here: http://bugzilla-attachments.gnome.org/attachment.cgi?id=135898 - but you'll have to patch manually at the moment to get it)

And finally, it might be worth taking a look at Seed, which is an alternative JavaScript implementation, but one which still understands GObject introspection. Seed is interesting in that it seems to provide more niceties, like a better REPL environment, more documentation, and a fuller standard library (filesystem and environment interfaces in particular).

What's next for JavaScript?

It looks as if the JavaScript world is hotting up at the moment. This is a recent news report on the state of JavaScript:

http://arstechnica.com/web/news/2009/12/commonjs-effort-sets-javascript-...

It mentions the CommonJS standardisation effort, which is attempting to resolve some of the shortcomings of JavaScript as a general purpose language.

Reading through the CommonJS wiki, I was particularly interested in Narwhal, which resolves some of the issues I mentioned in the last section: it includes a package manager, module system, and standard library. It also provides support for different JavaScript engine back-ends: currently this includes a SpiderMonkey adapter. Given that Gjs is broadly based on SpiderMonkey, it seems likely that creating a Gjs back-end for Narwhal wouldn't be too onerous.

One other thing which could potentially be useful in Narwhal is the availability of separate Narwhal environments (called "seas") with their own set of packages. This could help isolate applications from each other in an environment like Moblin, where you want third parties to be able to develop applications without trampling on each other's libraries.

Another tool of interest is jake, a port of Ruby rake to JavaScript, which seems to be optimised to work with Narwhal. This could provide a reasonable alternative to autotools for the simpler build requirements of a JavaScript application (where there's no need for a compile step or library linking).

A combination of Narwhal (packaging/modules), with the power of Gjs binding to pretty much any C library, would make an attractive environment to develop in.

I should also mention node.js, server-side JavaScript primarily designed for building network programs, but which incorporates a CommonJS module system. It's built on Google's V8 JavaScript engine. It's also interesting because it is designed to be highly performant and memory-efficient, and has an interesting event-driven I/O model, like EventMachine or Twisted.

I'm definitely looking forward to seeing what happens next.

Attachment	Size
ox.js.txt	5.31 KB
board.js.txt	3.46 KB