Ruby Tuesday: s33r (Ruby library for Amazon S3) available as gem

  • warning: Illegal string offset 'data' in /home/townxorg/public_html/includes/tablesort.inc on line 110.
  • warning: Illegal string offset 'data' in /home/townxorg/public_html/includes/tablesort.inc on line 110.

I've got round to packaging my latest attempt at an Amazon S3 library for Ruby into a gem. (For those of you unfamiliar with it, S3 is a network storage solution.) It is less complete than my previous effort, but more solid, more elegant, and with a test suite (which uses RSpec - I haven't added RSpec as a gem dependency (you can ignore the tests if you like), but for reference I developed the tests using RSpec 0.5.12). It has big chunks based on the Amazon S3 Ruby sample code, but I've wrapped it up and made it nice and neat. I've also done things like provide a mechanism for streaming files to S3 (the Amazon sample doesn't do this) using customisable chunk sizes (the Net::HTTP library in Ruby uses 1024 byte chunks by default; mine is customisable). There's lots left to do, of course, but I didn't want another Tuesday to go by without a Ruby Tuesday :).

Download the attached file and install it with a:

gem install s33r-0.1.gem

To use it from inside your Ruby script:

require 's33r'

If you want to see an example program which uses the library, have a look at bin/s3cli.rb. This is a simple command line client which you can use to post a file to S3 and email a link to the file to someone. Useful for sending big files to people without clogging their email inbox with enormous files. It is also intended as a demonstration of how to use the library. Full instructions are included at the top of the file.

By the way, I've tested this on Linux, but not on Windows or Mac.

Anyone aware of other S3 libraries for Ruby?

AttachmentSize
s33r-0.1.gem27 KB

Comments

easy

This certainly makes using the Amazon service a lot easier. I have been playing with various ruby installs on a rackspace account, but have not seen anything like this one. I hope that I can get it installed and find good use on my production machine. Thanks for all your work on this.

The library still seem a bit

The library still seem a bit slow, but it's difficult to track where the bottleneck is.. Thanks for sharing this informative read.

Thanks !

One idea I may like to add is a database list of the buckets with a "synch" option to, well - synch.
The database could also hold header and acl info?

Regards
Peter
(I was using the old pre gem version and did find it loaded a little slowly.)

Sync is something I'm

Sync is something I'm looking at at the moment. I definitely need this myself, as I'd like to do a kind of rsync to S3 eventually. And it would make sense to do some caching, though this would be tricky in a sync situation. The library still seem a bit slow, but it's difficult to track where the bottleneck is: in the Ruby code, the network, or S3 itself. I'm looking into it.

It din't work on window xp

It din't work on window xp

put_text() typo?

Thanks for making this gem elliot!

In the put_text() method, I think the params to put_resource() are transposed. Line 131 of client.rb should be:
put_resource(string, bucket_name, resource_key, headers)

instead of:
put_resource(bucket_name, resource_key, string, headers)

Or I could be completely wrong, I've only been trying it out for a few minutes.

Thanks for the comment

Dear Keaka, you're right, I was being inconsistent with parameter placements. I will sort this out before the next version.

Do you mean the S3 gem

Do you mean the S3 gem throws out an internal server error? Does this happen during connections to S3? Or are you talking about the Rails application?

It works fine on a Mac, but

It works fine on a Mac, but on FreeBSD, it chucks out a Http/1.1 Internal Server Error for any request. Any idea why this could be?