Accessing Amazon's SimpleDB

The next test in my 'proof of concept' phase of development is to see if I can access Amazon's SimpleDB cloud service. The reason for picking Amazon's SimpleDB is threefold:

1. IF (big 'if' I know) my efforts to develop a Home Smart Energy System turn out to be a resounded (financial) success, I don't want to have to put together all the hardware needed for the huge database that is going to be needed to store all the telemetry related to time-dependent energy usage and sensor configuration / control for potentially thousands (and more if I'm real lucky!) of homes. Cloud services offer a scaleable approach that gets billed in proportion to one's usage;
2. The Amazon SimpleDB has a particularly simple API;
3. And the Amazon cloud was the first of the big three (Amazon, Google, Microsoft) I believe so I'll go with the one that has years of postings to support forums to help with my own development.

I've performed a few tests in the past using VB.NET, but now I'm interested in access SimpleDB from IronPython running on Mono. At this point I'm running IronPython 2.6.10920.0 and Mono 2.4.2.3.

The first step is to get hold of a library to do all the work for me. For no particular reason I went for the C# library I found here. The version I downloaded was released on May 20, 2009. I also found a page in the IronPython Cookbook that talks about using an older version of this library in IronPython. The steps for creating the DLL are still valid, but the test scripts themselves are out of date so not much use to anyone (other than to waste some time wondering why one's scrips keep failing!). I have posted the version of the DLL I compiled here.

Boto is another more comprehensive library written in Python that is for all the Amazon cloud services, not just SimpleDB.

Next, you've got to get yourself a SimpleDB account from here. Once you have registered you'll be given a unique AccessKey ID, and a Secret Access Key, both of which are needed to access SimpleDB service.

One more step require: I had performed a few tests in IronPython that worked just fine and then failed when running under Mono. After some googling I found that the issue was that Mono doesn't trust anyone, and so any SSL web accesses fail by default... so of course my Mono IronPython SimpleDB scripts failed! To save you the hassle, all you need to do is install a pile of root certificates. This is done by executing the following at the command line:

mozroots --import --ask-remove --machine

More details about Mozroots can be found here. If you have the latest version of Mono then Mozroots should already be installed.

OK, now we're ready to go. I'd already added some data to SimpleDB and so we'll just look at a simple query in this posting. To run this for yourselves you're going to have to setup you AWS account with Amazon, and stick some data in SimpleDB. I'll add some data entry snippets another time.

The following code is for a simple SimpleDB test that I saved as simpleDB_examples.py:

import clr
clr.AddReference('Amazon.SimpleDB')
from Amazon.SimpleDB import AmazonSimpleDBClient
from Amazon.SimpleDB.Model import ListDomainsRequest, Select Request

service = AmazonSimpleDBClient('[Access Key ID]','[Secret AccessKey]')

# list domains

listDomainsRequest = ListDomainsRequest()
listDomainsRequest.MaxNumberOfDomains = 10
listDomainsResponse = service.ListDomains(listDomainsRequest)

print listDomainsResponse.ListDomainsResult.DomainName

# select items request

selectRequest = SelectRequest()
selectRequest.SelectExpression = 'Select * From [domain]'
selectResponse = service.Select(selectRequest)

# print first attribute namd and value for first returned item

print selectResponse.SelectResult.Item[0].Attribute[0].Name
print selectResponse.SelectResult.Item[0].Attribute[0].Value

Once saved, the test script can be run by executing:

mono ipy.exe simpleDB_examples.py

As I had already created a couple of domains (equivalent to 'tables'), the following was printed to screen on my Ubuntu machine:

List[str]([dmg_data', 'testDomain'])
node_type
end node

Another success! Now we have WinForms, SerialPort access and SimpleDB partially proven on Ubuntu with Mono and IronPython. With a Multi-Threading/Processing example I think I'm nearly ready to get stuck into the main application.

Side note: My first PlugPC (TonidoPlug) was delivered this week, so plenty of fun ahead!