Using the Plug class to access Amazon S3

Basics

At the core, our program is very simple.  It will allow a user to list the contents of a bucket and to upload a file.  We won't bother with a fancy user interface.  We really only want the core functionality.

First, we need the access (i.e. public) and private keys for S3.  We'll assume they are passed as the first and second argument to our application.  We'll store them into two global variables: _public_key and _private_key.  Also, we're going to initialize a global _s3 variable as a plug.  A plug combines the capabilities of the URI and WebRequest classes, but better!

_public_key = args[0];
_private_key = args[1];
Plug _s3 = Plug.New("http://s3.amazonaws.com");

Next, we'll expect a command as the third argument.  This can either be "list" or "add" for listing the contents of a bucket or uploading a file.

XDoc result = XDoc.Empty;
switch(args[2]) {
case "list":

    // TODO: handle list bucket
    break;
case "add":

    // TODO: handle upload file
    break;
default:

    // unrecognized command
    return;
}

List Bucket

Ok, now let's get to the interesting part.  S3 allows you to list a bucket by doing a GET operation on the bucket name.  We'll assume the bucket name is passed in as the fourth argument on the command line.  Here is what we need to do:

string bucket = args[3];
XDoc result = _s3.At(bucket).Get().AsDocument();
Console.WriteLine(result);

Kind of anti-climactic, but that's because plugs are so incredibly simple to use!

Upload File

Next, we want to support uploading a file for which we assume the file path is passed in as the fifth argument on the command line.  This is accomplished with a PUT operation on the bucket with a filename appended.  Again, Dream makes this amazingly simple:

string bucket = args[3];
string filepath = args[4];
string filename = System.IO.Path.GetFileName(filepath);
XDoc result = _s3.At(bucket, filename).Put(DreamMessage.FromFile(filepath)).AsDocument();
Console.WriteLine(result);

That's because a DreamMessage can be instantiated directly from a file.  This enables Dream to stream the request directly from disk to the remote server without memory overhead or complex coding.  The Dream framework takes care of it all!

S3 Authentication Header

Last, but not least, we have to handle the trickiest issue with S3: the generation of the authentication header.  Normally, this would be rather painful.  However, in Dream, all we need to do is configure our Plug appropriately.

First, let's create the the a method to generate the authentication header.  This method will assume it gets the message we want to send, the verb we're using, and the destination URI.

private static DreamMessage S3AuthenticationHeader(string verb, XUri uri, DreamMessage message) {

    // add amazon date header
    string date = DateTime.UtcNow.ToString("r");
    message.Headers[AWS_DATE] = date;

    // add authorization header
    string result = string.Format("{0}\n{1}\n{2}\n\n{3}:{4}\n{5}", 
        verb, message.Headers[DreamMessage.HEADER_CONTENT_MD5], message.ContentType, AWS_DATE.ToLower(), date, uri.Path
    );
    HMACSHA1 hmac = new HMACSHA1(Encoding.UTF8.GetBytes(_private_key));
    string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(result)));
    message.Headers[DreamMessage.HEADER_AUTHORIZATION] = string.Format("AWS {0}:{1}", _public_key, signature);
    return message;
}

Second, we'll change the initialization of our global S3 plug to use this handler.  This is accomplished by adding a pre-handler to the plug.  Pre-handlers are invoked just before a message is sent and they can be used for doing last minute modifications to the message.  In our case, this modification meant adding the correct authentication header. ­

Plug _s3 = Plug.New("http://s3.amazonaws.com").WithPreHandler(S3AuthenticationHeader);

Again, that' it!  Nothing else to do.  All uses of _s3 will now invoke our handler for generating the correct S3 authentication header.

Conclusion

Our sample S3 console application is now complete (although the attached version has a few usability improvements).  S3 allows for several more interactions, such as creating and deleting buckets, deleting files, and more.  However, these are left as an exercise to you (documentation for S3 is here).  The important take away of this example is that you can embed rich interactions with other RESTful web-services very easily into your own code without requiring much code.