OCamlPAM 1.0 Released!

PAM is a slick policy-based authentication mechanism. It abstracts away the method of authentication from applications and makes it possible to change the authentication method for a deployed application/service while running instead of making that decision at compile-time. I've come to love PAM because it makes single sign-on a possibility and lets me focus on my application logic rather than the details of, say, LDAP authentication.

Since I've been playing around with Objective Caml lately and I needed to do some authentication, I wrote an OCaml wrapper for PAM. Take a look, give it a go, and authenticate away!

Power Profiling: Saving Energy in Software

I came across a pretty nifty profiling tool a little while ago called PowerTOP. It gives you a live view of the power consumed by applications running on your Linux machine in an interface that resembles top. As a system-level power profiler, it gives you an idea of which applications or drivers are waking up the CPU or preventing it from entering a sleep state.

I think this kind of tool is awesome for both developers and end-users. As a user, it tells me which applications are draining my laptop's battery so I might close those applications if I'm not using them or I might find an alternative altogether. From what I understand, some people have gained significantly higher use-times for their laptops by just killing innocuous-looking apps that don't let the processor enter a deeper sleep state. As a developer, this is good news because it will give me a quantitative measure of my application's power consumption, giving me a means to determine the power-efficiency of different designs. This tool has also managed to expose bugs in a few applications!

Check out PowerTOP, fix your applications, and keep this planet green (and my laptop running longer and cooler)!

A Closer Look: Amazon S3

One of the most mature Amazon web services, Amazon Simple Storage Service (S3) provides a virtually unlimited data storage service. That's right: you can upload as much data as you'd like and it will be held on their machines with all the network capacity you could ever want and with redundancy built-in. Hard drive failures are easily the primary cause of server downtime and Amazon has taken the burden upon themselves to manage all the devices and failures that go along with it. As the name implies, the service is designed to provide simple access so you can't do funky things like mount the virtual filesystem directly.

I've been using S3 for over a year and I haven't had any reliability issues with it. Others have had brief outages but they were mostly when the service was first introduced. I'm quite happy with S3 but there's one missing feature that keeps it from being the ultimate simple storage service: range-PUT.

Suppose I've got a file on S3 and I want to update a small part of it. Without range-PUT, I would normally have to transfer the entire file again using the HTTP PUT method to store it on the remote host. Using the Content-Range header, I could specify just the range of bytes that have changed within the file and transfer just that portion. This feature would save a lot of bandwidth (and, consequently, money) if files often get modified partially.

Of course, supporting Content-Range opens up a can of worms. What happens if the file doesn't exist and the start of my range isn't offset 0? What if the file does exist but the start offset is beyond the end of file (i.e. not a simple append)? I can think of two solutions that seem reasonable: return an error or create the file if it doesn't exist and zero-pad the holes. The former would be easier to implement while the latter would produce a behaviour like Linux sparse files.

There are two major application classes that range-PUT would be suited to. The first would be the class of applications where we always append to the end-of-file. Log files would fit into this category but, more importantly, we could resume broken transfers. When uploading large files (S3 supports file sizes of up to 5GB), I've found that my connections often get dropped so if I could just append to an existing file, I could write an upload tool that would auto-resume. The second class of applications would be the ones that only update part of a file. In most cases, I'd imagine this kind of update would take place to change some file metadata. For example, if I modify the metadata for my MP3 file, I'd rather just upload the few changed bytes instead of uploading the whole MP3 again. The music is the same, it's just the metadata that has changed. This problem is even worse when dealing with video files.

S3 is a fantastic storage service. It's reliable, it's cheap, and it takes away the hassle of managing your own hardware or creating a highly-available, redundant persistent store. If S3 supported range-PUT, it would save a huge amount of bandwidth resulting in an even lower cost of operation.

A Closer Look: Amazon Web Services

Amazon has been doing some pretty nifty stuff lately. They've exposed their computing infrastructure to the rest of the world via RESTful web services. I think it's a brilliant move by Jeff Bezos and it realizes some of the technology promises of the last decade or so.

Amazon's web services have been getting a lot of great reviews from many bloggers and deservedly so. With their pay-as-you-use model, it's amazingly easy to scale up or down based entirely on workload; there's virtually no need for capacity planning and no need to manage physical systems at all! I've been using their web services for over a year now for a variety of tasks and it has been, for the most part, quite pleasant. They've saved me a lot of time and money but their services do have their faults. I'll be talking about some of the issues I've encountered with the design or implementation of their services over the next few posts.

Company Loyalty

Jerry Yang, CEO of Yahoo!, recently sent out a company-wide memo in response to Microsoft's $44bn hostile bid. Here's a rather hilarious "human readable" version of the memo courtesy of John Gruber.

While I don't 100% agree with the translation, I find it amusing that we actually need to translate. You'd think that such an important message would be a bit more... accessible.

The memo makes me wonder, though: how often does senior management actually point out the importance of their employees outside of annual company meetings and circumstances such as this? Yang is really asking for loyalty through a difficult time for his organization. Did he build that loyalty or is he just asking for it? I don't know, but I'd be curious to find out.

Protector

A while ago, I had come across this Flash game called Tower Defense. The idea of the game is to put up guard towers along a path to the castle to prevent enemy attackers from killing your citizens. It was fun but got frustrating as the levels wore on because the income didn't keep up with the increasing strength of the attackers.

Then, a couple of days ago, I came across Protector, a game based on the same principles but with an RPG-style twist. It's actually pretty awesome! I have to admit, for a Flash game, it's quite polished with a tutorial mode and everything. Here's a screenshot of the game in action:


I find that console games have gotten too complex to appreciate. They try to create an immersive experience but most of the time, I just want a quick fix so I can relax. Sure, there are first-person-shooters, but they're all about the same and they get pretty boring pretty fast. I have a feeling that the success of the Wii and its throw-back to the classic, simple games of the 90's show that there's still demand for quick gaming.

Google Web Toolkit: a quick look

While writing the last blog post, I had a miserable time trying to get the Java snippets formatted correctly. I suppose the problem is that I'm using Blogger instead of a full-blown publishing platform like Movable Type which has plugins for code highlighting. Once I got the post up, I decided to build a web service that would take a code snippet in your language of choice and return highlighted HTML code.

Along the way, I found a neat package called GeSHi, a generic code highlighter that would make my life a lot easier. I then got the idea of making an AJAXed front-end to that along with other useful web services... Eventually, I came across the Google Web Toolkit and it looks like the Google folks are really on to something!

Now, I'm sorry to say that I've done my share of AJAX code before, both client side and server side, and it has been one of the most painful coding experiences of my life. The runtime errors, the lack of decent developer tools, the browser incompatibilites, the ridiculous workarounds... web development has been just one enormous nightmare. The guys and gals at Google managed to take a large part of that away with GWT; they're actually compiling Java code to JavaScript with all of the compile-time checks (hooray, no more typo-bugs rearing their heads at runtime!), development tools (including a debugger!), and patterns kept intact! The really nice bit is that they abstract all of the cross-browser differences via their core library. Now this is a step in the right direction for web development!

Of course, it's not a perfect system. They're building this on top of Java 1.4 so we still don't have safe container classes or enumerated values (sigh). What's more, since AJAX is event driven by nature, I think it would be so much easier to use a functional source language so we can pass functions and closures around instead of anonymous inner classes.

When goodbyes aren't enough...

The Java core classes have a pretty nasty habit of throwing IOExceptions in close methods. Since IOExceptions are not derived from RuntimeException, you either have to catch them or declare them as thrown from the method calling close. Now, suppose I have a class that holds a couple of resources, r1 and r2. Let's see what happens if I try to release them together:

try
{
    r1.close();
    r2.close();
}
catch(IOException e)
{
    // Do whatever I need to here...
}

If r1's close happens to throw, r2's close will never be called and I might (in a long-running application) leak resources. The same problem occurs if I don't bother catching the exception and declare it thrown from my method. Putting the close calls in the finally clause won't fix the problem either. A working solution could be:

try
{
    r1.close();
}
catch(IOException e)
{
    // Do whatever I need to here...
}
try
{
    r2.close();
}
catch(IOException e)
{
    // Do whatever I need to here...
}

This code looks worse than checking for return codes! What's more, I, as a programmer, am getting rid of a resource. I have decided that the given resource is no longer of any use to me. I am throwing it away. And it's coming back!

I can't think of a single reason an exception (checked or unchecked) should be thrown from a method that is releasing a resource. In situations like this, a return code should be used instead of exceptions because the programmer has, by the very act of calling your method, declared that she is no longer interested in the resource.