[Beowulf] Utility Supercomputing...

[Mark Hahn]

>> again, if you have a bursty load, you want to use a shared facility,
>> of which, EC2 spot instances is just one example.
>
> When talking of using a shared facility: The advantage I've
> sometimes seen for "cloud" over the traditional "get time on a
> shared cluster"

I should be clear - the kind of cluster I'm talking about is 
more collegial than the big-time US national labs.  users
just get an account on Sharcnet (and most ComputeCanada) and 
help themselves to the scheduler.  no asking for time.
(we do have a mechanism for allocations as well, but that 
approach has a very different, generally less satisfying dynamic.)

> is that you get to define a lot more of the
> IT policy and configuration.

that's certainly true, but IMO most researchers just want to get 
the work done.

> You get to choose your OS, scheduler,
> compute node configuration, some subset of the security policy which is
> compatible with "cloud", etc. You also get your own IT folks managing
> it.

I would turn that around completely.  if you're using EC2,
you are queueing jobs in Amazon's cluster: the job allocates your VM hosts,
but you are by definition subject to Amazon's config - you just get
to layer your own OS/etc on the VM.  EC2 runs on hour granularity,
and not less than one instance, but it really is a cluster.

further, I can take my conventional PaaS cluster and provide a way for
user users to request that the job start a VM.  bang, quacks like EC2,
just less efficient and a lot more sysadmin work (for everyone).
(to really make it like EC2, there would be some additional work to set up 
per-job routing, not to mention auxiliary services like EBS)

regards, mark.