[Beowulf] cloud: ho hum?

[Mark Hahn]

> My take on it is if we've got a large, steady scientific HPC load then we'd
>want in-house capacity to cover that.

you mean "production", basically.  mostly fixed in size/length.

> But if we had a project that had small bursts of intense computation we
>might prefer to find a larger pool of compute resource - cloud could be one
>of the options. In fact cloud could well be the most straightforward. The
>alternative might mean slowing down a key piece of R&D project work.

you seem to be comparing to small HPC, sized to meed production demand.
I'm not talking about that at all: I'm assuming, perhaps unwarantedly,
that most large HPC facilities are like ours, with some modest production
demand, but with most of the workload already comprised of the interleaved
bursts from thousands of researchers.

> I'm not ignoring your points, I'm flagging up that our unusual burst of
>demand might be someone else's minor blip. It then becomes worth our while
>to offload that to an outsourced resource, whether cloud or not.

afaikt, you're just saying "bursts and production don't mix".  that's true,
but isn't it very small-scale?  handling burstiness just means finding a 
deep enough pool.  efficient use of that pool just means getting enough
(hopefully independently timed) bursters.  this is not an argument for 
outsourcing per se, or for private-sector somehow being more efficient.

there also seems to be a bit of class-warfare surrounding this issue:
the claim that "new" disciplines ("disciplines") like bioinformatics 
and big-data are poorly served by traditional HPC clusters.  they seem to 
resent spending on interconnect, for instance.  to me, this seems like 
novices being obliviously ignorant - sure, QDR to each node seems like 
a waste of money, but once you get 12,24,32 cores per node, you're going
to want to have something faster than Gb or 10Gb, even if you only ever 
use it for files, not MPI.  (for that matter, I think there's a natural
progression toward more complex processing as a field matures, which will 
lead fields that currently do serial farming towards "real" parallelism...)

there's a nasty "don't give them money because they don't do it right"
thing going on in the guise of cloud and (mostly) bioIT.

regards, mark hahn.