[Beowulf] 10 kCore cluster in Amazon cloud, costs ~1 kUSD/hour

[Ellis H. Wilson III]

On 04/07/11 11:03, Chris Dagdigian wrote:
> One of the nice things about their work is how "usable" it is to real 
> people with real production computing requirements - in the IAAS cloud 

I wonder what "real" people with "real" production computing
requirements means here.  See below for further thoughts on my thoughts
on "real" codes and where I suspect they arise.

> It's not a PR gimmick and limiting the definition of "cluster" to only 
> systems that run parallel applications would alienate quite a few of us 
> on this list :) In the life sciences a typical cluster might run a 
> mixture of 80-90% serial jobs with a small scattering of real MPI apps 
> running alongside.

I'm certainly a pragmatist here - use the machines as your organization
feels is best.  However I still have a strong suspicion that most jobs
are serial because of:

1. Lack of experience properly parallelizing codes
2. Lack of proper environment on one's own desktop (i.e. Linux or group
licenses)
3. In rare cases such rapid development and short lifetime of a code
that parallelizing it will take longer than poorly serially coding it
and tolerating the run-times.

I can only hope that within the decade the programming paradigm shifts
along with the hardware and the average bloke becomes at least exposed
to basic parallel programming concepts.

The machine is still a "cluster" - the way it's used shouldn't guide
what it is referred to.  That doesn't mean running serial jobs on a
machine tailored for parallel ones is the best way to use your
time/money.  Probably better for one to simply buy Linux desktops for
all the employees, put them on a typical GigE network and have the
employees submit jobs to some tiny server in the Bosses office which
routes jobs evenly to everyone's machine distributed throughout the
building.

ellis