[Beowulf] $1, 279-per-hour, 30, 000-core cluster built on Amazon EC2 cloud
raysonlogin at gmail.com
Tue Oct 4 08:58:12 PDT 2011
On Tue, Oct 4, 2011 at 11:26 AM, Lux, Jim (337C)
<james.p.lux at jpl.nasa.gov> wrote:
> The classic: nothing beats a station wagon full of tapes for bandwidth.
> (today, it's minivan with terabyte hard drives, but that's the idea)
BTW, I've heard horror stories related to routing errors with this
method - truck drivers delivering wrong tapes or losing tapes
(hopefully the data is properly encrypted).
> Notwithstanding that there ARE places that do cycle harvesting from
> desktop machines, but the management and sysadmin hassles are so extreme
> (I've written software to DO such harvesting, in pre-Beowulf days).
The technology part of cycle harvesting is solvable, the accounting
part is (IMO) much harder.
A few years ago I talked to a University HPC lab about deploying cycle
harvesting in the libraries (it's a big University, so we are talking
about 1000+ library desktops). The technology was there (BOINC
client), but getting the software installed & maintained means extra
work, which means an extra IT guy... and means no one wants to pay for
I wonder how many University labs or Biotech companies are doing
organization wide cycle harvesting these days, for example, with
technologies like BOINC:
> Where an Amazon could do themselves a favor (maybe they do this already)
> is to provide a free downloadable version of their environment for your
> own computer,
AMI is not private (in the end, it is IaaS, so the VM images are open).
In fact, StarCluster has AMIs for download & install (mainly for
developers who want to code for StarCluster locally):
And one can roll a custom StarCluster AMI and upload it to AWS, such
that the image settings are optimized to the needs:
> or some "low priority cycles" for free, to get people hooked.
AWS Free Usage Tier -- (most people just use the free tier as free hosting):
Grid Engine / Open Grid Scheduler
> Sort of like IBM providing computers for cheap to universities in
> the 60s and 70s. Razors, razor blades. Kindles, e-books. Subsidized
> cellphones, 10 cent text messages. Give us your child 'til 7, and he's
> ours for life.
More information about the Beowulf