[Beowulf] Clustering VPS servers

[Douglas Eadline]

> On Thu, Mar 21, 2013 at 08:42:17AM -0400, Joe Landman wrote:
>
>> Honestly this never made a great deal of sense to me.  Rather than
>> clustering VPSes, why not cluster bare metal JEOS kvm hypervisor built
>> machines?  Not quite VMware stuff.  We are using lots of kvm (obscene
>> amounts of it) in our projects across several OSes.  Linux,
>> Illumos/SmartOS.  We've been tweaking tiburon to handle such kvm boots,
>> so that turning on a very large cluster of virtual machines and having
>> them ready would take seconds to minutes at worst.  Not half hours to
>> several hours for provisioning.
>
> Not sure I plugged this here before: http://erlangonxen.org/
> It's proprietary so far, but such solutions can be replicated in
> the FLOSS domain.

Interesting, although I like Erlang for non HPC type things.

I sometimes like to think of the OS as a dynamic support library
for the application. From a performance standpoint,
most things have become bigger and faster and yet the
Linux kernel has stayed about the same size and once tuned
for HPC it is down right small in comparison. Of course,
some applications need other supporting software (Python,Perl)
but the amount of software needed directly on a node
is quite small (I use a fairly robust Warewulf ramdisk image
that is much less then 200M, which is less than 2.5% of system
RAM on a node with 8G) To be fair, the image also uses NFS
for things like /usr/share. I have often thought that
highly tuned application ramdisk images combined with
runtime provisioning by the scheduler is one way to
achieve "bare metal HPC virtualization"

--
Doug






>
>> VPS doesn't quite have the isolation of kvm, which is part of why I'd
>> like to see that.  But kvm doesn't have great PCIe pass through (yet,
>> its getting better).  VPS might be able to make better use of the
>> resource.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
> --
> Mailscanner: Clean
>


--
Doug

-- 
Mailscanner: Clean