[Beowulf] Clustering VPS servers

[Jonathan Aquilina]

Derailing this topic slightly do you have any good tutorials as to steps in
regards to compiling the kernel?

-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On
Behalf Of Douglas Eadline
Sent: 21 March 2013 14:19
To: Eugen Leitl
Cc: beowulf at beowulf.org
Subject: Re: [Beowulf] Clustering VPS servers


> On Thu, Mar 21, 2013 at 08:42:17AM -0400, Joe Landman wrote:
>
>> Honestly this never made a great deal of sense to me.  Rather than 
>> clustering VPSes, why not cluster bare metal JEOS kvm hypervisor 
>> built machines?  Not quite VMware stuff.  We are using lots of kvm 
>> (obscene amounts of it) in our projects across several OSes.  Linux, 
>> Illumos/SmartOS.  We've been tweaking tiburon to handle such kvm 
>> boots, so that turning on a very large cluster of virtual machines 
>> and having them ready would take seconds to minutes at worst.  Not 
>> half hours to several hours for provisioning.
>
> Not sure I plugged this here before: http://erlangonxen.org/ It's 
> proprietary so far, but such solutions can be replicated in the FLOSS 
> domain.

Interesting, although I like Erlang for non HPC type things.

I sometimes like to think of the OS as a dynamic support library for the
application. From a performance standpoint, most things have become bigger
and faster and yet the Linux kernel has stayed about the same size and once
tuned for HPC it is down right small in comparison. Of course, some
applications need other supporting software (Python,Perl) but the amount of
software needed directly on a node is quite small (I use a fairly robust
Warewulf ramdisk image that is much less then 200M, which is less than 2.5%
of system RAM on a node with 8G) To be fair, the image also uses NFS for
things like /usr/share. I have often thought that highly tuned application
ramdisk images combined with runtime provisioning by the scheduler is one
way to achieve "bare metal HPC virtualization"

--
Doug






>
>> VPS doesn't quite have the isolation of kvm, which is part of why I'd 
>> like to see that.  But kvm doesn't have great PCIe pass through (yet, 
>> its getting better).  VPS might be able to make better use of the 
>> resource.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin 
> Computing To change your subscription (digest mode or unsubscribe) 
> visit http://www.beowulf.org/mailman/listinfo/beowulf
>
> --
> Mailscanner: Clean
>


--
Doug

--
Mailscanner: Clean

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To
change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf