[Beowulf] Best Way to Use 48-cores For Undergrad Cluster?

[Joshua mora acosta]

If you do virtualization you may want at least to pin the guest OS to each
core and provide a "quota" of main memory local to that core to that guest OS.
In other words avoid remote accesses.
In that way you can at least guarantee some quality of service (capacity,
performance, security).
You could also have other queues that are bigger that would use more cores
with proportional amount of memory also local if resources are available.
You could even go further by "carving" the number of cores and memory amount
and the proper pinning provided when you submit the job it reads those
specifications and then if those HW resources are available then you would
start that guest OS with those specifications and then run that workload.
This last thing would be the dynamic provisioning.
Another good experiment would be the migration from one guest OS to another
guest OS (checkpoint on guest OS 1 and restart on guest OS 2).
You can "carve" resources also based on tasks. For instance you may need some
good preprocessing capability in terms of amount of RAM but just single core.
If having to "move" around memory for cores that are located on remote die
then you want to configure that region as memory interleaved.
Finally, knowing a bit about how the application stresses the HW you may want
to "carve" those resources in such a way that they do not produce "congestion"
on subsystems such as memory controller so other task being run on same die by
other guest OS do not suffer from the contention originated by the other very
demanding task running in the other guest OS.
It is certainly a great exercise to increase productivity rather than increase
performance of single tasks.
Another idea is to create queues that would be configured as low power
consumption where you would downclock the cores as much as you can without
affecting the others.
And before I forget, every device hanging from chipset (Eth, IB NICs,GPUs) can
be also virtualized thanks to IOMMU features.

Best regards,
Joshua Mora.

------ Original Message ------
Received: 07:57 PM CDT, 05/06/2010
From: Jon Forrest <jlforrest at berkeley.edu>
To: "beowulf at beowulf.org" <Beowulf at beowulf.org>Cc: 
Subject: [Beowulf] Best Way to Use 48-cores For Undergrad Cluster?

> Let's say you were going to set up a cluster
> for undergraduates to learn how to use
> SGE and run typical chemistry applications.
> Ultimate performance is not the primary goal.
> Let's say you get charged for rack space
> in your data center and there's very little
> budget to pay for space.
> 
> I see you can now get 48-cores in one 1U box.
> What do you think about running all the
> compute nodes as 1-core virtual machines on
> the one box? Or, would you just run
> the machine with one OS and a SGE
> queue with 47 slots (with 1 core for the frontend)?
> 
> This is probably not the kind of environment
> most of you run in, but it does present
> interesting issues.
> 
> Cordially,
> -- 
> Jon Forrest
> Research Computing Support
> College of Chemistry
> 173 Tan Hall
> University of California Berkeley
> Berkeley, CA
> 94720-1460
> 510-643-1032
> jlforrest at berkeley.edu
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
>