[Beowulf] Virtual resource manager

[Alan Louis Scheinine]

Andrew Holway wrote:
> I'd like to find out if there are any projects out there to develop a
> resource manager that can control a virtual cluster. We would like to
> explore the idea of using xen to deploy operating systems on nodes,
> checkpoint jobs and deploy MS ccs. Primarily interested in open source
> initiatives.

A good question.  I don't know the answer, nonetheless I would like to
mention one point of view.  From what I've seen with LSF and SGE, they
expect to have a certain set of computers with specific names.  In contrast,
with Xen that number of computers with different names and different
addresses is arbitrary.  But on the other hand, if you want to balance
the computational load, you need to know the number of actual processors.
I realize that you asked about a "resource manager" which does not necessarily
imply load balancing.  Nevertheless, to focus on the load balancing aspect,
it seems practical to have a batch system that is based on actual computers
so that the job manager knows how much real resources have been given, then
for a Xen-based job a set of computers runs a (parallel) job starts as a script
that creates the Xen processes and when finished returns the nodes to the job
queue pool as non-virtual machines.

I don't know what is available for others to use, we are developing something in-house.
It is not simple because the Xen processes will run parallel jobs, so they may
need a NIS server and DNS server for their specialized names and users.  Moreover,
for security in a Grid computing environment, each collection of Xen processes for
a parallel job will have its own VLAN.  So the script that starts the Xen collection
needs to also change the Ethernet switch.

I look forward to reading suggestions from other Beowulf list members.

Best regards,
Alan Scheinine
-- 

  Centro di Ricerca, Sviluppo e Studi Superiori in Sardegna
  Center for Advanced Studies, Research, and Development in Sardinia

  Postal Address:               |  Physical Address for FedEx, UPS, DHL:
  ---------------               |  -------------------------------------
  Alan Scheinine                |  Alan Scheinine
  c/o CRS4                      |  c/o CRS4
  C.P. n. 25                    |  Loc. Pixina Manna Edificio 1
  09010 Pula (Cagliari), Italy  |  09010 Pula (Cagliari), Italy

  Email: scheinin at crs4.it

  Phone: 070 9250 238  [+39 070 9250 238]
  Fax:   070 9250 216 or 220  [+39 070 9250 216 or +39 070 9250 220]
  Operator at reception: 070 9250 1  [+39 070 9250 1]
  Mobile phone: 347 7990472  [+39 347 7990472]