[Beowulf] A hello, and an introduction

[Robert G. Brown]

On Mon, 21 Feb 2005, Brian R Smith wrote:

> Hey Jeremy,
> 
> Its good to see another student admin at a university on here.  Welcome
> to the list.  There are a lot of top-notch people on here that you can
> learn a lot from.  I've been admining at my univ. for about 3 years now
> and plan on doing so even after I graduate.  
> 
> With a C.S. background, you'll probably find lots of interesting things
> involved with Cryptography or Image/Video processing.  I'm working on a
> video compression format right now and will likely write up a parallel
> encoder for AVI's into my format.
> 
> Maybe my boss will post and give you some ideas on what to research as
> he's working on his PhD and probably has a better idea than I do on what
> you can accomplish on a cluster as a C.S. major.  And I'm sure RGB could
> come up with some mind-blowers if you are really up to the task.

I don't know about mind blowers, but my column for I think April's CWM
is on "things you can do with your starter cluster".  It's far from
exhaustive, but it provides a bit of direction for this perennial
question.

As far as RESEARCH topics are concerned, you should probably contact me
off the list if you really do want any suggestions.  There is a bit of
difference between "generally interesting stuff you can do with a
beowulf" and "computer science research you can do" with a beowulf.  The
former is concerned with applications and simple demonstrations -- the
latter with tools, algorithms, timings, latency and so forth.

I do have one idea for god's own project that I've offered up to the
list a few times before (one inspired by work of Jack Dongarra and
others, in case you were wondering which god:-).  In a nutshell, it is
to build a microbenchmarking daemon that could be included in a standard
linux distribution and run as an initd-controlled tasks during startup.
During normal operation, it would borrow idle cycles and accumulate
benchmark/performance statistics and make them available via a socket
interface (probably UDP)

Any application, local or remote, could then query any host/node and get
a performance profile -- a matrix of key microbenchmark numbers.  These
in turn could be used to "autotune" both serial and parallel/distributed
applications.

Once the daemon existed and a fairly standard set of numbers developed
for a first cut at its output, one could then start thinking about
e.g. rewriting ATLAS so that it autotunes from the daemon results
instead of during build, so that a parallel application that partitions
does so automatically to take advantage of superlinear speedups that
might occur for certain partitionings, and so forth.

I'd think that there were all sorts of papers in there, no?  And a damn
nice GPL toolset in the end that could be tremendously useful to lots of
people.

And as a final benefit for those seeking fame, it would obviously become
THE microbenchmark tool for linux and likely other distros, as it would
be the one that is built right in.  In fact, the very first application
that uses it would be a simple command line or GUI interface to read and
plot its cumulated results...

I'd be happy to direct this and maybe even contribute, if any CPS
student-geeks out there find this interesting...

   rgb

> 
> Good luck and welcome to the list.
> 
> Brian Smith
> 
> On Fri, 2005-02-18 at 10:44 -0600, streich at uwm.edu wrote:
> > Hello all,
> > 
> > I'm new to the list and just thought I'd introduce myself, as will probably be
> > posting to the list a bit.  I'm a system administrator for a Beowulf cluster at
> > UW-Milwaukee.  It's a 22 node 2.4GHz Intel cluster running Linux that is
> > dedicated to studying clouds (using wrf and COAMPS (MPI based software)).  It's
> > a student job, and a lot of fun.  I'm a Computer Science major, and have all
> > the Computer Science course done (just have a few math classes left).
> > 
> > I'm starting to think about Grad school and Master Thesis stuff, though that is
> > a little way off.  Along this vein, if anyone has any suggestions as to hot
> > research topics a CS major with access to a few spare clock cycles Beowulf
> > cluster might be interested in, please feel free to send them to me. ;)
> > 
> > I've only been admin-ing the cluster for about a year, so I don't know how much
> > I'll be able to help people with questions...  But I may throw an idea out once
> > in a while.  I suppose here I may be asking more than answering the questions,
> > as it seems a lot of you have quite a bit of experience with larger clusters.
> > 
> > - Jeremy
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org
> > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu