[Beowulf] cluster building advice?

Wed Sep 26 04:25:43 PDT 2012

Hi all,

some comments from me, inserted in the text.

On Wednesday 26 September 2012 09:54:01 Vincent Diepeveen wrote:
> There's nearly nothing there at that link.
> 
> Just a handful of SRPMS.
> 
> My point of openfabrics is: most people build a cluster in order to
> be have more performance
> than a single machine can give. Not seldom that's also with latest
> hardware (not in my case
> but i expect most beowulf aren't using old hardware).
> 
> To get performance you want latest compilers and pretty much latest
> stable kernel therefore
> as a minimum requirement.

Agreed. However, that is a problem (compilers) when you are buying really 
cutting edge technology. Unless you go for the commercial compilers, you have 
to accept that the open source compilers like gcc etc. are lagging a bit 
behind. The main question is here: does you code you are using for your number 
crunching really benefit from the latest feature of the CPU?

> 
> Another issue with clusters is SAFETY.
> 
> Now by paying big cash to SLES or RHEL maybe you can buy safety -
> maybe someone wants
> to comment on that and prove to me on paper the safety of it. Yet
> where safety is the most important
> concern for most organisations with respect to HPC, i never see
> anyone comment on it.

I don't want to comment on the safety issue too much. However, I would say 
most people buy RHEL simply for the support issue: place the problem you got 
into somebody else court and you can have a cup of tea. :D
All distributions have safety issues like all OS have safety issues. We have 
to accept that in my opinion. However, the main issue is how quickly will they 
be patched and how does the installation of the patch affect your uptime of the 
cluster? Also, the latest software might not be the most stable one. 

> 
> But let's start with the performance issue.
> 
> Even trying to compile GCC 4.7.0 both 32 and 64 bits in SL was a
> problem though it had been
> released for months.
> 
> There is also clang now to mention.
> 
> Now if commercial companies like SLES and RHEL choose to use that,
> that's none of my business.
> 
> It's sad that OpenFabrics isn't there optimizing performance though
> and by default still uses some GCC version
> from nearly a decade ago which wasn't as improved for 64 bits as much
> as latest versions are, to say very polite.
>
if you want the latest software and you want to have it optimised to your 
machine you have to compile it yourself. Gentoo is then what you are looking 
for. However, I don't want to spent all my day compiling the OS and peripheral 
software, I want to spent time to compile the number crunching software and 
make sure that is working ok and is performing well on the machine I am using. 
As for OpenFabrics: I installed an InfiniBand cluster here last year and much 
to my surprise, despite having the latest hardware at the time the out of the 
box OpenFabrics from Debian was working well. Ok, I am lacking here of 
experience with other IB systems as this was my first one, but I pretty much 
doubt that a newer compiler would have given me a significant improvement of my 
performance. Correct me if I am wrong here.

> If i want to slowdown factor 2 with some software i don't need to
> build a cluster you know.
> 
> I can use my laptop if performance is not an issue.

True, and you can make your fry-up on the laptop as well :-)

In your case, I would suggest Debian. Here you can select between the stable 
(not the latest, but stable), testing (more up-to-date and relatively stable) 
and unstable (the latest software, including all the latest bugs hence 
unstable) versions. So it is up to you what to install. Also, you can get 
backports so running a stable version does not necessarily exclude you from 
running the latest kernel (what I am doing for example). 

My two pennies from a wet London. :D

Jörg

> 
> On Sep 26, 2012, at 10:32 AM, Jonathan Barber wrote:
> > On 26 September 2012 09:15, Vincent Diepeveen <diep at xs4all.nl> wrote:
> >> On Sep 26, 2012, at 5:30 AM, Christopher Samuel wrote:
> > [snip]
> > 
> >>>> The intention is for the system to be used for scientific
> >>>> computation. I am trying to decide on a linux distribution to use.
> >>>> Does it matter all that much?
> >>> 
> >>> If you're going to be using commercial codes then you probably want
> >>> something that's Red Hat or a rebuild (CentOS or Scientific Linux).
> >> 
> >> Used Scientific Linux for a while. It has no downloadable source
> >> code.
> > 
> > Say what now? You can download the source RPMs for the 6x build
> > from here:
> > http://ftp.scientificlinux.org/linux/scientific/6x/SRPMS/
> > 
> > Cheers
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
*************************************************************
Jörg Saßmannshausen
University College London
Department of Chemistry
Gordon Street
London
WC1H 0AJ 

email: j.sassmannshausen at ucl.ac.uk
web: http://sassy.formativ.net

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html