[Beowulf] Which distro for the cluster?

[Leif Nixon]

Joe Landman <landman at scalableinformatics.com> writes:

> Andrew M.A. Cater wrote:
>> On Wed, Jan 03, 2007 at 09:51:44AM -0500, Robert G. Brown wrote:
>>> On Wed, 3 Jan 2007, Leif Nixon wrote:
>>>
>>>> "Robert G. Brown" <rgb at phy.duke.edu> writes:
>>>>
>>>   b) If an attacker has compromised a user account on one of these
>>> workstations, IMO the security battle is already largely lost.  They
>
> s/largely/completely/g
>
> At least for this user, if they have single factor passwordless login
> set up between workstation and cluster.

Of course. But you want to contain the intrusion to that single user,
as far as possible. If your security hinges on no user passwords ever
being stolen, you can very easily wind up in a situation that
traditionally is said to involve creeks, but not paddles. I have two
thick binders sitting on my desk, containing stolen passwords from an
impressive range of commercial, academic and military institutions. 

>>> In general, though, it is very good advice to stay with an updated OS.
>
> ... on threat-facing systems, yes, I agree.
>
> For what I call production cycle shops, those places which have to churn
> out processing 24x7x365, you want as little "upgrading" as possible, and
> it has to be tested/functional with everything.  Ask your favorite CIO
> if they would consider upgrading their most critical systems nightly.

I see this in hospitals a lot. Some healthcare systems can't be
patched without reapplying for FDA approval, which is of course a
hideously complicated process. So hospitals wind up running software
which you can push over with a feather. Theoretically, they should be
running on an isolated network ("It's no problem, we have
firewalls!!!"), but it only takes a single mistake: somebody plugs in
an infected laptop, or somebody misconfigures a VLAN. Our local
hospital has fallen over due to worm infestations a couple of times.

> It all boils down to a CBA (as everything does).  Upgrading carries
> risk, no matter who does it, and how carefully things are packaged.  The
> CBA equation should look something like this:
>
> 	value_of_upgrade = positive_benefits_of_upgrade -
> 			   potential_risks_of_upgrade

With the security benefits being really hard to quantify. 

> You have a perfectly valid reason to upgrade threat facing nodes.  Keep
> them as minimal and as up-to-date as possible.  The non-threat facing
> nodes, this makes far less sense.  If you are doing single factor
> authentication, and have enabled passwordless access within the cluster:
>  ssh keys or certificates or ssh-agent based, once a machine that holds
> these has been compromised, the game is over.

I don't get this. What's the point of having a "secure" frontend if
the systems behind it are insecure? OK, there's one big point -
hopefully you can buy some time - but other than that? 

The goal is to be able to contain user level intrusions. If you can do
this, the game *isn't* over even if you have an intrusion spreading to
a cluster machine. A user level intrusion isn't too hard to deal with,
but a cluster-wide root intrusion... isn't much fun. Sure, you can
probably reinstall the entire cluster in an hour. To a vulnerable
state. Hooray.

-- 
Leif Nixon                       -            Systems expert
------------------------------------------------------------
National Supercomputer Centre    -      Linkoping University
------------------------------------------------------------