no 'commodity' OS is 'secure' Re: [Beowulf] Which distro for the cluster?

[Jim Lux]

At 08:04 AM 1/10/2007, Robert G. Brown wrote:
>On Wed, 10 Jan 2007, Andrew Piskorski wrote:
>
>>On Sun, Jan 07, 2007 at 03:49:50PM -0500, Robert G. Brown wrote:
>>
>>>I completely agree with this.  As I pointed out earlier in the thread,
>>>companies such as banks make "conservative" seem downright radical when
>>>it comes to OS upgrades.  They have to do a complete, thorough,
>>>comprehensive security audit to change ANYTHING on their machines -- as
>>>a requirement in federal law, IIRC.  To get them to take you seriously,
>>>you MUST be prepared to support the OS they install on (once it is
>>>successfully audited) forever -- until the hardware itself falls apart
>>>into itty-bitty bits.



<snip>

There is a world of difference between a Windows server set up in a bank
>environment, where they are running only a fully patched variant of
>Windows that has been really throroughly audited for holes, in a
>completely minimal installation (no gorp as all gorp must be audited and
>increases risk) with only certain very specific ports open and those
>watchdogged and externally firewalled, running software that only MS has
>written and debugged top to bottom, being administered by REAL MCSE's --
>not the ones that pick up their degrees from an online training program,
>but people with masters level CPS degrees AND MCSEs AND credentials from
>multiple additional training courses AND ten years of experience in the
>trenches.
<snip>


>Basically they have to find a hole in the daemon that manages the one
>open port (whose source has been micro-audited for e.g. leaks and buffer
>problems outside of the usual development stream and which may not even
>be the same source as what is in the open distribution version) AND
>figure out a way to slip inside without getting eaten by any of the
>automatic or human cereberus's that guard the door.  The idea that this
>occurs and folks succeed makes for a great film idea, of course, but
>I'll bet that nearly every successful attempt at a core system protected
>in depth like this is made EITHER with penetrations through HARDWARE or
>FIRMWARE holes -- tapping that good old powerline or the like to snoop
>keys -- or by insiders or with their knowing or unknowing collusion
>(snitching their magstripe card, bugging their bedroom where they talk
>in their sleep from all of the jolt cola they drink on the job:-).
>
>>Now, I assume that using any such non-mainstream system is probably
>>(so far, to date) significantly more painful, annoying, and thus
>>expensive than just running Linux.  (And thus is unlikely to be
>>appropriate for a Beowulf cluster.)
>>
>>But if you're a huge organization already throwing millions of dollars
>>into horribly painful manual re-audits of even trivial updates to
>>"commodity" operating systems for mission-critical "highly secure"
>>applications, then I strongly suspect that you're already well into
>>the same cost range where investing those $millions into the use of
>>secure-by-design systems might well make much more sense.
>
>Ah, a believer in rational decisioning, CBA, minimal TCO.  Don't you
>see, man, that you're up against a whole world of people that don't,
>actually, understand the rational process?  A world where 1/2 of its
>members have IQ's under 100, and where 100 \pm 10 is usually a bit iffy
>when it comes to being able to actually analyze things logically or
>mathematically?

<snip>

Banks, IT, and security..  My wife is a senior IT manager in a big 
bank, so I get to hear quite a bit about what's involved in this.

They take it quite seriously (backed up by federal and state 
regulations and laws)

First off... tons of money are spent on it.  As rgb pointed out, 
they're not out hiring kids out of highschool as sysadmins.  These 
folks get paid reasonably well and are quite skilled and competent.

Second.. there are many levels of checking and cross checking.  Not 
only is there a whole second independent group of people through whom 
all software changes must flow, but there's a third independent group 
of auditors making life a miserable hell for the aforementioned first 
two groups. And, within these groups, there are multiple levels of 
approval required to even contemplate making the change in the first 
place.  You'd have to suborn and coopt a lot of people to "sneak 
something in", and those people are paid quite well so it ain't going 
to be the "slip someone a few hundred bucks under the table to leave 
the door unlocked" sort of thing.

Third.. systems are designed to require multiple people to be 
involved in any significant transaction or event.  And, there are 
rules that require those people to take vacations and be 
"disconnected", so that there are always new/fresh eyes looking at 
the day to day operations.  This is basic accounting 101... be 
suspicious of clerical employees who never take a vacation, and have 
a different person write the checks vs checking the statement from 
the bank. (I learned that one the hard way)

Fourth.. there are big time criminal penalties involved.  That's a 
much bigger club than some civil action or a "theft of services" sort 
of prosecution.  The police WILL get involved, the FBI and Secret 
Service WILL get involved.

Fifth.. Everybody working in a position of trust has to have an 
Office of the Comptroller of Currency background check and 
pass.  Lots of bright people don't pass the check because of some 
crippling problem or stupid indiscretion in their deep dark 
past.  The guidelines are out on the OCC website somewhere, and most 
companies have their own list of infractions.  It's done by a sort of 
point count scheme.  I would imagine (but do not know) that having 
been involved in ANY sort of fraud or scam (whether computer related 
or not) is sufficient to immediately disqualify you.  The stories you 
hear about high-school or college hackers seeing the light and being 
hired to help secure things are just that.. stories.  They might hire 
a "black-hat" consultant to give advice or do a penetration attempt, 
but they're going to be well firewalled (as in physically separate 
locations, no connectivity, etc.) from actual operations.  They'd 
never get a job as a coder, thence to lie in wait as they get 
promoted over 15 years to a position where they could actually be 
able to do some damage.

Sixth.. these are financial transactions, and they can always be 
reversed.  This is sort of the ultimate "checkpoint/restore" 
mechanism.  There have been compromises and mistakes (hey, if you're 
processing millions of transactions a day, run of the mill software 
errors crop up) and the people affected always get "made 
whole".  Sometimes it might take some time, but it gets fixed 
eventually. (to the point where there are opportunists who wait for 
the inevitable mistakes and cash in on the penalties... Taking 
recording of mortage pay-offs as an example, if you record the 
document late or improperly (where the time line is defined by 
statute), the borrower gets some sort of compensation (as well as 
getting the transaction fixed). )


So, the actual cost and security status of the OS involved is 
insignificant in comparison to the enormous people and infrastructure 
costs already being spent.  Furthermore, the pecularities or not of 
the OS don't really have an effect.  You've got a huge staff of 
people who are very experienced in those peculiarities, whatever they 
are.  The whole system architecture (including the people 
architecture) is specifically designed to make security sort of 
automatic.  It's tedious, it's expensive, and it works fairly well.







James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875