[Beowulf] RE: MS Cray

[Lux, James P]

> -----Original Message-----
> From: David Mathog [mailto:mathog at caltech.edu]
> Sent: Wednesday, September 17, 2008 10:40 AM
> To: beowulf at beowulf.org
> Cc: Lux, James P
> Subject: Re: MS Cray
>
> "Lux, James P" <james.p.lux at jpl.nasa.gov> wrote:
>
> >   The other, here at JPL,
>
> I have heard about this contract before - and in my opinion,
> it is a horrible deal.  The taxpayers get reamed and the
> vendor makes out like a bandit.
>

actually not.. It's quite competitive with the sort of deal other folks get in industry.


> (SNIP)
>
> >At any given time, there's a dozen or so kinds of computers
> >(desktop/laptop/PC/Mac) available, but since the configurations are
> >changing, and they have a 3 year recycle time,
>
> the key point being the 3 year lease.
>
> (SNIP)

Well.. That's always a trade.. Buying comes out of the capital bucket, leasing comes out of the expense bucket, and they have very different treatments, accounting wise.  Don't forget that JPL works on a "cost reimbursement" basis: that is, we spend money, and the gov't, via NASA, reimburses those costs, and ONLY those costs.  There are literally bookshelves full of rules (the Federal Acquisition Regulations) that tell you what is allocable, accountable, and reimburseable. JPL cannot borrow money to pay for things: the US government will not pay for interest on borrowed money. Most capital investments have to be approved by an act of congress, and the amortization of that investment requires special treatment to make sure that costs are properly allocated to each project.  Leasing makes it easy..Accountants cost money too, if you do march down the amortization process.  Many, many commercial companies have similar sorts of issues, particularly with respect to transactions between divisions (e.g. see Regulation W for banking), all designed to prevent "hiding profits".


>
> >Interestingly, the monthly cost for both organizations is about the
> >same ( a few hundred bucks a month for hardware lease+service).
>
> Let "a few hundred" = $200, and of course there are 36 months
> in 3 years, so JPL pays the vendor $7200 for each machine,
> plus "support" for this term.  At the end of the lease the
> vendor gets the computer back, and they probably sell it for
> a few hundred dollars, just to sweeten the already cushy
> deal.


actually, a mid range desktop machine is about $150, and they actually do monthly comparisons against other vendors, so the cost is probably pretty reasonable for a large volume consumer.  Don't forget that part of the cost is the (legally required) paperwork to prove that the taxpayer's not getting the shaft. (e.g. those comparisons don't happen for free, and someone has to collate the data, put it into a report, etc.)


The office staff may need support once and a while, but one can assume that the average JPL engineer or scientist can more than handle all their own PC software issues, and at worst would just need to swap a machine if there was a major hardware issue.  In other words, the average support cost to the vendor for the technical staff is but a tiny fraction of what JPL pays them.



The office staff may need support once and a while,
> but one can assume that the average JPL engineer or scientist
> can more than handle all their own PC software issues, and at
> worst would just need to swap a machine if there was a major
> hardware issue.  In other words, the average support cost to
> the vendor for the technical staff is but a tiny fraction of
> what JPL pays them.

Actually, before they went to a centralized support model, they found that the "shadow administrator" and "shadow support staff" costs were huge, and more to the point, not accurately measureable (what you can't measure, you can't manage).  And, while most JPL technical staff are certainly capable of doing their own support, there's good reason for them not to: they typically have a more than full time job doing something more specialized (e.g. designing deep space communication links, or writing nav software for rovers, or designing and building rovers), and every minute they spend fooling with their computer is a minute not getting the *real job* of space exploration done.  It's like working on my house: I'm capable of painting the walls myself, but I hire a professional to do it, because there are better uses for my time (and the specialist will do a better job).

Support isn't trivial here, even for office staff. For a variety of reasons (and not unique to JPL.. Any other 5000+ employee, $1B year business will have similar ones), we have an amazingly wide array of various and sundry institutional applications to do things like timecards, keeping track of inventory, document mangement, etc. I'd venture that the desktop support staff spends more than 70% of their time dealing with non-OS related software issues (e.g. why is my email not getting through), and a very tiny fraction of their time responding to hardware problems or OS issues. For instance, they're rolling out a new "unified messaging system" which will integrate calendaring, based on MS Exchange Server (for whatever reason.. That's irrelevant), and rather than saying "Use Outlook", which would certainly reduce support costs, they ARE supporting half a dozen email clients, web clients, a bunch of calendaring clients, not to mention a fairly heterogenous set of hardware platforms.

Let us not forget IT security. This is a non-trivial matter when you have to manage tens of thousands of desktops and comply with dozens of pages of government regulation, NASA procedural instructions, etc.

The upshot is, you get a fair amount for your $130/month support subscription.  To put that in context, that's less than two hours of engineer time.


>
> Who's the vendor, Halliburton?

No, it's a division of Lockheed Martin, but basically anyone in the business of doing what they do charges about the same amount of money. The business *is* fairly competitive (and gets recompeted every 3-5 years, I think), and they changed vendors a few cycles ago.

For what it's worth, we have a similar sort of scenario when dealing with test equipment. Do you own it, and have inhouse inventory, (with all the peculiar government contract stuff about cost accounting), or do you have an outside vendor provide it on lease/rent.  Same sorts of heterogenous product line. Same sorts of issues with support (e.g. test and calibration of instruments). Same sort of problems with purchasing capital equipment on a cost reimbursement job (the government discourages you charging the full cost of your infrastructure to their job).
 A big disadvantage of in-house is that it leads to an inventory of ancient gear that becomes hard to maintain, and balkanized ownership (we bought that for Project X, and though Project X is long gone, the former staff of Project X still owns it). A big advantage of in-house is that you can walk across the street and pick up an oscilloscope.

Jim