[Beowulf] dollars-per-teraflop : any lists like the Top500?

[Lux, Jim (337C)]

Comments interspersed below... Joe's comments are generally right on, and I can provide some insight into how governments buy stuff (it's pretty strictly regulated.. far more so than in private industry, but some of the processes seem arcane and bewildering at first glance)
Jim


> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Joe Landman
> 
> Prentice Bisbal wrote:
> 
> > These large supercomputers at national labs or large universities are
> > often provided by the vendor for little or no profit (maybe even a loss)
> 
> s/maybe/usually/
> 
> > in exchange for prestige/advertising opportunities or R&D opportunities.
> 
> Hah.  Allow me to restate this.
> 
> Hah.
> 
> It is extraordinarily rare that an entity will give you permission to
> use their name in any advertising.  The most you can hope for is,
> generally, a press release.


I agree. Here at JPL we have pretty stringent rules under which we can do things in terms of using the name of JPL or NASA.  Granted, nothing stops you advertising your product "as sold to NASA", but under no circumstances could we provide any sort of endorsement or recommendation.  We can co-author a paper or report with the vendor which reports information from some activity. We can say what we did, and make generalized fact based recommendations.  


> 
> Prestige does not translate into (profitable) revenue.
> 
> > They make up this loss by selling to money-making corporations for a
> > much bigger margin.
> 
> Hmmm ....
> 
> I think there may be a fundamental disconnect between the assumptions of
> folks in academia and the reality of this particular market.  I am not
> bashing on Prentis.  I would like to point out that the "much larger
> margins" in a cutthroat business such as clusters are ... er ... not
> much larger.


I agree here, too. My wife works in commercial IT, and I'd say that they tend to beat the vendors down more than we do here in government funded work.  She (and her bosses) have to report against monthly, quarterly, and annual targets for everything.  Government work tends to be funded on an annual cycle (October 1st is the FY start date.. funding gets set based on proposals in March/April, and cast into concrete around now for the next FY).  

Where government work is concerned, the problems usually aren't profit margin (we're happy to give a reasonable rate of return, because screwing the clamps down onto the vendor's last penny usually doesn't work out well.. they'll put their top people on the jobs that have decent returns, especially if your job gets into difficulties).  It's the myriad other weird and not so weird  conditions (Drug Free Workplace Act, Buy American Act, Foreign Corrupt Practices Act, etc.) resulting from the dual role of government procurement: get something useful done and create social policy. That, and the absolute paranoia that the taxpayer might be getting the short end of the stick results in a substantially larger paperwork burden to prove they aren't.  All those folks standing up decrying "waste, fraud, and abuse"... In a commercial entity, one can do a cost benefit trade on, say, inventory losses vs time/effort to keep track of things.  Not in government, when someone will be sure to stand up and say "Agency X lost or misplaced 3 laptops out of the 123,000 they have, and this must stop now!"

Government work also often pays slow, but reliably.  But your bank may not understand, so getting operating capital can be a challenge. 

> What we are seeing is a fundamental change of business model, over to
> one that keeps upfront capital costs as low as possible, and pushes
> things to expense columns.

Yes.. it makes *this week/month/quarter's* numbers look better, and also gets you out of having to seek capital (which has been hard recently in these odd-times for credit).


> 
> > For example, I would not be surprised if IBM practically gave away
> > RoadRunner to Los Alamos in exchange for the computing expertise at Los
> > Alamos to help develop such an architecture and then be able to say that
> > IBM builds the world's fastest computers (and that your company can have
> > one just like it, for a price). Oh, and the users at Los Alamos probably
> > provide lots of feedback to IBM which helps them build better systems in
> > the future. (Don't shoot me if I'm wrong. I'm just theorizing here)
> 
> Don't sell the folks at TJ Watson/IBM Research short.  They are a very
> bright group.  The co-R&D elements are a way IBM can dominate the HPC
> bits at the high end, and provide something that looks like an in-kind
> contribution type of model so that LANL and others can go to their
> granting agencies and get either more money, or fulfill specific
> contract points.

This sort of thing is often done under a "Cooperative Research and Development Agreement" or CRADA.  This is basically a contract between the govt and the vendor which lays out who is doing what, what they're bringing to the table, and where the intellectual property rights will wind up.  For example, I'm working on a space mission now where several vendors have provided equipment under CRADAs. I don't know anything about what's in the agreement, but in general, it's something along the lines of "we give you a ride into space and you get to fly your box and test your new technology"  International MoUs for science instruments also work this way.  For NASA this is all covered under the "Space Act"

> 
> IBM is a business, and in most cases, won't generally have a particular
> business unit make a loss for "prestige" points.  That doesn't make the
> board/shareholders happy.

You got that right.. You'd have to put a number on the prestige and trade it against someone's advertising budget.


> 
> > I used to work at the Princeton Plasma Physics Lab (www.pppl.gov), a
> > Dept of Energy National Lab, and I can tell you many of the the systems
> > sold to PPPL when I worked there was sold under a NDA, preventing anyone
> > from discussing the price. Yes, the budgets are public information, but
> > that tells you the computer hardware budget for a year, not how much was
> > spent on each computer.
> 
> Well part of that is to prevent shopping the quote.  We see this *all*
> the time.  Someone asks for a quote, you provide it.  They then go and
> take your quote, elide specific company information, and then send it
> around asking others to beat it.

Exactly.. we do this with "source selection" all the time.  The proposals are proprietary and confidential, the reviewers on the Source Evaluation Board all sign NDAs and go work in a special room where all the materials are kept. It's a very, very big deal (even for relatively small procurements).  Once the selection is made, then we shred all the materials, and go to negotiate the actual contract with the vendor (which can't change substantially from what they proposed, because otherwise the losing vendors can legitimately complain).  

During the negotiation process (especially if it is a cost plus fixed fee) the vendor will need to provide a lot of financial details to allow our people to determine if the price is "fair" and that the vendor is giving the government the "lowest price" (if you want to stay out of trouble, do not sell us something for $1000 and then sell it to someone else for $900).  That financial detail is generally proprietary (e.g. as a vendor you don't want us telling everyone how much your people are paid and how much you pay in fringe benefits) and wouldn't be disclosed, but the total contract value, and a fair amount of other information, is disclosed.  



Jim