[Beowulf] large MPI adopters

[Andrea Di Blas]

hi dan, vincent, gerry, bodgan, and all of you,


first of all, thank you very much for your answers - I really appreciate.

what I understand from your messages is that there isn't what I was 
looking for - or maybe that my question wasn't clear enough.

what I was wondering about is if there are any MPI-based applications that 
are well known, either because they are being sold as such, or because 
they are known to be used by (large) companies. for instance, we all know 
that google uses mapreduce on top of gfs. we know that yahoo uses a 
version of mapreduce called hadoop. anything similar for MPI?

or - are there any commercial products based on MPI that one could buy and 
run in their own company/institution? for example, let's say I want do do 
some simulations of whatever kind, and I like matlab. can I go buy a 
cluster of some kind, and also buy some matlab implementation that will 
use MPI to run on my cluster? or some other MPI-based tool? is all 
software like that still for internal use only of the entities that 
develop it?

thanks again,


 	andrea


-- 
Andrea  Di Blas                     UCSC
                    School of Engineering
Tel: (831) 459 4193  Fax: (831) 459 4829

On Wed, 8 Oct 2008, Vincent Diepeveen wrote:

>
> On Oct 8, 2008, at 4:09 PM, Bogdan Costescu wrote:
>
>> On Tue, 7 Oct 2008, Vincent Diepeveen wrote:
>> 
>>> Drug type companies eat typical 0.5% system time.
>> 
>> Could you please explain what you mean here ? That they buy lots of CPUs 
>> but only use a fraction of them or only for a fraction of time ?
>> 
>
> When you add up others.
>
> Note i see in IWR you got a 160 node opteron cluster, what software eats 
> system time on that cluster?
> There must be graphs about it, want to share one? Also interesting to know is 
> how many cores most jobs eat!
>
> Note that i left out 1 important factor which is not a factor in every 
> nation: that's the amount of system time airindustry eats of supercomputers
> designing new stuff. It is very significant together with car manufacturers. 
> With 160 nodes machine, even if that would be power6 nodes,
> you can't even design a wing for aircraft.
>
>>> They are far behind, which is not so logical; i would consider it old 
>>> fashioned to do experiments on monkeys (Bar-ilan brain experiments on 
>>> monkeys rings a bell?), so doing everything in software is far more 
>>> logical.
>> 
>> Maybe your knowledge about the pharma industry is far behind ? "Doing 
>> everything in software is far more logical" only when you can exactly 
>> express biological processes in mathematical terms which is not true in the 
>> great majority of cases. F.e. molecular dynamics (often used to simulate 
>> interactions at molecular level, f.e. of a drug injected into the body) is 
>> based on several approximations and the implementations are subject to 
>> numerical errors. Would you take a drug that was produced based only on a 
>> simulation with molecular dynamics and found to be good enough ?
>> 
>
> Well, i sure avoid hospital visits, you?
>
> As you know the healthcare commission is basically old men who spread careful 
> chosen sentences in such a manner that
> everything looks fine on this planet and that there is nothing to worry 
> about; last time i spoke with them on EMF's a guy being spokesman
> on this subject, he didn't even know how the electricity system in his own 
> house worked (whether on the first floor of his house
> he had earth).
>
> So lucky to introduce to a new drug, there is some burocratic systems. There 
> is several phases of testing that a drug
> has to pass in order to get accepted as a drug that is legal. Not to confuse 
> with the 'advice' on whether a drug gets prescribed;
> they want way cheaper drugs for that usually than the ones that might maybe 
> be better (say 0.5% better for a lot of money extra,
> but if you're that guy that didn't get cured as a result of that, your family 
> will regret it bigtime).
>
> Computing power or not, most manufacturers simply want to sell a product, so 
> the amount of systemtime invested on a computer,
> in order to sell, is total irrelevant to them to get a drug accepted usually. 
> The first and most important part of the show to get a drug
> accepted is getting invited to the table to show your drug. Computing power 
> is nice to mention here when that is usually not a factor
> at all to get a new medicine accepted.
>
> But it may change, let's hope so...
>
> Maybe a new breed of company can do the job. A breed where making profit is 
> not most important :)
>
>> Similar concerns apply to all levels - the brain, as you mention it, is 
>> still very much an enigma, otherwise artificial intelligence would be all 
>> around us (or not, if deemed to dangerous...). And when you don't know how 
>> an organ or system works, how can you develops drugs only by simulations ?
>> 
>>> When someone is really in big need for big calculation power, one makes 
>>> dedicated cpu's for it.
>> 
>> I tend to agree, provided that the dedicated CPU brings a really big 
>> advantage, like an order of more of magnitude. F.e., to stay in the pharma 
>> related field, that's the bussiness case for DEShaw Research's molecular 
>> dynamics chip (Anton) which is supposed to bring a significant increase in 
>> speed for MD simulations compared with software solutions running on 
>> general purpose CPUs (or GPUs).
>
> Some hardware guys can better comment here. Making your own FPGA chippie is 
> rather cheap.
> Finding and paying a few good guys to get the chip done is toughest. You 
> basically need 1 good
> programmer, the rest can get hired parttime in fact. After chip works,
> to print a 1000 cards, it is like a 100 dollar for pci-card and 50 dollar a 
> chip.
>
> So $150k for computing power that total annihilates any supercomputer of now 
> and the NEAR future for your specific application.
>
> Real cpu's are a lot faster of course, price for them i do not have at hand. 
> I heard it could be cheaper
> than this in fact if you regurarly print a batch. Such a batch is producing 
> less cpu's than the FPGA, where you pay for 1 cpu a constant
> amount. I tend to remember that good cpu designers, not to mention memory 
> controller guys, are rather expensive folks, as
> they are very popular; on some other mailing list with some very funny 
> lastnames, i see nonstop job offers for that type of folks.
>
> Vincent
>
>
>> -- 
>> Bogdan Costescu
>> 
>> IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
>> Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850
>> E-mail: bogdan.costescu at iwr.uni-heidelberg.de
>