[Beowulf] large MPI adopters
Vincent Diepeveen
diep at xs4all.nl
Tue Oct 7 05:58:43 PDT 2008
Hi Dan,
I'd rather guess his question is more: "i want a big list of 10000
companies that are actively using clusters".
That list is not there and if it was there it is utmost secretly
compiled by some NSA type instance.
I'm now speaking more for western Europe, not necessarily for USA:
The research that companies perform is usually applied research: the
minimum requirement to produce a new product,
or when forced at gunpoint by government for safety reasons.
For fundamental research usually most computational resources get
thrown into battle, which is for nearly 100% sponsored
by government be it directly or indirectly.
In fact there is entire industries, especially clean energy research,
that is 100% paid by government subsidies.
Investigations after new products, in 99% of cases get funded by
government; usually the name of a big company is behind it,
but it's still government dudes who sit in that company doing the
research. What matters of course is who pays it.
The exception is persons who have their own company and are some sort
of big expert at a certain area. Take me.
I want to do some research after learning algorithms, at a very high
level (so far they didn't even conclude the right
value for material for chess, let alone that they were useful at
anything other than debugging your code, which is a big
important thing by the way).
Without getting the help of someone working at an university to run
such a thing overnight at a 160-200 cores,
i would of course only be able to run it at a single pc. Now i don't
get any money for all this from government,
so the big question is whether i'm supported by government then or not.
What i do know is that all public research here at government level
is major shit. This is explainable by the fact that a field is nonstop
on the move, only those who have been busy in a field for 10+ years
have any clue what happens there and are the first to try something;
the PHD's are simply busy with inferior software meanwhile having
massive hardware that's idle. A big problem is the nonpublishing of
accomplishments; First of all i'm not publishing *how* i'm doing the
learning. When some university offers me a job, i might publish things,
i'm not getting paid for publications. Why help "the competition"?
What might get published is the guy who helps me; one tiny thing of
all this he'll publish probably. That's maybe 1/100 of the conclusions
drawn.
Only the NSA type organisations know as they all spy on the internet,
and you BET that all big
countries know exactly what i'm doing, they all tap the internet like
crazy there.
If 1 researcher is real brilliant and CAN need some big computation,
there is at least a 100 spies who do know something of that field,
and have to interpret the tapped data. What i do not know, as i don't
work for any of the N*SA type organisations, is in how far *those*
verify things using massive hardware.
Most researchers have no clue how big the spying and anti-spying is;
most might get total scared if they'd realized how much they
get protected.
It is easier to steal it than to find an Einstein in your own nation
figuring it out.
That is the principle that every nation uses; CIA is a very tiny
organisation compared to what other nations have,
note that the weird thing is that CIA is notorious for violating
agreements with other nations (not spying in friendly nations,
to give an example; and also passing on information to their
companies that they got from the information stream they
got from friendly nations spying onto their own companies).
The fact that i write this down already means i have not been
employed in that field nor am; otherwise i would not even *mention*
the word.
Or to quote someone who works in that field when i said that there
gotta be an awful lot of civil servants in Netherlands busy in that
field,
as there is 3+ million tourists a year in Netherlands:
"we have tourism in Spain also".
Yet nothing on the internet is safe, that's the basic problem here.
It's too easy to hack internet lines. 1024 bits or 2048 RSA or
something that
gets used for SSH?
1024 bits is too easy to crack for 1 organisation of each country by
simply throwing specialized hardware at it.
2048 bits RSA is a tad harder, but also not a problem.
128 bits AES?
No manner to attack it is public known (would only deliver $20k,
which is too little anyway, for the risk you take publishing a method).
Yet even if there would be some sort of randomized GA-algorithm that
needs some sort of Pollard-Rho order O( 2 ^ 0.25n ),
then a simple dedicated cpu can already crack it handsdown.
Obviously most companies are not busy spamming the net what they do
with clusters or do not do.
Keeping it secret for their competitors is just too important, yet if
you ask me i feel big companies do real little
research in areas that do not lead directly to products of them.
There might be 1 or 2 exceptions, like oil companies,
but the question is in how far researchers there can be seen as
employees of that company as they get indirectly
sponsored by government whose interest in everything that happens
with oil is real big.
If you ask me however, way too little fundamental research gets
sponsored by government.
If you see just in a tiny nation like netherlands; the number of
'researchers' that work for government is just a very small part of
the number of professors. It's like 1 researcher for each 2
professors; PHD's not counted.
Note this is also what a few studies recently showed.
Just the comparision to intelligence agencies which can put into
action each one of them quarter of a million to millions of people,
and who hire real easy people be it direct or indirect, it is obvious
that the saying: "Better well stolen than bad invented",
gets written in just too big capitals.
As we're speaking of a 1000 individuals in a nation having 16.5
million inhabitants, and only very few of those have
time to devote a big part of their time to do research; maybe 5% is?
Most are too busy with meetings and other organisational work and
posting on the internet.
Realize that same nation also has over a 1000 companies with
more than 1000 employees and half a million charity organisations.
That really lets fundamental research look like something that gets
total neglected.
Vincent
On Oct 7, 2008, at 9:34 AM, <Dan.Kidger at quadrics.com>
<Dan.Kidger at quadrics.com> wrote:
> Andrea,
>
> MPI is of course used by many applications running on commercial
> clusters.
> Two obvious examples are computational chemistry by the drug companies
> and computational fluid dynamics (CFD) for aerospace companies and
> F1 design teams.
>
> These are all long-term 'traditional' uses of MPI for scientific/
> engineering codes.
>
> Is this what you are asking? Or are you thinking of non-traditional
> uses in say computational finance or gaming sites?
>
> Daniel
>
> -------------------------------------------------------------
> Dr. Daniel Kidger, Quadrics Ltd. daniel.kidger at quadrics.com
> One Bridewell St., Mobile: +44 (0)779 209 1851
> Bristol, BS1 2AA, UK Office: +44 (0)117 915 5519
> ----------------------- www.quadrics.com --------------------
>
>
>
>
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-
> bounces at beowulf.org] On Behalf Of Andrea Di Blas
> Sent: 13 August 2008 00:37
> To: beowulf at beowulf.org
> Subject: [Beowulf] large MPI adopters
>
> hello,
>
>
> I am curious about what companies, besides the national labs of
> course,
> use any implementation of MPI to support large applications of any
> kind,
> whether only internally (like mapreduce for google, for example) or
> not.
>
> does anybody know of any cases?
> thank you and best regards,
>
>
> andrea
>
>
>
>
> --
> Andrea Di Blas, UCSC
> School of Engineering
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
More information about the Beowulf
mailing list