[Beowulf] massive parallel processing application required

[Jim Lux]

At 02:03 PM 1/31/2007, Robert G. Brown wrote:
>On Wed, 31 Jan 2007, Mitchell Wisidagamage wrote:
>
>>Thank you very much for the fire dynamics idea. I will have a look at it.
>>
>>I did try to contact many e-science projects including some 
>>researchers at Oxford. But I got no reply. Then I went to get some 
>>contacts from a tutor who worked at a e-science project himself. He 
>>told me people, especially scientists are "very jealous" of their 
>>data. And not replying is a kind way of saying "no". And there's 
>>the problem of "who's this guy wanting my data", "what will he do with it?".
>>
>>I have given up the e-science idea. Now looking for other real 
>>world applications.
>
>Remember, NASA puts all (or at least a lot) of its e.g. weather data
>online.

Well.. not exactly NASA.. operational "weather" data is the province 
of NOAA.  NASA does research, not operational, data, so there's 
typically a time lag, especially for processed and calibrated data.

By and large, most environmental data collected by NASA winds up in 
DAACs (Distributed Active Archiving Centers). Physical Oceanography 
data, for instance, winds up at PO-DAAC... 
http://www-podaac.jpl.nasa.gov/ which has data for sea surface 
temperature, sea surface topography, and ocean vector winds acquired 
by NASA instruments.  This whole process is very well documented, and 
the data moves through the various levels of processing and into the 
archives in a regular and stately fashion.

But, for instance, the live data from a single instrument (e.g. 
QuikSCAT for ocean winds, on which I worked) also gets fed to a 
realtime process at NOAA within about an hour after it's received on 
the ground every 100 minutes, and thence to folks like NCAR who run 
numerical models, which then winds up at the NWS and makes the 
weather predictions more accurate on the evening news.  This is a bit 
harder to find in a reliable online source, especially if you want 
things gridded into standard geographic grids, etc.   It's all out 
there, but since the funding stream for distribution is more tenuous 
(NOAA doesn't have as much money as NASA for this sort of thing, but 
they do have "real time" requirements), the data tends to be a bit 
more "raw" or idiosyncratic, and not necessarily in HDF files, 
etc.  It tends to be in whatever format is convenient for them, which 
may or may not be convenient for you.


>  And there are many things one can do with it.  Look for the
>NOAA sites.  You can get sunspot data, proxy temperature data, and much
>more, and build your very own climate model.  If you do, don't be
>surprised if it fails to agree with the current one (due to be
>re-released today, IIRC, from the IPCC).

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875