[Beowulf] massive parallel processing application required

[Gerry Creager]

Jim Lux wrote:
> At 02:03 PM 1/31/2007, Robert G. Brown wrote:
>> On Wed, 31 Jan 2007, Mitchell Wisidagamage wrote:
>>
>>> Thank you very much for the fire dynamics idea. I will have a look at 
>>> it.
>>>
>>> I did try to contact many e-science projects including some 
>>> researchers at Oxford. But I got no reply. Then I went to get some 
>>> contacts from a tutor who worked at a e-science project himself. He 
>>> told me people, especially scientists are "very jealous" of their 
>>> data. And not replying is a kind way of saying "no". And there's the 
>>> problem of "who's this guy wanting my data", "what will he do with it?".
>>>
>>> I have given up the e-science idea. Now looking for other real world 
>>> applications.
>>
>> Remember, NASA puts all (or at least a lot) of its e.g. weather data
>> online.
> 
> Well.. not exactly NASA.. operational "weather" data is the province of 
> NOAA.  NASA does research, not operational, data, so there's typically a 
> time lag, especially for processed and calibrated data.
> 
> By and large, most environmental data collected by NASA winds up in 
> DAACs (Distributed Active Archiving Centers). Physical Oceanography 
> data, for instance, winds up at PO-DAAC... 
> http://www-podaac.jpl.nasa.gov/ which has data for sea surface 
> temperature, sea surface topography, and ocean vector winds acquired by 
> NASA instruments.  This whole process is very well documented, and the 
> data moves through the various levels of processing and into the 
> archives in a regular and stately fashion.
> 
> But, for instance, the live data from a single instrument (e.g. QuikSCAT 
> for ocean winds, on which I worked) also gets fed to a realtime process 
> at NOAA within about an hour after it's received on the ground every 100 
> minutes, and thence to folks like NCAR who run numerical models, which 
> then winds up at the NWS and makes the weather predictions more accurate 
> on the evening news.  This is a bit harder to find in a reliable online 
> source, especially if you want things gridded into standard geographic 
> grids, etc.   It's all out there, but since the funding stream for 
> distribution is more tenuous (NOAA doesn't have as much money as NASA 
> for this sort of thing, but they do have "real time" requirements), the 
> data tends to be a bit more "raw" or idiosyncratic, and not necessarily 
> in HDF files, etc.  It tends to be in whatever format is convenient for 
> them, which may or may not be convenient for you.

For research purposes, the National Centers for Environmental Prediction 
(ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/) makes available all 
their model runs on a 6-hourly schedule.  These data are available for 
~3 days, then expire off the servers here.  Historical data subsets are 
available via the National Climatic Data Center NOMADS portal 
(http://nomads.ncdc.noaa.gov/) which was designed to facilitate access 
to the datasets.  The National Centers for Atmospheric Research 
(http://www.ncar.ucar.edu/) allows access to some limited historic data 
in their archives without restriction and facilitates scientific 
research with accounts to scientists.

>>  And there are many things one can do with it.  Look for the
>> NOAA sites.  You can get sunspot data, proxy temperature data, and much
>> more, and build your very own climate model.  If you do, don't be
>> surprised if it fails to agree with the current one (due to be
>> re-released today, IIRC, from the IPCC).
> 
> James Lux, P.E.
> Spacecraft Radio Frequency Subsystems Group
> Flight Communications Systems Section
> Jet Propulsion Laboratory, Mail Stop 161-213
> 4800 Oak Grove Drive
> Pasadena CA 91109
> tel: (818)354-2075
> fax: (818)393-6875
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843