[Beowulf] SSD caching for parallel filesystems

[Vincent Diepeveen]

On Feb 12, 2013, at 1:13 AM, Lux, Jim (337C) wrote:

> Not simulations..
>
> Streaming the data out to a Digital to Analog converter..
>
> Basic box works like this:
> Attitude and guidance simulator gives a parallel bit word that  
> indicates range and doppler and target type.. that's used to set a  
> timer and frequency offset
> Radar gives a sync pulse to the target simulator that indicates  
> that the pulse will be transmitted in a short time, and a parallel  
> word telling it which kind of pulse it's going to transmit.
> Target simulator points to the correct pulse in the bulk storage  
> which holds a bunch of simulated returns based on target type and  
> pulse type  (e.g. a 4 ns pulse will have a different return than a  
> 1ms long pulse)
> Bulk store starts streaming the pulse into a FIFO.  A countdown  
> timer for range delay triggers the output of the FIFO to a DAC  
> which generates the actual pulse.
>
> So you don't have to simulate the returns in real time.. .they're  
> done off line in non-real time, and all the target box does is play  
> back the correct return at the correct time and Doppler.
>
> You could use a lot less sample returns and just adjust the  
> amplitude and phase, however, you'd like to simulate things like a  
> lateral translation across the edge of a rocky area to a flat sandy  
> area: the former has a much more ragged return that lasts longer,  
> while the latter is a more specular reflection and narrower.
>
> If the beam is significantly tilted relative to the surface normal,  
> the return is much longer in time than if it's pointing straight  
> down, because the "spot" that is illuminated has significant range  
> extent in the former case.
>
> This is why you wind up needing lots of different potential returns  
> for the same length transmitted pulse.
>
> And that's for non-coded pulses.  Throw in coding on the radar  
> pulse (chirps or PN codes) and it gets even more interesting.
>
> One might even take a sample radar out to a test site in the desert  
> (which looks an awful lot like the surface of Mars, at least to a  
> radar) and record actual returns from different types of terrain.
>
> I don't recall the costs, but in any case, a more important design  
> aspect is getting it to work at all, rather than driving the price  
> down to the minimum.

>
> I was responding to your question asking for an example of  
> something needing high bandwidth and small storage, but that  
> couldn't be adequately addressed by just buying a ton of  
> conventional RAM.
>
> These systems don't have a CPU.. it's just disk drive, FIFO, data  
> converter.

You're speaking here of a very specific NASA type problem.

It's like asking why one would design shoes for the guy who manages  
to jump 10 meters high - and then you show up with an astronaut 10+  
years
away from now who might jump on the Moon 10 meters high :)

>
> I suspect digital video recorders are another application.. very  
> similar kind of usage..


> You stream raw video in at some Gbytes/second and just dump it to  
> the drive(s). digital motion picture projectors at 4K resolution  
> are probably another example.  Although there, you are looking at  
> fairly large data sets.
>

Not at all, you just want petabytes of storage there.

Actually the solutions sold until recently used a DEC alpha 1Ghz  
21264 (i say from head), to steer the raid arrays.
Wasn't that around a 1GB/s bandwidth or so?

Price per petabyte of storage is everything there.

> The RED cameras stream to flash, SSD or conventional drives, for  
> instance.  9Megapixels/frame*30 fps is 270 megapixels/second.
> The newer EPIC cameras do 31.8Mpix/frame * 96 fps...    I think  
> they run about 400-500 Mbyte/sec to a SSD "magazine"
> The film business is used to interchanging magazines. A typical  
> 35mm cine camera has 400 ft and 1000ft magazines.  At the usual  
> 24fps, that works out to about 1 foot/second, so 400 or 1000  
> seconds of shooting time.  A comparable RED/EPIC magazine, then,  
> probably holds half a terabyte or so.
>
>
> http://www.red.com/store/products/redmag-4-pack
> Here you go.. a 4 pack of 256GB drives for $9100...

This is embedded hardware, again not some HPC type workload.

Not a good example therefore.

>
> 256GB is probably comparable to a 400 foot magazine at the highest  
> resolution.
>
>
> Jim Lux
>
>
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf- 
> bounces at beowulf.org] On Behalf Of Vincent Diepeveen
> Sent: Monday, February 11, 2013 3:30 PM
> To: beowulf at beowulf.org List
> Subject: Re: [Beowulf] SSD caching for parallel filesystems
>
> Ah you're doing the simulations a lot slower :) No big deal. You  
> get what you pay for :)
>
> You're speaking about 1 box here or so?
>
> What size SSD array size is in that box, what bandwidth does it  
> deliver to your CPU's and what price did you buy it for Jim?
> Then we can compare it with a harddrive raid array :)