[Beowulf] SC13 wrapup, please post your own
Ellis H. Wilson III
ellis at cse.psu.edu
Sat Nov 23 15:18:30 PST 2013
On 11/23/2013 01:40 PM, Joe Landman wrote:
> I didn't get a chance to see many booths ... I did get free the last
> hour of Thursday to wander, and made sure I got to see a few people and
Had a bit of the same experience, but in the reverse. Was cooped up in
tech talks the entire time and/or preparing for my own, so only got up
to the exhibit floor for a few hours on Wednesday.
> 1) not so many chip companies (new processor designs, etc.) there. I am
> not sure if this is an overall trend, but in general they do not appear
> to be getting VC backing much any more.
Micron did have a really cool new processor called Automata in the
Emerging Technologies area of the floor. Quick Google gives me the
following article about it, which seems pretty representative:
> 6) Big data, which I liken to be applied HPC for a number of industries
> was on most customers minds. How to build/compute with massive data
> sets (sort of the more general case of 5). Everyone knows this word
> Hadoop. Very few folks quite understand that its one of many tools to
> handle data at scale.
> What's profoundly interesting to me is that all of the issues that the
> "big data" world face are similar issues to what the HPC and more
> specifically the beowulf community have faced. How to scale
> computation. How to administer and run large resources. How to design
> the resources with scaling in mind.
I met a lot of folks in the technical program that weren't yet concerned
about data problems simply because they were still so compute-limited.
I see certain niches in HPC as staying that way for years to come. With
the exception of checkpointing, which they (sadly) try not to think
about and defer to the system architects and admins to figure out, their
problems more or less stay in memory the entire time.
This strikes at the core of a massive problem I witnessed at SC this
time around: the technical talks and the exhibition seem to have almost
isolated communities. There are obvious exceptions -- I consider myself
one of such, and do my best to keep abreast of where industry is going
so I can factor those trends into where the most relevant research can
be done. This even exemplifies itself in the difference between the
research being shown in the talks versus the industry presence on the
exhibition floor: tons of storage and networking providers on the
exhibition floor, but only a handful of storage and networking papers,
mostly isolated to Tuesday (but a few on Wednesday, including my own).
There were a number of sessions I attended claiming to be data-oriented
but turned out to be pretty traditional compute-oriented works with a
data sub-focus (seemingly) just to leverage the necessary buzz-words.
Sad really. I was under the impression we in academia are supposed to
be out in front of industry, testing the waters and leading the way
where industry might not yet have a play to deem such worthwhile, rather
than behind, optimizing the hell out of old problems as if they were a
homework problem. Oh well. However, to be fair, I think more industry
folks should try to show up to, and talk with, the academics in the tech
talks. This will help drive them towards modern-day problems and
enlighten them relative to what is on the minds of the "real world"
today. This is particularly true for the smaller companies, who are
more agile and willing to try the wacky things us academics explore (or
should be exploring). I recognizing sparing the personnel from a
smaller company is more difficult though. I had a few very interesting
conversations with persons from larger companies after my talk, but
nowhere near what I thought I would. Maybe that just reflects on the
quality of my talk or a lack of alignment with industry for one reason
or another :/.
Just my (admittedly biased) perspective though, and not the case for all
the talks, just a greater number than I was comfortable with.
That's all I have time to share for now -- more on the Hadoop bits
later. I also got a chance to talk with Doug at the bash, and although
I missed his presentations (VERY sad about that), I deeply want to learn
and discuss more about what he's doing over there. Very cool stuff in
general. Seems his Hadoop-foo has leap-frogged mine over the last year :D.
Department of Computer Science and Engineering
The Pennsylvania State University
More information about the Beowulf