[Beowulf] how Google warps your brain

[Lux, Jim (337C)]

> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Mark Hahn
> Sent: Thursday, October 21, 2010 11:31 AM
> To: Robert G. Brown
> Cc: Beowulf Mailing List
> Subject: Re: [Beowulf] how Google warps your brain
> 
> > The real beauty of clusters (to me) has always been at least partly the
> > fact that you could build YOURSELF a cluster, just for your own
> > research, without having to have major leadership, infrastructure,
> > space, or other resources.
> 
> sure, but the question is: under what circumstances would you want to?
> doing beowulf is indeed easy, and for certain scales and cost structures,
> cheap.  if you have a fairly constant personal demand for resources
> (always have 80-100 cores busy), then doing it yourself may still make sense.
> but the impetus towards centralization is sharing: if your usage is bursty,
> having your own cluster would result in low utilization.  and if the same
> funding went towards a large, shared facility, your bursts could be higher.
> 
> of course, there's still the issue of autonomy - you control your own
> cluster.  but in a sense, that's really just reflecting (un)responsiveness
> on the part of whoever manages the shared resource...
> 
> I'm pretty convinced that, ignoring granularity or political issues, shared
> resources save a lot in leadership, infrastructure, space, etc.


OTOH, it's just those granularity and cost accounting issues that led to Beowulfs being built in the first place.

I suspect (nay, I know, but just can't cite the references) that this sort of issue is not unique to HPC, or even computing and IT.  Consider libraries, which allow better utilization of books, at the cost of someone else deciding which books to have in stock.  And consider the qualitatively different experience of "browsing in the stacks" vs "entering the call number in the book retrieval system".. the former leads to serendipity as you turn down the wrong aisle or find a mis-shelved volume; the latter is faster and lower cost as far as a "information retrieval" function.

To me, this is the essence of personal computing.  It's not that it's more cost effective to have a computer on/under my desk (almost certainly not, esp with cheap bandwidth to a server), it's that I can do things that are somewhat ill defined and require creativity, and after all, that *is* what I get paid for. And this is because they've bought a certain amount of computational resources for me, and leave it up to me to use or not, as I see fit.  Compare to a front line customer service agent in a cube farm where they typically have a locked down configuration and are tightly scripted.  That computing environment is hardly personal (whether it's a screen and keyboard or an actual computer) and is essentially an early 21st century assembly line or battery farm.