[Beowulf] HPC Workflows

[Tim Cutts]

Ho ho.  Yes, there is rarely anything completely new.  Old ideas get dusted off, polished up, and packaged slightly differently.  At the end of the day, a Dockerfile is just a script to build your environment, but it has the advantage now of doing it in a reasonably standard way, rather than whatever random method any of us might have come up with independently in the past.  Adding central repositories and allowing you to base one Dockerfile on top of another are nice additions.  Neither of those ideas is new either, of course.

I agree with your last sentence, we in scientific IT definitely need to stick very close to the scientists.  But I don’t necessarily agree with the whole of the last paragraph.  IT may appear to be there to justify more IT, but I think that’s a vicious circle.  Many organisations see IT (whether scientific or enterprise) as a cost centre rather than as a strategic tool to meet their goals.  They become forced into justifying their own existence, and of course the political game then becomes they have to seek to expand in order to stay the same size, otherwise they will be cut.  Many years ago I listened to a talk by Joe Baguley from VMware, who said he could determine which was the case or not when talking to a customer with a single question,  which was:  “Who does your CIO report to?”  CEO:  good, CFO: bad

I suspect there’s some truth in that.

Regards,

Tim

> On 1 Dec 2018, at 14:29, John Hanks <griznog at gmail.com> wrote:
> 
> For me personally I just assume it's my lack of vision that is the problem. I was submitting VMs as jobs using SGE well over 10 years ago. Job scripts that build the software stack if it's not found? 15 or more. Never occurred to me to call it "cloud" or "containerized", it was just a few stupid scripts to solve some specific problem we had. I look at containers and cloud now and just don't get it. Early in my career I had a mentor who was from the IBM mainframe world. I recall excitedly explaining what I was playing around with with early Xen versions and he said "Yeah, we've been doing that for a long time." Now it's my turn to say "Yeah, I was doing that years ago" and scratch my head at what all the fuss is about. Such are the effects of the ravages of time.
> 
> Keeping distance from IT is always a good idea. The first rule of interacting with IT is: "IT is not here to solve your problem, IT is here to justify more IT. If your problem is solved, then it is the result of random chance, do not look for patterns." Best to sit as close to the scientists as possible.




-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.