[Beowulf] Docker in HPC

[Tim Cutts]

Yes, Pete, Guy and I have been debating this stuff for some time, together with some of our informatics coders.  Should virtualisation ever also be necessary (for example to ship a software stack to another site to analyse some of their data), it's also a very small step further from Docker to that, so using this sort of approach opens up a lot of possibilities in the future.

Regards,

Tim
-- 
Dr Tim Cutts
Acting Head of Scientific Computing
Wellcome Trust Sanger Institute



On 27 Nov 2013, at 12:18, John Hearns <hearnsj at googlemail.com> wrote:

> 
> 
> On 27 November 2013 12:01, Peter Clapham <pc7 at sanger.ac.uk> wrote:
> 
> In the bio-informatics arena the local software half life is
> approximately 6-12 months. This, along with the wide range of
> applications in use rapidly creates an environment where users can cross
> link or pick up binaries or libraries that they weren't expecting.
> Rolling containers with predefined environments would not only
> potentially alleviate these potential pitfalls BUT they could provide an
> environment in which data can be re-analysed at a future date in against
> the same pre-defined environment.
> Peter, that's very very interesting!
>  
> I must say that I first started thinking along these lines when working in High Energy Physics.
> Predating Docker by a loooooongggg way, I though VMs.
> At the time, the CERN LHC analysis depended on running on Redhat Linux, on a version which
> was going out of date, or had already gone out of date (can;t remember the version).
> Hence the creation of Fermi/CERN Linux.
> I thought that instead of physically installing what was soon to be an out-of-date distro on 1000s' of machines,
> wouldn't it be better to have a VM which you were sure would haev all the libraries and dependencies for your
> environment, and then ship it off to remote centres.
> Of course this is being done today.
>  
>  
> Regarding ISV codes, today you frequently see a vendor supplying an entire tree of software,
> complete with an MPI  plus a version of Python with libraries, plus X, Y,Z.
> They do this because they have to ship software which works to users, and don't want the inevitable
> 'Oh - my sysadmin installed Python 2.7 and your software does not work with that' type of query.
> I don't see that it is that far a step to packaging up Docker containers.
>  
> (Sorry - don't mean to single out Python here - I've just been workign with it for the past few days).
>  
>  
>  
>  
>  
>  
>  
>  
>  
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf




-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20131127/01c12528/attachment.html>