<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Here, we deploy some clusters on OpenStack, and some traditionally as bare metal. Our largest cluster is actually a mixture of both, so we can dynamically expand it from the OpenStack service when needed.
<div class=""><br class="">
</div>
<div class="">Our aim eventually is to use OpenStack as a common deployment layer, even for the bare metal cluster nodes, but we’re not quite there yet.</div>
<div class=""><br class="">
</div>
<div class="">The main motivation for this was to have a common hardware and deployment platform, and have flexibility for VM and batch workloads. We have needed to dynamically change workloads (for example in the current COVID-19 crisis, our human sequencing
has largely stopped and we’ve been predominantly COVID-19 sequencing, using an imported pipeline from the consortium we’re part of). Using OpenStack we could get that new pipeline running in under a week, and later moved it from the research to the production
environment, reallocating research resources back to their normal workload.</div>
<div class=""><br class="">
</div>
<div class="">There certainly are downsides; OpenStack is a considerable layer of complexity, and we have had occasional issues, although those rarely affect established running VMs (such as batch clusters). Those occasional problems are usually in the services
for dynamically creating and destroying resources, so they don’t have immediate impact on batch clusters. Plus, we tend to use fairly static provider networks to connect the Lustre systems to virtual clusters, which removes another layer of OpenStack complexity.</div>
<div class=""><br class="">
</div>
<div class="">Generally speaking it’s working pretty well, and we have uptimes of in excess of 99.5%</div>
<div class=""><br class="">
</div>
<div class="">Tim</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 1 Jul 2020, at 05:09, John Hearns <<a href="mailto:hearnsj@gmail.com" class="">hearnsj@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Jorg, I would back up what Matt Wallis says. What benefits would Openstack bring you ?
<div class="">Do you need to set up a flexible infrastructure where clusters can be created on demand for specific projects?<br class="">
<div class=""><br class="">
</div>
<div class="">Regarding Infiniband the concept is SR-IOV. This article is worth reading:</div>
<div class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_neutron_pike_admin_config-2Dsriov.html&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=gSesY1AbeTURZwExR_OGFZlp9YUzrLWyYpGmwAw4Q50&m=T0asmfOta_bLT2cXWrpERYigde5lOqHx2vVIH2WSIOw&s=VMHyCkd1eb1ztnzu4i617zrYxnddfDUUEkn1u45xQq0&e=" class="">https://docs.openstack.org/neutron/pike/admin/config-sriov.html
[docs.openstack.org]</a></div>
<div class=""><br class="">
</div>
<div class="">I would take a step back and look at your storage technology and which is the best one to be going forward with.</div>
<div class="">Also look at the proceeding sof the last STFC Computing Insights where Martyn Guest presented a lot of</div>
<div class="">benchmarking results on AMD Rome</div>
<div class="">Page 103 onwards in this report</div>
<div class=""><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__purl.org_net_epubs_manifestation_46387165_DL-2DCONF-2D2020-2D001.pdf&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=gSesY1AbeTURZwExR_OGFZlp9YUzrLWyYpGmwAw4Q50&m=T0asmfOta_bLT2cXWrpERYigde5lOqHx2vVIH2WSIOw&s=GNtI2S6yacqAS4bpUYbfq4bDe8nv9gXksMXaqCqgbro&e=" class="">http://purl.org/net/epubs/manifestation/46387165/DL-CONF-2020-001.pdf
[purl.org]</a></div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class=""><br class="">
</div>
</div>
</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, 30 Jun 2020 at 12:21, Jörg Saßmannshausen <<a href="mailto:sassy-work@sassy.formativ.net" class="">sassy-work@sassy.formativ.net</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Dear all,<br class="">
<br class="">
we are currently planning a new cluster and this time around the idea was to <br class="">
use OpenStack for the HPC part of the cluster as well. <br class="">
<br class="">
I was wondering if somebody has some first hand experiences on the list here. <br class="">
One of the things we currently are not so sure about it is InfiniBand (or <br class="">
another low latency network connection but not ethernet): Can you run HPC jobs <br class="">
on OpenStack which require more than the number of cores within a box? I am <br class="">
thinking of programs like CP2K, GROMACS, NWChem (if that sounds familiar to <br class="">
you) which utilise these kind of networks very well. <br class="">
<br class="">
I cam across things like MagicCastle from Computing Canada but as far as I <br class="">
understand it, they are not using it for production (yet). <br class="">
<br class="">
Is anybody on here familiar with this?<br class="">
<br class="">
All the best from London<br class="">
<br class="">
Jörg<br class="">
<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank" class="">
Beowulf@beowulf.org</a> sponsored by Penguin Computing<br class="">
To change your subscription (digest mode or unsubscribe) visit <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__beowulf.org_cgi-2Dbin_mailman_listinfo_beowulf&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=gSesY1AbeTURZwExR_OGFZlp9YUzrLWyYpGmwAw4Q50&m=T0asmfOta_bLT2cXWrpERYigde5lOqHx2vVIH2WSIOw&s=oVEwBKwlVDhzh5JPMjRBZxSAaRPRnCoMIkT-73oONAo&e=" rel="noreferrer" target="_blank" class="">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf [beowulf.org]</a><br class="">
</blockquote>
</div>
_______________________________________________<br class="">
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" class="">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br class="">
To change your subscription (digest mode or unsubscribe) visit <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__beowulf.org_cgi-2Dbin_mailman_listinfo_beowulf&d=DwIGaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=gSesY1AbeTURZwExR_OGFZlp9YUzrLWyYpGmwAw4Q50&m=T0asmfOta_bLT2cXWrpERYigde5lOqHx2vVIH2WSIOw&s=oVEwBKwlVDhzh5JPMjRBZxSAaRPRnCoMIkT-73oONAo&e=" class="">
https://urldefense.proofpoint.com/v2/url?u=https-3A__beowulf.org_cgi-2Dbin_mailman_listinfo_beowulf&d=DwIGaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=gSesY1AbeTURZwExR_OGFZlp9YUzrLWyYpGmwAw4Q50&m=T0asmfOta_bLT2cXWrpERYigde5lOqHx2vVIH2WSIOw&s=oVEwBKwlVDhzh5JPMjRBZxSAaRPRnCoMIkT-73oONAo&e=</a>
<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
</body>
</html>