<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Generally OpenMPI will be able to autodetect much of the IB setup
you just need to make sure you have UCX. With modern OpenMPI you
will need to build a version of PMIx to hook into Slurm. Slurm
will also need to be built against PMIx as well for best
experience. Thus in terms of order of operations:</p>
<p>1. Make sure UCX is installed and that it is detecting the IB.</p>
<p>2. Make sure to install PMIx</p>
<p>3. Get Slurm up and running built against PMIx (you don't need to
build it against UCX).</p>
<p>4. Build OpenMPI against PMIx, Slurm, and UCX. Generally OpenMPI
will autodetect these but you can provide commandline options to
ensure that.</p>
<p>5. Test and verify it is working.</p>
<p>-Paul Edmon-<br>
</p>
<div class="moz-cite-prefix">On 10/20/2021 6:08 AM, leo camilo
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CA+TOSGfvmBLX=EgA8qjnSsSJQ3J0wK-90cyUeM5odGtVSz_rXw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div> I have recently acquired a few ConnectX-3 cards and an
unmanaged IB switch (IS5022) to upgrade my department's
beowulf cluster. <br>
</div>
<div><br>
</div>
<div>Thus far, I have been able to verify that the cards and
switch work via the MFT and opensource tools in ubuntu, </div>
<div><br>
</div>
<div>Though, I was wondering if anyone knew of any guide or
resources for setting up a cluster for MPI based computations
in a linux/debian environment? Some guides about how to make
it work with SLURM would also be appreciated.<br>
</div>
<div><br>
</div>
<div>Thanks in advance for any suggestions, I am often a user of
clusters, but it is my first time setting one up.<br>
</div>
<div><br>
</div>
<div>Cheers, </div>
<div><br>
</div>
<div>Leonardo<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a>
</pre>
</blockquote>
</body>
</html>