<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Yes. Oh, I’m sure a modern multicore box could be purchased for my specific workflow from back then – and the codes are easy to run anywhere – Fortran my friends, Fortran – compiles everywhere, uses BLAS and LAPACK.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But it was sort of an example of where deskside HPC (defined as “need more than you can buy in a single box”) might be useful.<o:p></o:p></p>
<p class="MsoNormal">Because, after all, one of the things I was modeling was an array of 288 antennas spanning 1.6 km out at the Owens Valley Radio Observatory Long Wavelength Array. – I was modeling interactions between adjacent antennas. But, of course,
it would be cool to model all 288 (now more than 300, I think, they added some), and since the modeling is a combination of O(N^2) (building the interaction matrices) and O(N^3) (solving the linear equations), I can consume as much horsepower as one might
have. And yes, you’d really want to do this as a multi-grid sort of model, because the interaction between an antenna and another antenna that is 1km away is pretty small.<br>
<br>
http://www.tauceti.caltech.edu/LWA/<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Beowulf <beowulf-bounces@beowulf.org> on behalf of "Michael H. Frese" <Michael.Frese@NumerEx-llC.com><br>
<b>Date: </b>Wednesday, August 25, 2021 at 2:01 PM<br>
<b>To: </b>"beowulf@beowulf.org" <beowulf@beowulf.org><br>
<b>Subject: </b>[EXTERNAL] Re: [Beowulf] Deskside clusters<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p>Jim,<o:p></o:p></p>
<p>I'm reluctant to expose my ignorance, but I think I have some experience to share. And I don't believe this is a cluster issue.<o:p></o:p></p>
<p>My company -- and I do mean my company -- did 2-d multiphysics MHD simulations up until about 4 years ago using our proprietary multiblock domain decomposition code. The diffusive magnetic field solver used all the vector operators in EM. So we share that
much of the problems you are wanting to run.<o:p></o:p></p>
<p>At the end, we did our computations on 4 workstations connected by ancient single-channel Infiniband -- approximately 10 GB/sec but with ~1 micro second latency which was critical for the small messages we were sending.<o:p></o:p></p>
<p>About 10 years ago Mellanox was dumping those and we bought 24 cards and two switches, one 8 port and one 16 port. They cost us a little more than GB ethernet cards, but they were 30-40 times shorter latency.<o:p></o:p></p>
<p>So we built two clusters in 2012. Both used handbuilt deskside workstation boxes which we set up on commercial retail store chrome racks -- NOT 1U or 2U.<o:p></o:p></p>
<p>As time went by, we upgraded to dual core, then quad core, then 8 core CPUs with motherboards and memory to match. We used CentOS 5 for the Infiniband until it wasn't suitable for the motherboards required and then we went to CentOS 7. We never paid for
RHEL.<o:p></o:p></p>
<p>At the last version, we found that we often were able to run problems on single 8-core machines.<o:p></o:p></p>
<p>We stuck with AMD chips because they were faster for the money than Intel's. The last CPUs we bought were capable of hyperthreading so we could run 16 jobs on each 8-core box.<o:p></o:p></p>
<p>I bet one deskside workstation running an <a href="https://urldefense.us/v3/__https:/www.google.com/search?client=firefox-b-1-d&q=amd*12*core__;Kys!!PvBDto6Hs4WbVuu7!bVe3g1e9q03D81iZ-dSQQp6f9gyrtPyg2o75hhV5x0FB1hYkboXeLuT2UsYVy45CZbn5OJw$">
AMD Ryzen 9 5900X 12-core CPU</a> would be able to do the job you want, quietly, at low power, and hence low cooling requirement. Your problems seem to be embarrasingly parallel, but if you need communication between processes you could use in-memory communication,
which is LOTS faster than any Infiniband. And CentOS is fully equipped to get you an MPI with that.<o:p></o:p></p>
<p>Python is available. Don't know about your NEC or plotting software. Source for NEC could be built on your new workstation if necessary.<o:p></o:p></p>
<p>You need a friendly sysadmin and programmer to set it up, get it going, and get you around the list of approved workstations.<o:p></o:p></p>
<p>Hope this isn't too far from your requirements. Good luck!<o:p></o:p></p>
<p>Mike<o:p></o:p></p>
<p><o:p> </o:p></p>
<p><o:p> </o:p></p>
</div>
</body>
</html>