<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">ML has been around forever (way back to perceptrons, I built a 2 neuron 3 input classifier using opamps in 1974).. it was the computational horsepower that made empirical experiments with more than 3 layers possible, along with some datasets
to train/test with.<o:p></o:p></p>
<p class="MsoNormal">Without huge datasets of labeled images, for instance, it is hard to try new approaches.<o:p></o:p></p>
<p class="MsoNormal">Fast computers, sitting on a desk, without needing to ask a “high performance computing resource allocation committee” for runtime lets someone do the “hey, what if I stack up 10 layers of neurons” and grind it over night and find “hey
it works, I don’t have any clue why, but it’s cool none-the-less”.. <o:p></o:p></p>
<p class="MsoNormal"><br>
Then, some commercial applications (things like sorting images, or processing video streams) and, all of a sudden, there you go.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This, to me, is the true value of Beowulf – commodity stuff (meaning cheap), ganged up in multiples, lets you have a “personal” HPC capability. Something a bit more than a fast desktop machine, but one I don’t have to justify the use
(or non-use) of. A speedup of 10-50X is the difference between “one try overnight” and “one try in a few minutes when I have some spare time in between the other 13 things I have to do”.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Beowulf <beowulf-bounces@beowulf.org> on behalf of "beowulf@beowulf.org" <beowulf@beowulf.org><br>
<b>Reply-To: </b>John Hearns <hearnsj@googlemail.com><br>
<b>Date: </b>Thursday, May 9, 2019 at 9:54 AM<br>
<b>To: </b>"beowulf@beowulf.org" <beowulf@beowulf.org><br>
<b>Subject: </b>[EXTERNAL] Re: [Beowulf] Frontier Announcement<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Gerald that is an excellent history.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">One small thing though: "Of course the ML came along"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">What came first - the chicken or the egg? Perhaps the Nvidia ecosystem made the ML revolution possible.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">You could run ML models on a cheap workstation or a laptop with an Nvidia GPU.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Indeed I am sitting next to my Nvidia Jetson Nano - 90 dollars for a GPU which can do deep learning.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Prior to CUDA etc. you could of course do machine learning, but it was being done in universities.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I stand to be corrected.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, 9 May 2019 at 17:40, Gerald Henriksen <<a href="mailto:ghenriks@gmail.com">ghenriks@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On Wed, 8 May 2019 14:13:51 -0400, you wrote:<br>
<br>
>On Wed, May 8, 2019 at 1:47 PM Jörg Saßmannshausen <<br>
><a href="mailto:sassy-work@sassy.formativ.net" target="_blank">sassy-work@sassy.formativ.net</a>> wrote:<br>
>><br>
>Once upon a time portability, interoperabiilty, standardization, were<br>
>considered good software and hardware attributes.<br>
>Whatever happened to them?<br>
<br>
I suspect in a lot of cases they were more ideals and goals than<br>
actual things.<br>
<br>
Just look at the struggles the various BSDs have in getting a lot of<br>
software running given the inherent Linuxisms that seem to happen.<br>
<br>
In the case of what is relevant to this discussion, CUDA, Nvidia saw<br>
an opportunity (and perhaps also reacted to the threat of not having<br>
their own CPU to counter the integrated GPU market) and invested<br>
heavily into making their GPUs more than simply a 3D graphics device.<br>
<br>
As Nvidia built up the libraries and other software to make life<br>
easier for programmers to get the most out of Nvidia hardware AMD and<br>
Intel ignored the threat until it was too late, and partial attempts<br>
at open standards struggled.<br>
<br>
And programmers, given struggling with OpenCL or other options vs<br>
going with CUDA with its tools and libraries, went for what gave them<br>
the best performance and easiest implementation (aka a win/win).<br>
<br>
Of course then ML came along and suddenly AMD and Intel couldn't<br>
ignore the market anymore, but they are both struggling from a distant<br>
2nd place to try and replicate the CUDA ecosystem...<br>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf" target="_blank">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>