[Beowulf] GPU based cluster fo machine learning purposes

[Douglas Eadline]

> On Thu, Apr 10, 2014 at 02:07:40PM +0000, Lux, Jim (337C) wrote:
>> On 4/10/14 5:28 AM, "Piotr Król" <pietrushnic at gmail.com> wrote:
>>
>> By the time you buy a power supply, add memory, I/o shields, vestigial
>> chassis, your little $50 motherboard is now a $300 computer.
>>
>
> The question is if they can beat $14.7/DP GFLOPS (13GFLOPS/$192) ?

Here are some data points (based on actual measurements):

For about $3,000 (US) you can easily put together a 4 node
X86 system with $6/DP GFLOPS running HPL. Doubling the
number of nodes would probably preserve this
price-to-performance and get you close to 1 DP TFLOP.

In terms of Power/Performance it would run at
around .75 Watt/DP GFLOP running HPL.

Please note, I qualify all my measurements with the
benchmark used to attain these numbers. Depending on
what you want to do, these numbers may vary.

A few notes:

- When using low cost HW for high performance, pay attention
to GbE chip sets. The low cost chipsets (like RealTek)
do not perform well under load. Low cost Intel GbE chipsets
work the best, but newer versions have increased
the small packet latency.

- System tuning and optimization is important. Don't
expect "out of the box" hardware to give you spec sheet
numbers (sometimes it never does). Compilers matter.

- GPU solutions are great if you are sure your problem fits
the hardware. CPUs may not provide the sexy SP FLOPS
rates but they work on a larger problem set and generally
require less optimization than native GPU solutions.
If you want to scale a GPU solution across many nodes,
you may find the ratio of high local compute to low
interconnect speed may be a limiting factor.

--
Doug

>
>> For a prototyping cluster, small size isn't often a real driver (unless
>> you're trying to pack it into a small box for some other reason: lunch
>> box
>> beowulf clusters that fit under a plane seat).  Going to a more
>> conventional (slightly larger) consumer oriented motherboard and an
>> inexpensive consumer oriented power supply might actually give you
>> better
>> bang for the buck.
>
> Sure, (un)fortunately size and power consumption are very important (I
> think more than price - of course in reasonable boundaries). It would be
> great if I could run few nodes on high-end battery in future. So for
> example considering Jetson TK1 power consumption on about 20W I should
> be able to run about 4 nodes with 100W UPS lithium battery. Correct me
> if I introduced confusion here. For sure I have to learn a lot about
> power consumption and clarify this requirements.
>
>> Overclocking and cluster computing don't go together very well.
>> Clusters
>> are sufficiently complex beasts that you don't need the additional
>> failure/flakiness/thermal management hassles that comes from
>> overclocking.
>>
> I will remember this advice.
>>
>>
>> > There is unleashed performance of
>> >VideoCore IV GPU (24 SP GFLOPS) but there is no C compiler for that
>> >(only reverse engineered assembly).
>>
>> Unless you really enjoy hacking at a very low level, you want to pick
>> hardware for which YOU aren't responsible for making the OS and tools
>> work.  You want to spend your time on
>> A) hardware assembly
>> B) learning how to effectively use multiple nodes and a communications
>> fabric
>
> Yes I'm really enjoying low level hacking I'm BIOS developer :P
> Unfortunately I would like to avoid hobbyist approach in this project.
> I'm getting hard lesson that both point A and B are most of the work
> when building cluster.
>
>> Hah.. If you want a real low power/high performance.. Consider the
>> teensy3.1, a sort of super arduino using the Freescale K20 processor
>> based
>> on the ARM Cortex architecture. 30mA, runs at 72 Mhz clock rate, does a
>
> Hah ! :) This one is really nice, unfortunately it doesn't have MMU so
> not Linux and probably a lot of effort for adapting code for this
> platform would be required.
>
> Thanks,
> Piotr Król
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
> --
> Mailscanner: Clean
>
>


--
Doug

-- 
Mailscanner: Clean