[Beowulf] 'dual' Quad solution from Tyan

Wed Mar 1 17:56:21 PST 2006

Actually Ricardo,

You might consider ordering the S2881 board instead of S2882 if expansion 
slots is not the problem (it just has 2 pci-x
and the build in video card is real ugly bad supported in windows 64 bits).

It's the board i bought as it has a dual channel memory as you can see in 
the manual.

The S2882 board doesn't show that.

It is true that Tyan boards are not a good board to get problems with, their 
helpdesk is non-existing seemingly (they never answerred in past any 
question i shipped them), and they are difficult to install.
In general i had smaller problems with Asus and Supermicro always, though 
todays dual opteron dual core systems
might be more complex to install anyway.

It is difficult to figure out with that 2881 tyan board where to put the 
powerled on, before you know you make the same mistake like me and put it a 
few pins further. Only detailed study later reveals there is a few 
undocumented pins at the
end there, which you should ignore.

Other BIG problem on these tyan serverboards is it have no AGP nor 
pci-express for videocard.
In your case no problem. In my case a bit bigger problem. I underestimated 
the bad driver support from ATI.
Ati dicks everyone with their 64 bits util, because it doesn't do what it 
says it's doing. In my example already the
past 10 years you can click in ATI at "100 Hz" vertical, but in reality it 
will never put it physical beyond 75Hz.

That's a problem if the CRT is in a field of several microtesla.

Also though it claims it has the latest bios, it simply doesn't have the 
latest bios for the 280's,
so you need to flash it before doing anything else.
Put in 1 dual core cpu, boot it with win98 boot floppy, flash it, then 
continue.

But the memory is dual channel on each memory channel, so that's a lot 
faster than the memory latency from other tyan motherboards and with dual 
core that's very very nice.

Perhaps there is soon better and cheaper boards on the market from other 
manufacturers (iwill, asus ?) that also are dual channel and have 0 of the 
tyan board problems.

Those 2 gigabit NICs of course are not the weakest part of your software. If 
they are, then obviously you should reprogram your software or buy good 
highend network, which will work great in pci-x 133Mhz that the boards have.

3500 euro a node sounds extremely cheap to me personally for a dual core 
dual opteron node.

Most 2 times slower nodes (like having dual xeons) sold by cluster 
manufacturers go to governments for 5000-6000 euro a node.

Those dual core chips really eat a lot of power. 95 watt is no exception 
anymore a chip. Better put some good PSU inside.

You sure you still can get 80GB harddrives in the shops?

That raid thing inside those tyan boards is working fine, but it's a very 
slow raid controller. Silicon Image.
Extremely slow when i put it in raid10 (4 x 300GB S-ATA 150).

Now good raid cards are very expensive of course, but if you can get some 
cheapo dual channel opteron board,
where the dual channel is on each memory channel and not using RAM from both 
cpu's in order to be dual channel,
then obviously it's much better to put in some 30 euro raid card in each 
node, than to pay a lot more for a tyan board.

My time was limited though and i wanted both raid10 + U320 SCSI, so i picked 
of course the first dual channel memory
board i saw and that was this tyan S2881.

Note that certain tyan boards are not so tiny, you need a case where it fits 
in.
The power supply i use is thermaltake 680 watt by the way. Was relative 
cheap here.

Y.M.M.V.

Vincent
----- Original Message ----- 
From: "Joe Landman" <landman at scalableinformatics.com>
To: "Ricardo Reis" <rreis at aero.ist.utl.pt>; <beowulf at beowulf.org>
Sent: Wednesday, March 01, 2006 6:21 PM
Subject: Re: [Beowulf] 'dual' Quad solution from Tyan

> On Tue, 28 Feb 2006 22:38:31 +0000 (WET), Ricardo Reis wrote
>> Thank you all for your reply's.
>>
>>   1. The system will be used for CFD intensive calculation, using
>> comercial and in the house codes, MPI flavor;
>
> You want smaller systems then.
>
>>   2. The cluster I've thought to build initially would be:   * 8
>> nodes (including master), with dual motherboards (2 Opteron CPUs,
>>  single core)   * 16 Opteron 2.4GHz;   * 4 GB per node (32 GB total)
>> ;   * 1 80 Gb disc (SATA II) per node for system and scratch space;
>>  * 2 80 Gb disc (SATA II) for system on master, on RAID 1;   * 3 500
>> Gb disc (SATA II) for storage, home;   * 2 Gigabit switch, one for
>> MPI, another for system and NFS;   * Motherboard is the Tyan
>> S2882G3NR-D;
>
> Not the best choice of MB.  Uses broadcom NICs, and we have seen higher 
> than
> we like failure rates with Tyan MBs at our customers sites.
>
>>    3. I thought that the lantency in this VX50 would be far less
>> than in the Gigabit network;
>
> Possibly, but at a much higher cost.  If latency is your issue go with
> Infinipath or Infiniband (for the moment).  I have been hearing 
> interesting
> things about 10Gbe, but haven't had a chance to look into it yet in great 
> depth.
>
>>    4. The solution for cluster vs. VX50 is around less 3500 euro for
>> the VX50;
>
> Interesting.  You could get a bunch of single CPU boards, load them with 
> dual
> core units, and come in at a lower price point.
>
>>    5. I thought also that the requirements in HVAC would be less for
>> the VX50;
>
> Fewer PS, more fans, more noise, single point of failure (the last one is 
> bad).
>
>>    6. I'm aware and thinking that this technology is new and can be
>> a single-point of failure, regarding the cluster option;
>
> Yes.
>
>>    7. Why 2 single core are better than a dual core? because of
>> sharing resources?
>
> Actually for CFD, it depends upon the code and the memory access patterns. 
> If
> you fill up the memory channel with one core, the second core will have to
> wait to access the memory.
>
>>
>>    thanks for your knowledge sharing,
>>
>>   Ricardo Reis
>>
>>   "Non Serviam"
>>
>>   n.p.: http://radio.ist.utl.pt
>>   n.r.: http://atumtenorio.blogspot.com
>>                      <- Send with Pine Linux/Unix/Win/Mac OS->
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
> --
> Scalable Informatics LLC
> http://www.scalableinformatics.com
> phone: +1 734 786 8423
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
> 