[Beowulf] PXE boot with X7DWT-INF IB onboard card

[Douglas Eadline]

> On 11/13/12 3:32 PM, Beat Rubischon wrote:
>> "PXE Boot" is a nice container (buzzword? :-) for a hand full of simple
>> steps. First the PXE Boot ROM asks for a DHCP address, second it will
>> load PXElinux over tftp. PXElinux will ask again for a configuration
>> file and finally loads the kernel and an initial ramdisk. Again tftp is
>> used.
>>
>> DHCP and tftp are damn simple protocols on top of IPv4 UDP to avoid a
>> full featured TCP/IP stack in the boot ROMs. They are also pretty slow.
>> Loading a file over TFTP isn't limited by the network, it's usually
>> limited by the slow implementation of the client side stack in the boot
>> ROM. Improvements by switching from GigE to IB won't give you any
>> performance improvements. It's usually also not "big data". We are
>> talking about ~100MBytes for a kernel and an initrd.
>
> Yes, I am reading (and practising) how to boot over IP using DHCP and
> tftp now. Still struggling to get it successfully booted.

Might I suggest:

http://warewulf.lbl.gov/trac

and:

http://hpc.admin-magazine.com/Articles/Warewulf-Cluster-Manager-Master-and-Compute-Nodes

--
Doug


>
>>
>> Your task is now to add all IB relevant stuff into the initrd to make
>> sure your adapters are brought up and you are able to mount your central
>> storage over NFS using IPoIB, NFSoRDMA, iSCSI using IPoIB, SRP or
>> whatever you like to use. BTW: There is no difference in this initrd if
>> you will load it over IB or GigE. It's absolutely the same work.
>>
>> The only advantage of booting over IB using a tool like FlexBoot from
>> Mellanox would be the possibility to eliminate the separate GigE
>> connection to the nodes. But wait, we don't have IPMI over IB. I was
>> told that the Mellanox chips have a dedicated foot providing IPMI, but
>> no mainboard vendor ever connected this one to the BMC. At the end
>> you'll need the GigE cabling for IPMI and I see no point to fiddle
>> around with the firmwares and BIOSses to make the first steps going over
>> IB.
>
> Hum... looks like I still need a lot of readings :).
>
> On 11/13/12 4:20 PM, Jon Tegner wrote:
>> How are you planning to boot your nodes?
>>
>> I have used perceus (http://www.perceus.org/) and was happy with it.
>> There is also Warewulf (http://warewulf.lbl.gov  or
>> http://hpc.admin-magazine.com/Articles/Warewulf-Cluster-Manager-Master-and-Compute-Nodes)
>> which I haven't used.
>>
>> Anyone who has compared the two?
>
> I dont know :(. All of this is new to me. I am reading and testing with
> tftp/dhpc/nfs now. Will try any other recommendation when I am able to
> boot up the nodes.
>
> On 11/13/12 4:30 PM, Beat Rubischon wrote:
>> I usually use the PXE mechanism to boot the standard installer image of
>> my distribution of choice. Every recent distribution brings you the
>> needed files on your installation media - have a look in images/pxeboot/
>> of your RHEL / CentOS / Scientfic DVD, boot/*/loader of your SuSE DVD,
>> the pxe subdirectory of the Ubuntu boot image directory... Once bootet,
>> you deliver the packages using http, nfs etc. Just copy the content of
>> the DVD or FTP directory to your headnode and make it available.
>
> Yes, I did try this, and was able to boot Scientific Linux 6.3 LiveCD.
> But still struggling on next steps.
>
>> To automate the installation by itself every distrbution accepts
>> automatic installation instructions. RedHat calls it Kickstart, SuSE
>> AutoYaST, Debian / Ubuntu Preseed. You'll find plenty of examples in the
>> web. And you'll find plenty of tools automatic this stuff.
>>
>> No need to use CDs oder DVDs. No need to answer any questions. Only need
>> to learn a little bit of systems management :-)
>>
>> Or to pay somebody delivering you support and / or software (-:
>
> Sure, I prefer learning :).
>
> On 11/13/12 5:19 PM, Eugen Leitl wrote:
>> On Tue, Nov 13, 2012 at 10:45:36AM +0700, Duke Nguyen wrote:
>>> Thanks for all suggestions and comments. Looks like we will go for a
>>> gigabyte switch and boot nodes over Gb. Not to mention that this
>> You probably mean GBit Ethernet. Gigabyte is a motherboard vendor.
>>
>
> Yes, that was what I meant.
>
> On 11/13/12 5:29 PM, Hearns, John wrote:
>> My honest advice to you is not to do any of this.
>>
>> There are lots of reliable, knowledgeable companies out there who will
>> only
>> Be too willing to partner with you and construct a cluster, plus
>> expandable storage for you.
>>
>> I suggest that you start looking on various sites, eg Clustermonkey and
>> Hpcwire,
>> And invite some vendors in to talk to you.
>>
>> Many vendors have pre-packaged clusters which would suit you just fine.
>
> Thanks for your very honest advice! Unfortunately it is not similar
> situation like in US, where we can find companies with appropriate
> services to help and assist us. In Vietnam, where we are now, cluster
> systems are already rare. In fact Intel is helping Vietnamese Goverment
> to build a first, big HPC cluster:
>
> http://www.hpcwire.com/hpcwire/2012-10-01/hpc_comes_to_vietnam.html
>
> Not to mention that the service's price, if available in Vietnam, might
> be far beyond our funding. Last, but not least, we are kind of building
> and learning for ourselves, so I prefer to learn (the hard way) :D.
>
> D.
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
> --
> Mailscanner: Clean
>


-- 
Doug

-- 
Mailscanner: Clean