[Beowulf] Should I go for diskless or not?

[Brian D. Ropers-Huilman]

On Wed, May 13, 2009 at 06:34, Dr Cool Santa <drcoolsanta at gmail.com> wrote:
> I have a cluster of identical computers. We are planning to add more nodes
> later. I was thinking whether I should go the diskless nodes way or not?

That depends on your needs. Some codes will work best with a local
scratch disk, so you might need the disk there anyway.

> Diskless nodes seems as a really exciting, interesting and good option,
> however when I did it I needed to troubleshoot a lot. I did fix it up, but I
> had to redo the filesystem, but the past experiences didn't make much of a
> difference. I still need to fix up everything, I kinda need your help to
> decide.

The right tools can make a big difference here. We use xCat and
ScaliManage on our clusters that are currently (or have in the past)
run diskless. Even running disk-full, the right tools matter.

> Also, performance wise, I was thinking that diskless is not a good option,
> and since performance matters . . .

The only performance issues are that you consume memory for the OS,
which takes it away from applications. Performance wise, you actually
gain by having all the OS in memory (note, I'm not talking about
NFS-root) as there's no need to go to disk for OS issues.

> Can somebody outline the pros and cons of each or just give me thier
> opinion.

For what it's worth, MSI has been moving to diskless (last two systems
plus the one currently on order) and like it.

-- 
Brian D. Ropers-Huilman, Director
Systems Administration and Technical Operations
Minnesota Supercomputing Institute                 <bropers at msi.umn.edu>
599 Walter Library                                   +1 612-626-5948 (V)
117 Pleasant Street S.E.                             +1 612-624-8861 (F)
University of Minnesota                               Twin Cities Campus
Minneapolis, MN 55455-0255                       http://www.msi.umn.edu/