[Beowulf] General cluster management tools - Re: Southampton engineers a Raspberry Pi Supercomputer

[Ellis H. Wilson III]

On 09/16/2012 04:43 PM, Andrew Holway wrote:
>>> This is also demonstrably false. Just because cluster vendor A is
>>> using a completely open source stack does not mean that you have any
>>> less risk then Cluster Vendor B with their proprietary closed source
>>> stack.
>>
>> Risk is a function of your control over the stack against small or large
>> change of business operations of one of the suppliers.  If one of the
>> critical elements of your stack is completely closed, you have no
>> control over that aspect, and cannot change it out without incurring
>> great cost/time/effort, yes, that is, by definition, an increased risk
>> versus a functionally similar part (of similar operational level and
>> quality) which is completely open.
>
> Its part of the risk. If you sign a contract with a cluster software
> vendor that they will support those features for X amount of years
> then you have mitigated that risk. In comparison, you have no control
> over the open source community. It could be that support for a
> critical part of your infrastructure is no longer supported because
> the lead dev decides he wants to spend more time licking frogs in the
> amazon. You are powerless to prevent this.
>
> By your criteria I could easily argue that the risk with opensource is
> higher then with proprietary software.

Something that I think is critical that is being discussed in a 
roundabout fashion, but deserves to be tackled directly, is the size of 
the code-base.  If we are talking about a small to medium size project, 
I think it is pretty hard to argue against the merits of 
open-sourcedness.  If we're talking about a large project this becomes a 
very different situation.

For instance, if the lead dev for a smallish project goes away or you 
want Feature X that isn't currently implemented, no biggie -- just do 
the development in-house.  I do this all the time on my Gentoo box with 
certain packages which I wish I had a feature for.  By the same token, 
if Gentoo decides it doesn't want to provide ebuilds for a particular 
application anymore, provided it isn't some sprawling behemoth I can 
simply ./config make make install {someplace sane} myself and skirt 
package management.

The other side is if you are working with some largish open or semi-open 
source project that just...stops.  In that case, you're in /more/ hot 
water than if it were proprietary.  If you had signed a proprietary 
contract with CompanyX to have support for X years and they change their 
mind or die out in those years, the legal system will have your back and 
you'll get your cut in reparations (even t some extent if they 
bankrupt).  If this is open source -- get ready for a few sleepless 
weekends to migrate your architecture to some maintained infrastructure 
(open-source or not).

So, I think the utilized phrase "horses for courses" is particularly 
appropriate.  I think the level of skill the chariot driver has in 
coding plays a pretty significant role as well.  Personally, even for 
medium-large to large projects I would err on the side of open-source, 
but I can think of lots of people who don't have my random experiences 
with lots of diverse, largish projects to bolster their confidence in 
supporting those kinds of code-bases themselves.

Sorry to be yet another perpetrator of the over-used mantra of YMMV on 
this list! :D

Best,

ellis