[Beowulf] SGI to offer Windows on clusters
Robert G. Brown
rgb at phy.duke.edu
Mon Apr 16 15:36:29 PDT 2007
On Mon, 16 Apr 2007, Andrew M.A. Cater wrote:
>> This is directly associated with the wish for a minimal install -- I
>> have a system sitting upstairs right now that thinks that it has a
>> problem with a library that a) I've updated repeatedly so that I'm
>> certain that the image I'm installing from matches the one on Duke's
>> mirrors; b) I could care less about anyway -- if I could easily figure
>> out what it is that thinks that it needs it I'd eliminate the toplevel
>> package and get the system to install.
>
> I think you need to look at Debian :) The bare minimum install does have
> some bloat - perl, for example - but it does work.
I don't consider perl, or python in the case of RPM-based distros, to be
bloat per se, as long as the perl installation is self-contained
relative to the base libraries and doesn't cause an explosion in
dependency space. An automated installer has to run on top of
something, and the choices that makes sense are bash (eeeesh --
slackware anyone?), python, perl, or a native compiled application,
ideally a STATIC native compiled application or one that uses only the
core libc dynamic or something like that.
If I were writing it it would get written in C, making it fiendishly
difficult to write in the first place but absolutely portable with
minimal dependency once it was debugged and done. I also would have to
think that its memory footprint in operation would be absolutely
minimal. For a variety of reasons major distro builders opted for an
interpreted language instead -- making it relatively easy to write but
requiring an entire interpreter as part of a baseline core install.
Interpreters are obviously much more susceptible to the three c's --
(feature) creep, cruft, and crack and add a lot of "weight" to the core,
but (sigh) they ARE useful for more than "just" bootstrapping an
automated install tool like anaconda or yum or apt-get.
And I am thinking about Debian. Fairly seriously at this point. Seth
is leaving Duke (I don't think that is secret anymore at this point, in
fact he might be de facto "gone" by today) and while I'm hoping that his
successors will keep up the campus mirrors and distribution repos, I'm
not THAT optimistic that they will. If Fedora does indeed unify and
flatten out its core space with extras and make it even EASIER to blur
dependency separation planes, well -- in two releases it could easily
end up a disaster beyond imagining.
>> The main reason I'm whining online like this is that I really see things
>> going in a really bad, really wrong way fast here. Every new release of
>> FC or RH is bigger and more complex. Very little intelligent effort
>> seems to have been expended in hierarchical decompositions of this
>> increasingly vast system and application space -- even into just this:
>> "systems" vs "application" space -- and as a consequence certain already
>> annoying but not quite critical problems are about to be magnified
>> enormously.
>
> Don't use RH based systems.
Or, try to get them to do things sanely. I'll certainly give this a
shot before abandoning FC, as overall I've been fairly happy with FC and
I do like kickstart.
>> Well, its about to stop being easy. It may stop being POSSIBLE. 500
>> packages was easy. 1500 packages wasn't easy, but with enough effort it
>> was doable. 6000+ packages is doable only because it is effectively
>> decomposed into 1500+4500 or thereabouts. 7000, 9000, 11000 packages as
>> a single set is going to be a living nightmare. FC "as a whole" will
>> simply "never" work for the whole package set in its lifetime, and
>> nobody will know what bugs and dependencies really need to be fixed
>> first as they are twined throughout the entire huge space.
>>
>
> There's no ready way that Fedora can release every 6-9 months or even
> every 18 and retain its quality if it retains its "bleeding edge"-ness.
>
> Debian only makes it because there are, effectively, three parallel
> streams which differ in package churn rather than stability.
>
> "Unstable" may sometimes break. If it doesn't, packages
> gradually percolate down after a period of relative bug-freeness and
> become "testing" - and get tested and refined for up to two years until
> that "testing" gets released as "stable". Then it gets supported as
> unconditionally stable - with security and bugfixes applied but
> minimal change - for the next two years or so. Lather, rinse, repeat.
> The charge that Debian is "always out of date" is simultaneously true -
> and false: it can be as up to date as your last upgrade - and whole
> distribution updates are pushed to the mirrors every twelve hours.
I'm aware of this -- this is something that it manages much better than
the RH-derived distros. Friends who develop for it also argue that it
is much easier to develop for because it has recently been adopting a de
facto standard "development sandbox" that can be loaded as e.g. a vmware
appliance. That way they don't have to worry that the systems they
develop ON will be radically different and somehow things will build on
them but still not install portably. If they can build and install on
the development sandbox, the package is very, very likely to just work
on all derived Debian systems. That is, they are already using
something like an a-b core system as I described it, although I don't
know that they've been completely deliberate about the hierarchical
decompositions involved. Obviously they must have at least in the sense
that "stable" is a kind of a-b set, or perhaps and a-b extended to c in
various places set.
> Sounds like we need to revisit the discussion of how to replace Extreme
> Linux with a "distribution-agnostic" distribution with .rpm, .deb and
> .tgz - July 1998 or so? :) :)
Well, it would certainly help if there were an "Extreme Linux" concept
to revisit, but I think it has been dead since oh yeah, maybe 1999 or
so. I personally think that one could still put together a fairly
minimal common core -- perhaps the union of the perlish debian minimal
install and the pythonoid yum rpm minimal install (where IIRC FC7 is
going to use yum for all package installs, making it MUCH easier to
partition or interrupt robustly and hopefully much easier to minimally
package). If "just" RH/Fedora and Debian agreed to advance "just" this
core synchronously, there is a de facto standard for the LBS project
with the market clout to make it work.
With luck one can even get the two to admit full interoperability of all
tools required to install a distro to this point and a common layout of
/etc, /bin, /lib, /usr and so on -- so that a-level installs are the
same on ALL distros.
It would then be truly lovely to actually do the partitioning at the
b-level and try to build an LBS consensus through b, or at least agree
on where they should all agree (e.g. basic compilers and libraries and
common shared tools and maybe X) and where they should agree to disagree
at b-level. With luck in a few years we might even be able to extend
a-b level compatibility so that all major distros install the complete
a-b level development core "identically".
THEN as far as I'm concerned, at c and higher levels there is a solid
modular base to build on and if you want to mix things all up and play
experimental games, have at it. However, commercial applications would
themselves ALMOST ALWAYS BE SELF CONTAINED at the c level and rely only
on a-b level libraries, and so we might actually get to where
(commercial) application portability is more than just a dream, to where
issues of who "supports" userspace c-d level stuff and for how long is
irrelevant to the stability of the base.
This is what is so irritating. RH spun of Fedora because they were
concerned with the cost and relative lack of profit associated with
supporting RH distros with long tails. However, instead of improving
the hierarchical organization of linux to facilitate less expensive and
more reliable support -- supporting the core for a long time, but
freezing out c-d application level support relatively quickly, for
example (and putting it off onto a FC-like facility that just manages
forward and back porting of major changes into a projection onto that
core, where possible) they made "FC" into a sort of revolving rawhide.
In one sense this has worked well because it actually matches the
frenetic pace of both software tools and desktop and laptop hardware
much better. I'm sorry, but a year is too long to wait for new hardware
to work, and FC has tracked new hardware and hardware support tools
quite aggressively, which is one reason I really do like it. In
another, though, it has REALLY sucked because instead of self-organizing
to manage its own exploding complexity it has dis-organized.
This may prove to be a really good thing in a wierd sort of way. If
this pushes the distro over a catastrophe theory threshold, then it may
THEN self-organize, with some solid measurements and experience of the
catastrophe. The solution it then adopts might actually be far more
robust than any I'm considering here on theoretical grounds, and might
even "change everything" -- move to a completely new paradigm or
hierarchical organization. I know that Seth is very interested in
taking yum and packages in general to this higher level, if he has the
chance whereever he ends up. There are smart people out there, and if
enough of them work on this there could be some real surprises in store.
Maybe FC 7 already has some of them -- we'll see.
I'd love for linux to once again be "extreme". For the last decade it
has worked far too hard to become mainstreme instead (sic deliberate:-)
which gives up too much initiative to companies like Microsoft.
Microsoft is slow as molasses and about as creative as a chinese
sweatshop that turns out reproductions of the art of great masters
(sorry, had to get at least ONE jab at MS in this rant:-) but which is
really good at convincing their customers that THEIR black velvet
reproduction of Elvis is the only one that is truly original and that
all other Elvi are likely to not correctly invoke the Spirit of Rock and
Roll. Linux's primary advantage has long since been its scalability,
and it is time to take that advantage to the next level and cleanly
support "all the software in the world" so that it just works, most of
it for free, across all linux distros.
I promise, I'll stop now.
rgb
>
> Andy
>
>> rgb
>>
>>>
>>>
>>
>> --
>> Robert G. Brown http://www.phy.duke.edu/~rgb/
>> Duke University Dept. of Physics, Box 90305
>> Durham, N.C. 27708-0305
>> Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu
>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>
--
Robert G. Brown http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu
More information about the Beowulf
mailing list