[Beowulf] What services do you run on your cluster nodes?
Robert G. Brown
rgb at phy.duke.edu
Thu Sep 25 12:20:15 PDT 2008
On Wed, 24 Sep 2008, Greg Lindahl wrote:
> On Wed, Sep 24, 2008 at 01:35:10PM -0400, Robert G. Brown wrote:
>
>> So the tradeoff is really a familiar one. Code/Data efficiency vs
>> Code/Data readability and robustness.
>
> The depressing part about this is that XML proponents are unusually
> blind to the unreadability, unportability, and lack of robustness of
> XML. It's just another one of many attempts to do this kind of thing.
> It lacks critical features present in some earlier attempts, and
> repeats some well-known mistakes, too. Yet here we have rgb telling us
> that XML is the answer. This time for sure! Yeah, right.
Not at all. It is AN answer, an approach, a philosophy, a very general
specification. It isn't even, really, a standard, except insofar as it
specifies hierarchical parsing rules (strictly nested tags etc). XMLish
can be done well or it can be done poorly. It almost precisely mirrors
many of the problems with object oriented design, by the way, because
proponents of OO languages all suppose that writers of the code have an
accurate and optimal picture of the "right" data objects for the data
hierarchy and utilization inherent to their code, which is generally not
the case. So I'm happy enough to argue the opposite point of view as
well.
XML CAN make a decent, reasonably human readable, ascii based
encapsulation of suitably hierarchical data. Or you can produce a
botched up mess. C++ (as a religion) can make life simple if applied to
code with well-defined hierarchical data objects that accurately refliec
the actual hierarchies in the data, or it can make a gawdawful mess that
has people preparing to go after the author with pitchforks and torches
as they struggle to figure out how layer after layer obfuscates the
data. Same problem, really.
The fundamental problem is (as Don said as well) that as far as I know
there ARE NO really good solutions to the problem of the representation,
encapsulation, and transmission of hierarchical data structures in a
portable and efficient way. If you know of one, please correct me (and
that isn't a sarcastic request but a serious one). Linus and the
lmbench folks with whom I discussed this didn't have any really great
suggestions -- there are one or two libraries that "can" do this, but
they are typically fairly narrow in what they will work well for, and
they tend NOT to even pretend to be human readable etc on the far end.
XML may be the best thing since sliced bread or it may suck, but one CAN
create XML-compliant markup that CAN do a good job of representing, in
human readable and parsable form, hierarchical data objects that are in
turn good (but "fat" -- not at all information theoretically efficient)
realizations of the "ideal" data objects associated with many problems
in computing. That is why Open Office is xml based. That is why MS
Office is (now) xml based. That is why many other tools are being
written with XML-compliant interfaces. It may not be ideal, but it
often is adequate, and in 100 years people will be able to recover the
data content of e.g. open office documents as long as ASCII itself
hasn't been lost (and even if lost, it is obviously trivially recovered
as long as the alphabet itself isn't lost:-) where the secret decoder
ring for MS Office's many generatations of binary encoded files will
long since have been lost for good.
How far beyond "adequate" it can get depends on how good a match there
is between problem and solution, on how good the
programmer/implementer(s) are, and IMO on how many generations of
software design one has had to get the data objects worked out so that
they are right. I don't think ANY nontrivial OO program design is worth
a damn before the third generation, and some may take until the fifth or
sixth.
In this PARTICULAR case, I prefer to let the output speak for itself:
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
init
send
Content-Length: 7676
<?xml version="1.0"?>
<xmlsysd init="1">
<system>
<time tv_sec="1222370155" tv_usec="111430">3:15:55 pm</time>
<identity>
<hostname>lilith</hostname>
<hostip>192.168.1.129</hostip>
</identity>
<users tv_sec="1222370155" tv_usec="111961">4</users>
</system>
<proc>
<stat tv_sec="1222370155" tv_usec="111973">
<cpu id="0">
<user>29602</user>
<nice>0</nice>
<sys>4576</sys>
<idle>1149413</idle>
</cpu>
<cpu id="1">
<user>27829</user>
<nice>0</nice>
<sys>5405</sys>
<idle>1196517</idle>
</cpu>
<intr>4756531</intr>
<ctxt>6474925</ctxt>
<processes>11909</processes>
<procs_running>2</procs_running>
<procs_blocked>0</procs_blocked>
</stat>
<meminfo tv_sec="1222370155" tv_usec="113117">
<memory>
<total unit="kB">4061508</total>
<free unit="kB">3329628</free>
<used unit="kB">731880</used>
<buffers unit="kB">27520</buffers>
<cached unit="kB">288980</cached>
</memory>
<swap>
<total unit="kB">6144820</total>
<free unit="kB">6144820</free>
<used unit="kB">0</used>
</swap>
</meminfo>
<net>
<dev tv_sec="1222370155" tv_usec="113378">
<interface id="0" devtype="lo" devid="">
<name>lo</name>
<ip>127.0.0.1</ip>
<host>localhost</host>
<receive>
<bytes>7981289</bytes>
<packets>9190</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>7981289</bytes>
<packets>9190</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
<interface id="1" devtype="eth" devid="0">
<name>eth0</name>
<ip>0.0.0.0</ip>
<host>NotConf</host>
<receive>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
<interface id="2" devtype="wmaster" devid="0">
<name>wmaster0</name>
<ip>0.0.0.0</ip>
<host>NotConf</host>
<receive>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
<interface id="3" devtype="wlan" devid="0">
<name>wlan0</name>
<ip>192.168.1.129</ip>
<host>lilith</host>
<receive>
<bytes>9781361</bytes>
<packets>12912</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>2213728</bytes>
<packets>13836</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
<interface id="4" devtype="vmnet" devid="1">
<name>vmnet1</name>
<ip>172.16.234.1</ip>
<host>unknown</host>
<receive>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>0</bytes>
<packets>20</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
<interface id="5" devtype="vmnet" devid="8">
<name>vmnet8</name>
<ip>192.168.100.1</ip>
<host>unknown</host>
<receive>
<bytes>0</bytes>
<packets>0</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<frame>0</frame>
<compressed>0</compressed>
<multicast>0</multicast>
</receive>
<transmit>
<bytes>0</bytes>
<packets>20</packets>
<errs>0</errs>
<drop>0</drop>
<fifo>0</fifo>
<collisions>0</collisions>
<carrier>0</carrier>
<compressed>0</compressed>
</transmit>
</interface>
</dev>
<sockstat tv_sec="1222370155" tv_usec="967040">
<used>538</used>
<tcp>
<inuse>19</inuse>
<orphan>0</orphan>
<timewait>2</timewait>
<alloc>22</alloc>
<mem>30</mem>
</tcp>
<udp>
<inuse>8</inuse>
</udp>
<udp>
<inuse>0</inuse>
</udp>
<raw>
<inuse>1</inuse>
</raw>
<frag>
<inuse>0</inuse>
<memory>0</memory>
</frag>
</sockstat>
</net>
<loadavg tv_sec="1222370155" tv_usec="967193">
<load1>0.08</load1>
<load5>0.06</load5>
<load15>0.01</load15>
</loadavg>
<cpuinfo tv_sec="1222370155" tv_usec="967376">
<processor id="0">
<vendor_id>GenuineIntel</vendor_id>
<family>6</family>
<model_num>23</model_num>
<model_name>Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz</model_name>
<clock units="MHz">800.000</clock>
<cachesize units="KB">6144</cachesize>
<cores>2</cores>
</processor>
<processor id="1">
<vendor_id>GenuineIntel</vendor_id>
<family>6</family>
<model_num>23</model_num>
<model_name>Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz</model_name>
<clock units="MHz">800.000</clock>
<cachesize units="KB">6144</cachesize>
<cores>2</cores>
</processor>
</cpuinfo>
<sysvipc tv_sec="1222370155" tv_usec="967867">
<msgbufs>0</msgbufs>
<msgtot>0</msgtot>
<sembufs>0</sembufs>
<semtot>0</semtot>
<shmbufs>20</shmbufs>
<shmtot>7471472</shmtot>
</sysvipc>
<version>2.6.24.7-92.fc8</version>
<uptime tv_sec="1222370155" tv_usec="968710">
<up>11904.46</up>
<idle>11564.50</idle>
</uptime>
</proc>
</xmlsysd>
quit
Now, anybody unable to read this and make sense of it? Some parts
perhaps -- sysvipc parts are pretty specialized, but they accurately
reflect existing /proc objects (that frankly I don't use and could and
maybe one day will leave off).
All of this information is EASY to extract and use if you know what
parts of proc they are from (and its pretty easy to see where that is if
you know /proc at all). The message isn't intended to BE the human UI,
but it is a damn good PROGRAMMER'S UI seeking to see what is working and
whether or not they are successfully getting the contents of messages.
rgb
--
Robert G. Brown Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977
More information about the Beowulf
mailing list