dealing with lots of sockets (was Re: [Beowulf] automount on high ports)
larry.stewart at sicortex.com
Wed Jul 2 17:48:32 PDT 2008
>> Sure, but it is way inefficient. Every single process you fork means
>> another data segment, another stack segment, which means lots of
>> memory. Every process you fork also means that concurrency is
>> only by context switching, which means loads of expense on changing
>> MMU state and more. Even thread switching is orders of magnitude
>> than a procedure call. Invoking an event is essentially just a
>> procedure call, so that wins big time.
My experience is likely (a) dated or (b) inapplicable, but what's the
a group if you can't toss it out?
Back in 1994, with 90 MHz pentiums, NCSA's httpd was the leading
with a design that forked a new process for every request. This
works, and provides nice isolation for those cases where your
is buggy. It is also a poor-man's threading system in that it lets
not worry about blocking behavior of network sockets and so forth. It
a trifle slow however, being limited to 40 or so requests per second.
My obligatory internet startup wrote a new single-threaded single-
server based on select(2) with careful attention to the
blocking or not nature of the kernel calls and were able to handle
of connections per second on the same hardware and over 1000 open
before breaking the stack. Alas it was never made open source and the
More recently at SiCortex, We've been using libevent to write single
applications that do multithreaded things. On our 16 megabyte 70 MHz
freescale embedded boot processors, this is very handy for reducing
footprint. On the x86 front end, a single process has no difficulty
1000 streams of console data this way. I'd hate to have a process for
of those! We're also using conserver for console access and that is
also written with
a single linux process multiplexing 50 or so consoles. I don't know
conserver's internals are threads or events.
So if anyone wants to try an easy to use event library, I can
The learning curve is modest. It does require a little turning inside
out to do things
like have a tftp client as a libevent task but its not bad.
More information about the Beowulf