[Beowulf] Re: "hobbyists"

[Robert G. Brown]

On Fri, 20 Jun 2008, Perry E. Metzger wrote:

>
> "Robert G. Brown" <rgb at phy.duke.edu> writes:
>> On Fri, 20 Jun 2008, Chris Samuel wrote:
>>> ----- "Joe Landman" <landman at scalableinformatics.com> wrote:
>>>
>>>> People spend lots of time and effort on security theater.  Make up odd
>>>> rules for passwords.  Make them hard to guess and crack.  Well, is
>>>> that the vector for break-ins?  Weak passwords?
>>>
>>> Yeah - sadly.. :-(
>>
>> Do you have an recent contemporary evidence for that?
>
> Yes, Run a box with sshd on it connected to the internet and watch your
> logs for a few days. You will find numerous attempts to try thousands
> of possible account names and passwords -- brute force cracking.

Well, yeah, sure, I know about that as I DO watch my logs -- I just
haven't heard of one of these attacks SUCCEEDING in pretty much forever,
for obvious reasons.

These attacks run through a list of "likely user names" (including well
known system names like "root") and then run through a short list of
"likely passwords" such as user=test password=test.  Short because ssh
enforces a roughly 1 second interval between tries, which limits the
total number of attempts that can be made to < 100,000/day, with a lot
less actual informational volume to the attempted space.  In fact, to me
it looks like most of the attacks are generated by more or less the same
code and probably reuse most of the same names and password guesses so
that the actual full repertoire tested is really pretty miniscule, even
compared to a dictionary attack. To hit they have to a) match a
username; and b) determine the password that goes with it.  On a site
that runs any sort of a password checker and avoids the dumbest of dumb
passwords (almost by definition, the contents of their list:-) their
probability of success is minute.  They only pluck the very lowest of
the low hanging fruit, fruit that is so low it is basically rotting in
the gutter attracting flies.

> Here is an extract from the log on a real machine, one of mine, from
> last night:
>
> Jun 19 20:56:53 smaug sshd[2577]: Invalid user secretariat from 70.90.14.154
> Jun 19 20:56:54 smaug sshd[2522]: Invalid user secretar from 70.90.14.154
> Jun 19 20:56:55 smaug sshd[23949]: Invalid user present from 70.90.14.154
> Jun 19 20:56:56 smaug sshd[3440]: Invalid user test from 70.90.14.154
> Jun 19 20:56:57 smaug sshd[8809]: Invalid user test from 70.90.14.154
> Jun 19 20:56:58 smaug sshd[21600]: Invalid user teste from 70.90.14.154
> Jun 19 20:56:59 smaug sshd[314]: Invalid user teste from 70.90.14.154

Sure, it goes on and on.  I don't really LIKE seeing this, especially on
a server with sensitive information, but that is precisely why one
configures such servers with tight controls and runs a password checker.

> These attacks are done by automated malware that spreads itself around
> from machine to machine for nefarious purposes -- good luck trying to
> track down the puppet masters. I've tracked the bad guys down a few
> times but they're always somewhere like Bucharest anyway, and the
> locals don't care to arrest them.

Agreed.  All part of the Russian Diabolical Supercomputer we discussed
six months ago or so -- a cloud of unknown proportions and evil
intent:-)

> It is true that this is only one of many modern attack vectors and
> that buffer overflows, drive by malware downloads into browsers, etc.,
> are all far more common ways in, but you will indeed get hacked by
> automated systems if you leave an sshd on and have accounts with weak
> passwords.

Agreed as well.  Which is why professionals generally check for weak
passwords (as do most of the password tools nowadays).  As I said, this
sort of attack only checks for weak combinations in a tiny, tiny
fraction of the phase space, and usually they only run a their
dictionary once (or for at most a day or two) and then move on.  So many
IP numbers to try to crack, so little time.

BTW, one can prevent nearly all of the attacks like the ones above just
by running sshd on a nonstandard port.  They are almost never driven by
a real portscanner-driven attack, the attackers are mindless droids
doing what droids do, which is looking for stupid passwords on a handful
of possible accounts on the standard sshd port and then moving on.  Ssh
running on port umpty-eleven ensures that your server will almost never
get touched (compared to almost always), but if one adds a few simple
lines to your .ssh/config file you can ssh to it completely
transparently.  That's what I did to eliminate most of the "noise" like
that above, which is harmless I suppose but does make me nervous...;-)

Oh, and don't forget the port you used or you'll have to use nmap to
find it again from the outside...;-)

Moving the port also reduces the network load on your servers by a tiny
amount; it costs resources just to say nope, no, not here, wrong
username, sorry, try again later 10,000 or so times a day, and it clogs
up log monitoring programs and /var/log as well.  It doesn't help much
on user workstations or login servers, obviously -- they tend to have to
have standard port numbers so people can find them -- but boy does it
help with the protected/restricted access servers and the like.

>> There are also still -- relatively rarely -- buffer overwrite attacks
>> discovered.
>
> Rarely? You haven't been reading full-disclosure lately I see. There
> are a half dozen new such vulns found a day.

Not like in the old days, when lpd nearly always had one, syslogd had
one for years, etc.  The core daemons run by a standard linux box get
fairly strongly scrutinized.  That doesn't mean that there aren't any,
only that ones in tools that are at all likely to be managing a public
port are discovered relatively infrequently and patched very quickly, in
linux at least (and I don't care about Windows).  nmap a box and unless
somebody is doing something really fancy with it you're likely to get
what, anywhere from 0 to 10 ports, the high end of that only if it is
some sort of server?

As always, with exceptions, especially for novices who run all sorts of
things including e.g. games servers without much of a clue.  And I have
no idea what is discovered in the Windows tree these days.

I'm not trying to minimize the threat posed by buffer overwrite attacks,
BTW.  They are very serious, obviously.  However, again, the risk is
very, very significantly mitigated by tools such as yum or apt.  In the
linux community it isn't that uncommon for an exploit to be discovered
in some important app or daemon Wednesday morning, for it to be closed
up by Wednesday afternoon, for the patched code to be dropped into the
major toplevel repos by Wednesday evening, to be mirrored overnight and
automatically updated on user desktops between Thursday and Saturday
morning depending on your distance down the distribution/repo tree.  Not
a lot of room for traction there, either.

>> Most coders "get it"
>
> No, most of them don't. I've done a lot of code audit in my day. The
> average C programmer turned out these days really thinks you use
> system("rm filename") to unlink a file, and that's the good part of
> their code. For a laugh, google for the "daily wtf" and start reading
> some of the stuff you see.

Very funny!  My own favorite bits of coding humor are a draw between the
Tao of Programming and the fake C++ interview (both on my website,
although I didn't write either of them:-).  I can see why this site
would make a cynic out of you, though.  And I have no doubt that its
examples are real.

But no surprise, really -- I teach coding.  To be fair, you have to
realize that most schools don't teach C programming at all any more, and
many haven't taught a variant of C such as C++ for quite a while either.
There aren't even many competent TEACHERS of C left at schools --
they've been superceded by people teaching java, C++, pascal, visual
basic, etc.  Consequently, I find myself teaching C to independent study
students at Duke -- quite bright kids, usually, often have some idea of
how to code in some language or other -- maybe java (which they do
currently teach, sigh).

The sad truth of the matter is that programming, especially in C, is
really quite difficult.  It takes:

   a) a certain kind of personality -- most computer geeks are
self-selected into the group because they have the right kind.  Here I
am, typing away at 2:30 am because everybody else is in bed and I can
CODE.  I have the right kind too.  Trying to teach a non-coder type to
code is effectively impossible.

   b) a certain amount of native intelligence.  After all, you're
building a machine out of words, so to speak.  In one sense, it is
easier than calculus and differential geometry; in another it embraces
the difficulty of both as one often has to code problems in calculus or
even differential geometry (or worse, linear algebra:-).  Not for the
stupid.

   c) some mix of mentoring by a guru and working your ass off.  A bright
and motivated person can teach themselves C, but I'll be damned if I
think that they can teach themselves to code C >>well<< in less than
years of painful mistake-driven lessons unless they have a guru or some
other way of learning some of the simple wisdom that makes it possible
to turn out decent, readable, code.  And even with a guru and working
quite hard, it still takes years, just fewer years.

I taught myself C with no guru (I already had taught myself fortran,
pascal, basic, and had learned PL/1 and a couple of assember languages
in school) and hence had to learn everything the hardest of ways (and
have made spectacular mistakes over the years).  I still am learning,
and still the hard way -- just this week I had to solve a really nasty
"impossible" bug of my own creation where a variable changed values
literally between lines of code where nothing touched it (when
manipulating pointers nothing is impossible, including impossible
bugs;-).  C is very, very difficult and there is No Safety Net.  You can
perform sublime miracles of coding or make a complete, utter mess with
almost no help at all either way from the compiler or linker.

So I would never mock the poor souls that have yet to learn of unlink or
fstat, only enlighten them.  And pray for deliverance myself from my own
ignorance as I did so, as C is >>complex<< (oh, wait, no, it is
double:-) and I'm certain I still am doing at least some things stupidly
or non-optimally for all that I've written a few thousand lines of C in
the last week and a half in a 20,000 line program (with 2/3 of it
comments and it is running, thank you;-).

BUT, with all of that said -- the end of man strcpy these days contains
gems like:

BUGS
   If the destination string of a strcpy() is not large enough (that is,
   if the programmer was stupid or lazy, and failed to check the size
   before copying) then anything might happen.  Overflowing fixed length
   strings is a favorite cracker technique.

It helps to have a guru to bop you upside the head and tell you "use a
command with an 'n' in it to copy, move, manipulate buffers and here's
why" but in the end, as MY guru once told me, RTFM.  Then read it again.

Don't be lazy and stupid!

>> But weak passwords that are brute force guessed[...]?
>> Only on a poorly managed network,
>
> That would be 95% of networks. I've done a lot of network audits in my
> day, too.

Well, Duke's are pretty good, or the people being cracked are deeply
ashamed and I'm not hearing of them.  Which was not always the case, BTW
-- I used to help people debrief cracking attempts and helped write two
distinct revisions of its security guidelines and policy. So I'm not
sure I'd agree with your 95% of all networks are poorly managed -- that
is a rather large and possibly hyperbolic assignment of incompetence,
after all, more rhetoric than substance.  However, my experience is far
from universal, and there IS certainly plenty of incompetence out there.
And there is no doubt, five users in a hundred WILL choose incredibly
dumb passwords if left to their own devices and two of them will get
angry at you for telling them they have to change them to something more
difficult to guess because then they have to remember something
difficult to guess instead of their own birthday.

We actually have had a really competent "security czar" on campus for a
decade or so as well as good sysadmin groups and a few really bright
lights to lead them, so the mentorship here is perhaps better than
average.  That's what really makes the difference in the long run.
People who take their job seriously and work in a supportive environment
will usually do it decently and avoid the most obvious mistakes.  People
who aren't really bright enough to be a computer geek, who don't have
the right temperament to be a computer geek, and who've had no
mentoring, people who basically chose computing as a career to make
money -- they become MCSEs, don't they?  Not linux people...;-)

    rgb

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977