Parallel Network Traffic Analysis

[Horatio B. Bogbindero]

> 
>         I am looking for some ideas on what sorts of network traffic
> analysis
> would merit the use of a cluster and some parallelization.  Specifically
> I am not looking for ideas just related to processing the headers of
> very high volume networks, but other types of useful analysis of
> captured/real time network traffic, so most likely something that looks
> deeper into the packets of an average small to mid sized campus internet
> connection (ruffly 1*T1 - Partial T3/OC3).  
>         One thing that crept to mind in the beginning of this process
> for me
> was the use of a cluster for doing intrusion detection, by using known
> patterns in the payloads of packets in conjunction with other common
> network intrusion detection techniques.  I perceive problems with this
> idea though.  The first, is that I'm not sure this problem merits more
> power than a single high end workstation has as there are plenty of
> network IDS products that run on a single  PentiumII/PIII class system. 
> Second, the amount of research and data acquisition for the project to
> be useful would be staggering due to the countless types of attacks and
> patterns related to those attacks that would need to be cataloged and
> accounted for in order for the software to be useful.
>         Anyway, are there other types of deep packet analysis that
> anyone knows of that merit the use of cluster/beowulf technology?
> 
hmmm. this sounds nice. our campus network here might be more or less the
same size of your campus network. the core of the network is a fore runner
atm switch. btw, the campus backbone is a 155mbps optical link(OC3??). we
are doing experiments on networking traffic analysis too. however, the
research team here is pretty discouraged due to the lack on incentives to
do research in our university.

first of all we capture packets by letting the forerunner "dump" the
network traffic to a BSD box running NetraMET. the NetraMET box "dumps"
the unanalysed data to a data recorder box. at this stage of the project,
the data is just sitting there.

THE IDEA: with this raw data we can use a neural network to detect
patterns. in your case, network intrusion patterns. clusters/high
performance computing systems will be required because if the sheer volume
of the traffic to be analysed.  the "dumped" data can be used to train the
neural network that if there is a "change" in the network behavior or if
if detects "patterns" that are suspicious it can give out an alert.
(hehehe. maybe even call a program that beeps the systems administrator.
here in the philippines we like using gnokii to send SMS messages to the
system administrator.)
 
will this work? we have some neural networks people and network analysis
people but they do not talk to each other. i just happen to be part of
both projects. 

---------------------
william.s.yu at ieee.org
 
Life is like a tin of sardines.
We're, all of us, looking for the key.
		-- Beyond the Fringe