[vortex-bug] Solved&Summary: RedHat 6.2 and two 3c905B ethernet cards
Thomas Mandl
Thomas.Mandl@neuroth.co.at
Thu, 20 Jul 2000 09:43:46 +0200
Hi List(s),
finaly i got some spare time to write down the solution for my problem
which i posted to redhat-install-list@redhat.com and
vortex-bug@scyld.com. Just thought it could be interesting for other
people too.
The problem:
============
>i'm configuring a Linux RedHat 6.2 box as a router between two ethernets
>using two 3x905B cards and i encountered the following serious problem:
>IP connection works fine, if i have only one 3c905B card in my linux box.
>When i add a second one both interfaces can *NOT* connect to either side (no
>ping trough eth0 nor eth1 is possible). First i thought i've misconfigured
>my routing tables/ifconfig params, but this was not true. Routing
>tables/ifconfig are setup properly. When looking at the 'dmesg output' both
>3c905B cards appear (one at IRQ 10 and one at IRQ15). I do not believe it's
>a hardware/IRQ conflict, but rather some problem with the network drivers
>for these cards.
>I use a stock 2.2.14-5.0 kernel and eth0|1 kernel module is 3c59x version
>v0.99H (11/17/98) from Donald Becker.
>Has anybody experienced similar problems using two 3com ethernet cards?
>Where to start? Is it worth compiling 3Com's original drivers from
>http://support.3com.com/infodeli/tools/nic/linuxdownload.htm
>and installing these drivers or will it work with redhat's native drivers
>from Donald Backer?
The Solution:
=============
Quick info: It was definitively a *simple and stupid* problem, but
nevertheless worth to be mentioned, since many people might have the
same or a similar problem. So here we go:
Step 1: I had a working network connection with only one ethernet card
(i always use 3c905B or 3c905C cards). The default assignment/name for
the primary (let's call it primary here) ethernet card is eth0.
Step 2: When i physically added the second ethernet interface (a 3c905B,
eth1) i assumed that the primary card (which was previously eth0) will
keep it's assignment and will still be eth0! The additionaly added new
interface should become eth1! ***THIS IS NOT TRUE***. The Linux kernel
assiged (swapped) the two ethernet cards and my primary card (eth0)
became eth1 and vice versa. This was somehow confusing, and i found no
explanation for this fact. Well i simply had to swap the ethernet cables
(twisted pair) and i was able to 'ping' both sides of my router! Does
anybody know how the 2.2.14 kernel assigns eth0, eth1, etc... ??? Does
it depend on which IRQ is used by the cards?
I experimented with this on two machines (see the interrupt lists below)
cat "/proc/interrupts" on machine 1 shows:
10: ... XT-PIC eth0
11: ... XT-PIC eth1
...
...
15: ... XT-PIC aic7xxx (my SCSI controller)
cat "/proc/interrupts" on machine 2 shows:
10: ... XT-PIC aic7xxx, eth1
11: ... XT-PIC eth0
...
...
Thus saying the lower IRQ get assigned eth0 and the higher IRQ get
assigned eth1 can not be verified! Any ideas???
For those of you who are interested in a step by step HOW-TO read ahead:
A quick how-to setup a linux router with two interfaces and ipchains
using RedHat 6.2 (kernel 2.2.14-5.0):
1. Install both ethernet cards and make sure you have ***no*** IRQ
conflicts. Consult the documentation of your motherboard to find out the
mapping between PCI Slot# and IRQ #. For my ASUS P3B-F this is as
follows: (a little ASCII chart)
+----------+
| AGP Slot |
+----------+
+-------------+
| PCI Slot #1 |
+-------------+
+-------------+
| PCI Slot #2 |
+-------------+
+-------------+
| PCI Slot #3 |
+-------------+
+-------------+
| PCI Slot #4 |
+-------------+
+-------------+
| PCI Slot #5 |
+-------------+
+-------------+
| ISA Slot #1 |
+-------------+
+-------------+
| ISA Slot #2 |
+-------------+
IRQ Table (for the ASUS P3B-F motherboard)
INT-A INT-B INT-C INT-D
-----------+---------------------------------
PCI Slot 1 | shared -- -- --
PCI Slot 2 | -- shared -- --
PCI Slot 3 | -- -- shared --
PCI Slot 4 | -- -- -- shared
PCI Slot 5 | -- -- -- shared
PCI Slot 6 | -- -- shared --
AGP Slot | shared -- -- --
USB | -- -- -- shared
Generally, an IRQ must be assigned exclusively to one use. In a standard
design there are 16 IRQs available but most of them are already in use,
leaving 6 IRQs free for expansion cards. If your motherboard has PCI
audio on board, then an additional IRQ will be used, If your motherboard
has also MIDI enabled, another IRQ will be used, leaving 4 IRQs free. If
using PCI cards on shared slots, make sure that the driver support
"Shared IRQ". Conflicts will arise between the two PCI groups that will
make the system unstable or cards inoperable.
As you can see (see my /proc/interrupts for machein 1 and machine 2
above) sharing the IRQ with another device (e.g. the SCSI controller or
the AGP slot) will work, but you should be carefull. (Btw. as a hardware
engineer i can not understand how this brain damaged design could ever
make it into the market, but beeing compatible to old designs (PC-XT/AT)
seems to be essential for a successful product - since the PIC is still
8259 compatible ;-| ).
2. Now that you have physically installed eth0 and eth1 continue to
configure these cards (either by using 'linuxconf' or by editing the
files in /etc/sysconfig/network-scripts/ifcfg-eth0 and ifcfg-eth1
manually). Assign both cards a unique IP address (for example
193.155.244.40 for eth0 to connect to your company intranet, and
192.168.0.1 for eth1 to 'hide' your little private network from the
193.155.244.* network).
3. Enable IP_FORWARDING and AUTOMATIC_DEFRAGMENTATION (needed for
masquerading). To do this in RedHat 6.2 (kernel 2.2.14) edit
/etc/sysctl.conf. For older kernels (RH 6.1, 2.2.12) you have to enable
these features in the /proc/sys/net/ipv4/ip_* files by changing the file
contents (0 means disabled, 1 means enabled).
My /etc/sysctl.conf file is here:
# See /usr/doc/kernel-doc-2.2.14/networking/ip-sysctl.txt for detailed
info
# about possible kernel/networking parameters
#
# Enable packet forwarding, we are a router
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enable automatic defragmentation (needed for masquerading, LVS)
net.ipv4.ip_always_defrag = 1
# Disables the magic-sysrq key
kernel.sysrq = 0
# Disable ICMP Echo Request from broadcast addresses but answer direct
pings!
net.ipv4.icmp_echo_ignore_broadcasts = 1
4. Then restart the 'network service' and check if your routing tables
are OK.
They must look like:
root@linuxrouter#> netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
193.155.244.40 0.0.0.0 255.255.255.255 UH 0 0 0
eth0
192.168.0.1 0.0.0.0 255.255.255.255 UH 0 0 0
eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
eth1
193.155.244.0 0.0.0.0 255.255.255.0 U 0 0 0
eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
0.0.0.0 193.155.244.1 0.0.0.0 UG 0 0 0
eth0
5. Last step. enable masquerading using ipchains (or your packets from
the private network 192.168.0.0 will not be able to travel through your
linux router.
A simple test rule for this is:
set ipchain's 'input' and 'output' chain default policy to ALLOW and the
forward chain to MASQ.
Do this by typing: 'ipchains -A forward -j MASQ'
Then you should be able to 'ping' the 'real network' from your private
192.168.0.* IP's
This is only a short summary, but i hope it will be usefull. Consult the
relevant HOW-TO's and man pages for routing, ipchains, networking etc to
find out more.
Btw. i've found an excellent book about securing RedHat Linux 6.1 and
6.2 which also tells you lots of generall stuff (e.g. how to setup a
second ethernet interface, ...)
You will find it at: http://www.openna.com/books/book.php. It's free and
available as PDF document. Read it! It's excellent.
Hope this helps.
Many thanks to all people who responded to my post.
Best regards from Ausria
Thomas
--
Ing. Thomas Mandl
System, Network & Security Admin phone: +43 1 877 56 45/53
Thomas NEUROTH Ges.m.b.H. fax: +43 1 876 49 20
Hietzinger Hauptstrasse 22 e-mail: Thomas.Mandl@neuroth.co.at
A-1130 Vienna/Austria NIC-HDL: TM4373, TM539-RIPE