I am trying to setup an Ubuntu hosting server, currently just for development, and the server has two NICs, each sitting on a different network. eth0 is on 192.168.200.* and eth1 is on 192.168.101.* and each one has a static IP. eth0 is the public facing NIC card and eth1 is strictly for internal access to the server. I initially only setup eth0 and added the eth1 card when I needed it. eth0 was working find until I added eth1, now, can't get any connectivity on eth0 unless I pull eth1 out of the box. The configuration on each system is as follows:

auto eth0
iface eth0 inet static
      address 192.168.200.94
      netmask 255.255.255.0
      network 192.168.200.0
      broadcast 192.168.200.255
      gateway 192.168.200.253

auto eth1
iface eth1 inet static
      address 192.168.101.64
      netmask 255.255.255.0
      network 192.168.101.0
      broadcast 192.168.101.255
      gateway 192.168.101.254

Again eth0 worked fine until I added eth1. I have seen this happen with Windows servers if you have a Default Gateway setup for both NICs, but I am not sure if this works the same on Ubuntu. My resolv.conf file looks like so:

nameserver 192.168.101.59
nameserver 192.168.101.58
domain domain.local
search domain.local

Per request here is the Routing table

192.168.101.0       *                   255.255.255.0     U     0      0      0      eth1
192.168.200.0       *                   255.255.255.0     U     0      0      0      eth0
default             192.168.101.254     0.0.0.0           UG    100    0      0      eth1
default             192.168.200.253     0.0.0.0           UG    100    0      0      eth0