the secret imbecile - Page 43

FreeBSD 8.1 unstable network connection

- by frankcheong

I have three FreeBSD 8.1 running on three different hardware and therefore consist of different network adapter as well (bce, bge and igb). I found that the network connection is kind of unstable which I have tried to scp some 10MB file and found that I cannot always get the files completed successfully. I have further checked with my network admin and he claim that the problem is being caused by the network driver which cannot support the load whereby he tried to ping using huge packet size (around 15k) and my server will drop packet consistently at a regular interval. I found that this statement may not be valid since the three server is using three different network drive and it would be quite impossible that the same problem is being caused by three different network adapter and thus different network driver. Since then I have tried to tune up the performance by playing around with the /etc/sysctl.conf figures with no luck. kern.ipc.somaxconn=1024 kern.ipc.shmall=3276800 kern.ipc.shmmax=1638400000 # Security net.inet.ip.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.icmp.maskrepl=0 net.inet.icmp.log_redirect=0 net.inet.icmp.drop_redirect=1 net.inet.tcp.drop_synfin=1 # Security net.inet.udp.blackhole=1 net.inet.tcp.blackhole=2 # Required by pf net.inet.ip.forwarding=1 #Network Performance Tuning kern.ipc.maxsockbuf=16777216 net.inet.tcp.rfc1323=1 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 # Setting specifically for 1 or even 10Gbps network net.local.stream.sendspace=262144 net.local.stream.recvspace=262144 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.mssdflt=1460 net.inet.tcp.sendbuf_auto=1 net.inet.tcp.sendbuf_inc=16384 net.inet.tcp.recvbuf_auto=1 net.inet.tcp.recvbuf_inc=524288 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 net.inet.udp.recvspace=262144 kern.ipc.maxsockbuf=16777216 kern.ipc.nmbclusters=32768 net.inet.tcp.delayed_ack=1 net.inet.tcp.delacktime=100 net.inet.tcp.slowstart_flightsize=179 net.inet.tcp.inflight.enable=1 net.inet.tcp.inflight.min=6144 # Reduce the cache size of slow start connection net.inet.tcp.hostcache.expire=1 Our network admin also claim that they see quite a lot of network up and down from their cisco switch log while I cannot find any up down message inside the dmesg. Have further checked the netstat -s but dont have concrete idea. tcp: 133695291 packets sent 39408539 data packets (3358837321 bytes) 61868 data packets (89472844 bytes) retransmitted 24 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 50756141 ack-only packets (2148 delayed) 0 URG only packets 0 window probe packets 4372385 window update packets 39781869 control packets 134898031 packets received 72339403 acks (for 3357601899 bytes) 190712 duplicate acks 0 acks for unsent data 59339201 packets (3647021974 bytes) received in-sequence 114 completely duplicate packets (135202 bytes) 27 old duplicate packets 0 packets with some dup. data (0 bytes duped) 42090 out-of-order packets (60817889 bytes) 0 packets (0 bytes) of data after window 0 window probes 3953896 window update packets 64181 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 45192 discarded due to memory problems 19945391 connection requests 1323420 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 21133581 connections established (including accepts) 21268724 connections closed (including 32737 drops) 207874 connections updated cached RTT on close 207874 connections updated cached RTT variance on close 132439 connections updated cached ssthresh on close 42392 embryonic connections dropped 72339338 segments updated rtt (of 69477829 attempts) 390871 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 13990 keepalive timeouts 2 keepalive probes sent 13988 connections dropped by keepalive 173044 correct ACK header predictions 36947371 correct data packet header predictions 1323420 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 1323420 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 1323420 cookies sent 0 cookies received 1864 SACK recovery episodes 18005 segment rexmits in SACK recovery episodes 26066896 byte rexmits in SACK recovery episodes 147327 SACK options (SACK blocks) received 87473 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 5141258 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 1 with no checksum 0 dropped due to no socket 129616 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 5011642 delivered 5016050 datagrams output 0 times multicast source filter matched sctp: 0 input packets 0 datagrams 0 packets that had data 0 input SACK chunks 0 input DATA chunks 0 duplicate DATA chunks 0 input HB chunks 0 HB-ACK chunks 0 input ECNE chunks 0 input AUTH chunks 0 chunks missing AUTH 0 invalid HMAC ids received 0 invalid secret ids received 0 auth failed 0 fast path receives all one chunk 0 fast path multi-part data 0 output packets 0 output SACKs 0 output DATA chunks 0 retransmitted DATA chunks 0 fast retransmitted DATA chunks 0 FR's that happened more than once to same chunk 0 intput HB chunks 0 output ECNE chunks 0 output AUTH chunks 0 ip_output error counter Packet drop statistics: 0 from middle box 0 from end host 0 with data 0 non-data, non-endhost 0 non-endhost, bandwidth rep only 0 not enough for chunk header 0 not enough data to confirm 0 where process_chunk_drop said break 0 failed to find TSN 0 attempt reverse TSN lookup 0 e-host confirms zero-rwnd 0 midbox confirms no space 0 data did not match TSN 0 TSN's marked for Fast Retran Timeouts: 0 iterator timers fired 0 T3 data time outs 0 window probe (T3) timers fired 0 INIT timers fired 0 sack timers fired 0 shutdown timers fired 0 heartbeat timers fired 0 a cookie timeout fired 0 an endpoint changed its cookiesecret 0 PMTU timers fired 0 shutdown ack timers fired 0 shutdown guard timers fired 0 stream reset timers fired 0 early FR timers fired 0 an asconf timer fired 0 auto close timer fired 0 asoc free timers expired 0 inp free timers expired 0 packet shorter than header 0 checksum error 0 no endpoint for port 0 bad v-tag 0 bad SID 0 no memory 0 number of multiple FR in a RTT window 0 RFC813 allowed sending 0 RFC813 does not allow sending 0 times max burst prohibited sending 0 look ahead tells us no memory in interface 0 numbers of window probes sent 0 times an output error to clamp down on next user send 0 times sctp_senderrors were caused from a user 0 number of in data drops due to chunk limit reached 0 number of in data drops due to rwnd limit reached 0 times a ECN reduced the cwnd 0 used express lookup via vtag 0 collision in express lookup 0 times the sender ran dry of user data on primary 0 same for above 0 sacks the slow way 0 window update only sacks sent 0 sends with sinfo_flags !=0 0 unordered sends 0 sends with EOF flag set 0 sends with ABORT flag set 0 times protocol drain called 0 times we did a protocol drain 0 times recv was called with peek 0 cached chunks used 0 cached stream oq's used 0 unread messages abandonded by close 0 send burst avoidance, already max burst inflight to net 0 send cwnd full avoidance, already max burst inflight to net 0 number of map array over-runs via fwd-tsn's ip: 137814085 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 1200 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 300 packets reassembled ok 137813009 packets for this host 530 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 61 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 137234598 packets sent from this host 0 packets sent with fabricated ip header 685307 output packets dropped due to no bufs, etc. 52 output packets discarded due to no route 300 output datagrams fragmented 1200 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: echo reply: 305 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 530 echo: 305 305 message responses generated 0 invalid return addresses 0 no return routes ICMP address mask responses are disabled igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 376748 ARP requests sent 3207 ARP replies sent 245245 ARP requests received 80845 ARP replies received 326090 ARP packets received 267712 total packets dropped due to no ARP entry 108876 ARP entrys timed out 0 Duplicate IPs seen ip6: 2226633 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 2226633 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 2226633 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 8 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Input histogram: UDP: 2226633 Mbuf statistics: 962679 one mbuf 1263954 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not continuous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output netstat -m 516/5124/5640 mbufs in use (current/cache/total) 512/1634/2146/32768 mbuf clusters in use (current/cache/total/max) 512/1536 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1303/1303/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1153K/9761K/10914K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/8/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Anyone got an idea what might be the possible cause?

Read the article

Can't seem to get C TCP Server-Client Communications Right

- by Zeesponge

Ok i need some serious help here. I have to make a TCP Server Client. When the Client connects to server using a three stage handshake. AFterwards... while the Client is running in the terminal, the user enters linux shell commands like xinput list, ls -1, ect... something that uses standard output. The server accepts the commands and uses system() (in a fork() in an infinite loop) to run the commands and the standard output is redirected to the client, where the client prints out each line. Afterward the server sends a completion signal of "\377\n". In which the client goes back to the command prompt asking for a new command and closes its connection and exit()'s when inputting "quit". I know that you have to dup2() both the STDOUT_FILENO and STDERR_FILENO to the clients file descriptor {dup2(client_FD, STDOUT_FILENO). Everything works accept when it comes for the client to retrieve system()'s stdout and printing it out... all i get is a blank line with a blinking cursor (client waiting on stdin). I tried all kinds of different routes with no avail... If anyone can help out i would greatly appreciate it TCP SERVER CODE include #include <sys/socket.h> #include <stdio.h> #include <string.h> #include <netinet/in.h> #include <signal.h> #include <unistd.h> #include <stdlib.h> #include <errno.h> //Prototype void handle_client(int connect_fd); int main() { int server_sockfd, client_sockfd; socklen_t server_len, client_len; struct sockaddr_in server_address; struct sockaddr_in client_address; server_sockfd = socket(AF_INET, SOCK_STREAM, 0); server_address.sin_family = AF_INET; server_address.sin_addr.s_addr = htonl(INADDR_ANY); server_address.sin_port = htons(9734); server_len = sizeof(server_address); bind(server_sockfd, (struct sockaddr *)&server_address, server_len); /* Create a connection queue, ignore child exit details and wait for clients. */ listen(server_sockfd, 10); signal(SIGCHLD, SIG_IGN); while(1) { printf("server waiting\n"); client_len = sizeof(client_address); client_sockfd = accept(server_sockfd, (struct sockaddr *)&client_address, &client_len); if(fork() == 0) handle_client(client_sockfd); else close(client_sockfd); } } void handle_client(int connect_fd) { const char* remsh = "<remsh>\n"; const char* ready = "<ready>\n"; const char* ok = "<ok>\n"; const char* command = "<command>\n"; const char* complete = "<\377\n"; const char* shared_secret = "<shapoopi>\n"; static char server_msg[201]; static char client_msg[201]; static char commands[201]; int sys_return; //memset client_msg, server_msg, commands memset(&client_msg, 0, sizeof(client_msg)); memset(&server_msg, 0, sizeof(client_msg)); memset(&commands, 0, sizeof(commands)); //read remsh from client read(connect_fd, &client_msg, 200); //check remsh validity from client if(strcmp(client_msg, remsh) != 0) { errno++; perror("Error Establishing Handshake"); close(connect_fd); exit(1); } //memset client_msg memset(&client_msg, 0, sizeof(client_msg)); //write remsh to client write(connect_fd, remsh, strlen(remsh)); //read shared_secret from client read(connect_fd, &client_msg, 200); //check shared_secret validity from client if(strcmp(client_msg, shared_secret) != 0) { errno++; perror("Invalid Security Passphrase"); write(connect_fd, "no", 2); close(connect_fd); exit(1); } //memset client_msg memset(&client_msg, 0, sizeof(client_msg)); //write ok to client write(connect_fd, ok, strlen(ok)); // dup2 STDOUT_FILENO <= client fd, STDERR_FILENO <= client fd dup2(connect_fd, STDOUT_FILENO); dup2(connect_fd, STDERR_FILENO); //begin while... while read (client_msg) from server and >0 while(read(connect_fd, &client_msg, 200) > 0) { //check command validity from client if(strcmp(client_msg, command) != 0) { errno++; perror("Error, unable to retrieve data"); close(connect_fd); exit(1); } //memset client_msg memset(&client_msg, 0, sizeof(client_msg)); //write ready to client write(connect_fd, ready, strlen(ready)); //read commands from client read(connect_fd, &commands, 200); //run commands using system( ) sys_return = system(commands); //check success of system( ) if(sys_return < 0) { perror("Invalid Commands"); errno++; } //memset commands memset(commands, 0, sizeof(commands)); //write complete to client write(connect_fd, complete, sizeof(complete)); } } TCP CLIENT CODE #include <sys/types.h> #include <sys/socket.h> #include <stdio.h> #include <string.h> #include <netinet/in.h> #include <arpa/inet.h> #include <unistd.h> #include <stdlib.h> #include <errno.h> #include "readline.c" int main(int argc, char *argv[]) { int sockfd; int len; struct sockaddr_in address; int result; const char* remsh = "<remsh>\n"; const char* ready = "<ready>\n"; const char* ok = "<ok>\n"; const char* command = "<command>\n"; const char* complete = "<\377\n"; const char* shared_secret = "<shapoopi>\n"; static char server_msg[201]; static char client_msg[201]; memset(&client_msg, 0, sizeof(client_msg)); memset(&server_msg, 0, sizeof(server_msg)); /* Create a socket for the client. */ sockfd = socket(AF_INET, SOCK_STREAM, 0); /* Name the socket, as agreed with the server. */ memset(&address, 0, sizeof(address)); address.sin_family = AF_INET; address.sin_addr.s_addr = inet_addr(argv[1]); address.sin_port = htons(9734); len = sizeof(address); /* Now connect our socket to the server's socket. */ result = connect(sockfd, (struct sockaddr *)&address, len); if(result == -1) { perror("ACCESS DENIED"); exit(1); } //write remsh to server write(sockfd, remsh, strlen(remsh)); //read remsh from server read(sockfd, &server_msg, 200); //check remsh validity from server if(strcmp(server_msg, remsh) != 0) { errno++; perror("Error Establishing Initial Handshake"); close(sockfd); exit(1); } //memset server_msg memset(&server_msg, 0, sizeof(server_msg)); //write shared secret text to server write(sockfd, shared_secret, strlen(shared_secret)); //read ok from server read(sockfd, &server_msg, 200); //check ok velidity from server if(strcmp(server_msg, ok) != 0 ) { errno++; perror("Incorrect security phrase"); close(sockfd); exit(1); } //? dup2 STDIN_FILENO = server socket fd? //dup2(sockfd, STDIN_FILENO); //begin while(1)/////////////////////////////////////// while(1){ //memset both msg arrays memset(&client_msg, 0, sizeof(client_msg)); memset(&server_msg, 0, sizeof(server_msg)); //print Enter Command, scan input, fflush to stdout printf("<<Enter Command>> "); scanf("%s", client_msg); fflush(stdout); //check quit input, if true close and exit successfully if(strcmp(client_msg, "quit") == 0) { printf("Exiting\n"); close(sockfd); exit(EXIT_SUCCESS); } //write command to server write(sockfd, command, strlen(command)); //read ready from server read(sockfd, &server_msg, 200); //check ready validity from server if(strcmp(server_msg, ready) != 0) { errno++; perror("Failed Server Communications"); close(sockfd); exit(1); } //memset server_msg memset(&server_msg, 0, sizeof(server_msg)); //begin looping and retrieving from stdin, //break loop at EOF or complete while((read(sockfd, server_msg, 200) != 0) && (strcmp(server_msg, complete) != 0)) { //while((fgets(server_msg, 4096, stdin) != EOF) || (strcmp(server_msg, complete) == 0)) { printf("%s", server_msg); memset(&server_msg, 0, sizeof(server_msg)); } } }

Read the article

Redirecting or routing all traffic to OpenVPN on a Mac OS X client

- by sdr56p

I have configured an OpenVPN (2.2.1) server on an Ubuntu virtual machine in the Amazon elastic compute cloud. The server is up and running. I have installed OpenVPN (2.2.1) on a Mac OS X (10.8.2) client and I am using the openvpn2 binary to connect (in opposition to other clients like Tunnelblick or Viscosity). I can connect with the client and successfully ping or ssh the server through the tunnel. However, I can't redirect all internet traffic through the VPN even if I use the push "redirect-gateway def1 bypass-dhcp" option in the server.conf configurations. When I connect to the server with these configurations, I get a successful connection, but then an infinite series of error messages: "write UDPv4: No route to host (code=65)". Traffic routing seems to be compromised because I am not able to access anything anymore, not even the OpenVPN server (by pinging 10.8.0.1 for instance). This is beyond me. I am finding little help on the web and don't know what to try next. I don't think it is a problem of forwarding the traffic on the server since, first, I have also took care of that and, second, I can't even ping the VPN server locally through the tunnel (or ping anything at all for that matter). Thank you for your help. Here is the server.conf. file: port 1194 proto udp dev tun ca ca.crt cert ec2-server.crt key ec2-server.key # This file should be kept secret dh dh1024.pem server 10.8.0.0 255.255.255.0 ifconfig-pool-persist ipp.txt push "redirect-gateway def1 bypass-dhcp" client-to-client keepalive 10 120 comp-lzo persist-key persist-tun status openvpn-status.log verb 3 And the client.conf file: client dev tun proto udp remote servername.com 1194 resolv-retry infinite nobind persist-key persist-tun ca ca.crt cert Toto5.crt key Toto5.key ns-cert-type server comp-lzo verb 3 Here is the connection log with the error messages: $ sudo openvpn2 --config client.conf Wed Mar 13 22:58:22 2013 OpenVPN 2.2.1 x86_64-apple-darwin12.2.0 [SSL] [LZO2] [eurephia] built on Mar 4 2013 Wed Mar 13 22:58:22 2013 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables Wed Mar 13 22:58:22 2013 LZO compression initialized Wed Mar 13 22:58:22 2013 Control Channel MTU parms [ L:1542 D:138 EF:38 EB:0 ET:0 EL:0 ] Wed Mar 13 22:58:22 2013 Socket Buffers: R=[196724->65536] S=[9216->65536] Wed Mar 13 22:58:22 2013 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:135 ET:0 EL:0 AF:3/1 ] Wed Mar 13 22:58:22 2013 Local Options hash (VER=V4): '41690919' Wed Mar 13 22:58:22 2013 Expected Remote Options hash (VER=V4): '530fdded' Wed Mar 13 22:58:22 2013 UDPv4 link local: [undef] Wed Mar 13 22:58:22 2013 UDPv4 link remote: 54.234.43.171:1194 Wed Mar 13 22:58:22 2013 TLS: Initial packet from 54.234.43.171:1194, sid=ffbaf343 d0c1a266 Wed Mar 13 22:58:22 2013 VERIFY OK: depth=1, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funst ... ost.domain Wed Mar 13 22:58:22 2013 VERIFY OK: nsCertType=SERVER Wed Mar 13 22:58:22 2013 VERIFY OK: depth=0, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funst ... ost.domain Wed Mar 13 22:58:23 2013 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 13 22:58:23 2013 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 13 22:58:23 2013 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 13 22:58:23 2013 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 13 22:58:23 2013 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA Wed Mar 13 22:58:23 2013 [ec2-server] Peer Connection Initiated with 54.234.43.171:1194 Wed Mar 13 22:58:25 2013 SENT CONTROL [ec2-server]: 'PUSH_REQUEST' (status=1) Wed Mar 13 22:58:25 2013 PUSH: Received control message: 'PUSH_REPLY,route 10.8.0.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.8.0.6 10.8.0.5' Wed Mar 13 22:58:25 2013 OPTIONS IMPORT: timers and/or timeouts modified Wed Mar 13 22:58:25 2013 OPTIONS IMPORT: --ifconfig/up options modified Wed Mar 13 22:58:25 2013 OPTIONS IMPORT: route options modified Wed Mar 13 22:58:25 2013 ROUTE default_gateway=0.0.0.0 Wed Mar 13 22:58:25 2013 TUN/TAP device /dev/tun0 opened Wed Mar 13 22:58:25 2013 /sbin/ifconfig tun0 delete ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address Wed Mar 13 22:58:25 2013 NOTE: Tried to delete pre-existing tun/tap instance -- No Problem if failure Wed Mar 13 22:58:25 2013 /sbin/ifconfig tun0 10.8.0.6 10.8.0.5 mtu 1500 netmask 255.255.255.255 up Wed Mar 13 22:58:25 2013 /sbin/route add -net 10.8.0.0 10.8.0.5 255.255.255.0 add net 10.8.0.0: gateway 10.8.0.5 Wed Mar 13 22:58:25 2013 Initialization Sequence Completed ^CWed Mar 13 22:58:30 2013 event_wait : Interrupted system call (code=4) Wed Mar 13 22:58:30 2013 TCP/UDP: Closing socket Wed Mar 13 22:58:30 2013 /sbin/route delete -net 10.8.0.0 10.8.0.5 255.255.255.0 delete net 10.8.0.0: gateway 10.8.0.5 Wed Mar 13 22:58:30 2013 Closing TUN/TAP interface Wed Mar 13 22:58:30 2013 SIGINT[hard,] received, process exiting toto5:ttntec2 Dominic$ sudo openvpn2 --config client.conf --remote ec2-54-234-43-171.compute-1.amazonaws.com Wed Mar 13 22:58:57 2013 OpenVPN 2.2.1 x86_64-apple-darwin12.2.0 [SSL] [LZO2] [eurephia] built on Mar 4 2013 Wed Mar 13 22:58:57 2013 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables Wed Mar 13 22:58:57 2013 LZO compression initialized Wed Mar 13 22:58:57 2013 Control Channel MTU parms [ L:1542 D:138 EF:38 EB:0 ET:0 EL:0 ] Wed Mar 13 22:58:57 2013 Socket Buffers: R=[196724->65536] S=[9216->65536] Wed Mar 13 22:58:57 2013 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:135 ET:0 EL:0 AF:3/1 ] Wed Mar 13 22:58:57 2013 Local Options hash (VER=V4): '41690919' Wed Mar 13 22:58:57 2013 Expected Remote Options hash (VER=V4): '530fdded' Wed Mar 13 22:58:57 2013 UDPv4 link local: [undef] Wed Mar 13 22:58:57 2013 UDPv4 link remote: 54.234.43.171:1194 Wed Mar 13 22:58:57 2013 TLS: Initial packet from 54.234.43.171:1194, sid=a0d75468 ec26de14 Wed Mar 13 22:58:58 2013 VERIFY OK: depth=1, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funst ... ost.domain Wed Mar 13 22:58:58 2013 VERIFY OK: nsCertType=SERVER Wed Mar 13 22:58:58 2013 VERIFY OK: depth=0, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funst ... ost.domain Wed Mar 13 22:58:58 2013 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 13 22:58:58 2013 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 13 22:58:58 2013 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 13 22:58:58 2013 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 13 22:58:58 2013 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA Wed Mar 13 22:58:58 2013 [ec2-server] Peer Connection Initiated with 54.234.43.171:1194 Wed Mar 13 22:59:00 2013 SENT CONTROL [ec2-server]: 'PUSH_REQUEST' (status=1) Wed Mar 13 22:59:00 2013 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,route 10.8.0.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.8.0.6 10.8.0.5' Wed Mar 13 22:59:00 2013 OPTIONS IMPORT: timers and/or timeouts modified Wed Mar 13 22:59:00 2013 OPTIONS IMPORT: --ifconfig/up options modified Wed Mar 13 22:59:00 2013 OPTIONS IMPORT: route options modified Wed Mar 13 22:59:00 2013 ROUTE default_gateway=0.0.0.0 Wed Mar 13 22:59:00 2013 TUN/TAP device /dev/tun0 opened Wed Mar 13 22:59:00 2013 /sbin/ifconfig tun0 delete ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address Wed Mar 13 22:59:00 2013 NOTE: Tried to delete pre-existing tun/tap instance -- No Problem if failure Wed Mar 13 22:59:00 2013 /sbin/ifconfig tun0 10.8.0.6 10.8.0.5 mtu 1500 netmask 255.255.255.255 up Wed Mar 13 22:59:00 2013 /sbin/route add -net 54.234.43.171 0.0.0.0 255.255.255.255 add net 54.234.43.171: gateway 0.0.0.0 Wed Mar 13 22:59:00 2013 /sbin/route add -net 0.0.0.0 10.8.0.5 128.0.0.0 add net 0.0.0.0: gateway 10.8.0.5 Wed Mar 13 22:59:00 2013 /sbin/route add -net 128.0.0.0 10.8.0.5 128.0.0.0 add net 128.0.0.0: gateway 10.8.0.5 Wed Mar 13 22:59:00 2013 /sbin/route add -net 10.8.0.0 10.8.0.5 255.255.255.0 add net 10.8.0.0: gateway 10.8.0.5 Wed Mar 13 22:59:00 2013 Initialization Sequence Completed Wed Mar 13 22:59:00 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:00 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:01 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:01 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:01 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:02 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:02 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:02 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:02 2013 write UDPv4: No route to host (code=65) Wed Mar 13 22:59:02 2013 write UDPv4: No route to host (code=65) ... The routing table after a connection WITHOUT the push redirect-gateway (all traffic is not redirected to the VPN and everything is working fine, I can ping or ssh the OpenVPN server and access all other Internet resources through my default gateway): Destination Gateway Flags Refs Use Netif Expire default user148-1.wireless UGSc 50 0 en1 10.8/24 10.8.0.5 UGSc 2 7 tun0 10.8.0.5 10.8.0.6 UH 3 2 tun0 127 localhost UCS 0 0 lo0 localhost localhost UH 6 6692 lo0 client.openvpn.net client.openvpn.net UH 3 18 lo0 142.1.148/22 link#5 UCS 2 0 en1 user148-1.wireless 0:90:b:27:10:71 UHLWIir 50 0 en1 76 user150-173.wirele localhost UHS 0 0 lo0 142.1.151.255 ff:ff:ff:ff:ff:ff UHLWbI 0 2 en1 169.254 link#5 UCS 1 0 en1 169.254.255.255 0:90:b:27:10:71 UHLSWi 0 0 en1 71 The routing table after a connection with the push redirect-gateway option enable as in the server.conf file above (all internet traffic should be redirected to the VPN tunnel, but nothing is working, I can't access any Internet ressources at all): Destination Gateway Flags Refs Use Netif Expire 0/1 10.8.0.5 UGSc 1 0 tun0 default user148-1.wireless UGSc 7 0 en1 10.8/24 10.8.0.5 UGSc 0 0 tun0 10.8.0.5 10.8.0.6 UHr 6 0 tun0 54.234.43.171/32 0.0.0.0 UGSc 1 0 en1 127 localhost UCS 0 0 lo0 localhost localhost UH 3 6698 lo0 client.openvpn.net client.openvpn.net UH 0 27 lo0 128.0/1 10.8.0.5 UGSc 2 0 tun0 142.1.148/22 link#5 UCS 1 0 en1 user148-1.wireless 0:90:b:27:10:71 UHLWIir 1 0 en1 833 user150-173.wirele localhost UHS 0 0 lo0 169.254 link#5 UCS 1 0 en1 169.254.255.255 0:90:b:27:10:71 UHLSW 0 0 en1

Read the article

JavaMail not sending Subject or From under jetty:run-war

- by Jason Thrasher

Has anyone seen JavaMail not sending proper MimeMessages to an SMTP server, depending on how the JVM in started? At the end of the day, I can't send JavaMail SMTP messages with Subject: or From: fields, and it appears other headers are missing, only when running the app as a war. The web project is built with Maven and I'm testing sending JavaMail using a browser and a simple mail.jsp to debug and see different behavior when launching the app with: 1) mvn jetty:run (mail sends fine, with proper Subject and From fields) 2) mvn jetty:run-war (mail sends fine, but missing Subject, From, and other fields) I've meticulously run diff on the (verbose) Maven debug output (-X), and there are zero differences in the runtime dependencies between the two. I've also compared System properties, and they are identical. Something else is happening the jetty:run-war case that changes the way JavaMail behaves. What other stones need turning? Curiously, I've tried a debugger in both situations and found that the javax.mail.internet.MimeMessage instance is getting created differently. The webapp is using Spring to send email picked off of an Apache ActiveMQ queue. When running the app as mvn jetty:run the MimeMessage.contentStream variable is used for message content. When running as mvn jetty:run-war, the MimeMessage.content variable is used for the message contents, and the content = ASCIIUtility.getBytes(is); call removes all of the header data from the parsed content. Since this seemed very odd, and debugging Spring/ActiveMQ is a deep dive, I created a simplified test without any of that infrastructure: just a JSP using mail-1.4.2.jar, yet the same headers are missing. Also of note, these headers are missing when running the WAR file under Tomcat 5.5.27. Tomcat behaves just like Jetty when running the WAR, with the same missing headers. With JavaMail debugging turned on, I clearly see different output. GOOD CASE: In the jetty:run (non-WAR) the log output is: DEBUG: JavaMail version 1.4.2 DEBUG: successfully loaded resource: /META-INF/javamail.default.providers DEBUG: Tables of loaded providers DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Sun Microsystems, Inc], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Sun Microsystems, Inc], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Sun Microsystems, Inc]} DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Sun Microsystems, Inc], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Sun Microsystems, Inc], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Sun Microsystems, Inc], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc]} DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc] DEBUG SMTP: useEhlo true, useAuth true DEBUG SMTP: trying to connect to host "mail.authsmtp.com", port 465, isSSL false 220 mail.authsmtp.com ESMTP Sendmail 8.14.2/8.14.2/Kp; Thu, 18 Jun 2009 01:35:24 +0100 (BST) DEBUG SMTP: connected to host "mail.authsmtp.com", port: 465 EHLO jmac.local 250-mail.authsmtp.com Hello sul-pubs-3a.Stanford.EDU [171.66.201.2], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 52428800 250-AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN 250-DELIVERBY 250 HELP DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg "" DEBUG SMTP: Found extension "PIPELINING", arg "" DEBUG SMTP: Found extension "8BITMIME", arg "" DEBUG SMTP: Found extension "SIZE", arg "52428800" DEBUG SMTP: Found extension "AUTH", arg "CRAM-MD5 DIGEST-MD5 LOGIN PLAIN" DEBUG SMTP: Found extension "DELIVERBY", arg "" DEBUG SMTP: Found extension "HELP", arg "" DEBUG SMTP: Attempt to authenticate DEBUG SMTP: check mechanisms: LOGIN PLAIN DIGEST-MD5 AUTH LOGIN 334 VXNlcm5hjbt7 YWM0MDkwhi== 334 UGFzc3dvjbt7 YXV0aHNtdHAydog3 235 2.0.0 OK Authenticated DEBUG SMTP: use8bit false MAIL FROM:<[email protected]> 250 2.1.0 <[email protected]>... Sender ok RCPT TO:<[email protected]> 250 2.1.5 <[email protected]>... Recipient ok DEBUG SMTP: Verified Addresses DEBUG SMTP: Jason Thrasher <[email protected]> DATA 354 Enter mail, end with "." on a line by itself From: Webmaster <[email protected]> To: Jason Thrasher <[email protected]> Message-ID: <[email protected]> Subject: non-Spring: Hello World MIME-Version: 1.0 Content-Type: text/plain;charset=UTF-8 Content-Transfer-Encoding: 7bit Hello World: message body here . 250 2.0.0 n5I0ZOkD085654 Message accepted for delivery QUIT 221 2.0.0 mail.authsmtp.com closing connection BAD CASE: The log output when running as a WAR, with missing headers, is quite different: Loading javamail.default.providers from jar:file:/Users/jason/.m2/repository/javax/mail/mail/1.4.2/mail-1.4.2.jar!/META-INF/javamail.default.providers DEBUG: loading new provider protocol=imap, className=com.sun.mail.imap.IMAPStore, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=imaps, className=com.sun.mail.imap.IMAPSSLStore, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=smtp, className=com.sun.mail.smtp.SMTPTransport, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=smtps, className=com.sun.mail.smtp.SMTPSSLTransport, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=pop3, className=com.sun.mail.pop3.POP3Store, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=pop3s, className=com.sun.mail.pop3.POP3SSLStore, vendor=Sun Microsystems, Inc, version=null Loading javamail.default.providers from jar:file:/Users/jason/Documents/dev/subscribeatron/software/trunk/web/struts/target/work/webapp/WEB-INF/lib/mail-1.4.2.jar!/META-INF/javamail.default.providers DEBUG: loading new provider protocol=imap, className=com.sun.mail.imap.IMAPStore, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=imaps, className=com.sun.mail.imap.IMAPSSLStore, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=smtp, className=com.sun.mail.smtp.SMTPTransport, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=smtps, className=com.sun.mail.smtp.SMTPSSLTransport, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=pop3, className=com.sun.mail.pop3.POP3Store, vendor=Sun Microsystems, Inc, version=null DEBUG: loading new provider protocol=pop3s, className=com.sun.mail.pop3.POP3SSLStore, vendor=Sun Microsystems, Inc, version=null DEBUG: getProvider() returning provider protocol=smtp; type=javax.mail.Provider$Type@98203f; class=com.sun.mail.smtp.SMTPTransport; vendor=Sun Microsystems, Inc DEBUG SMTP: useEhlo true, useAuth false DEBUG SMTP: trying to connect to host "mail.authsmtp.com", port 465, isSSL false 220 mail.authsmtp.com ESMTP Sendmail 8.14.2/8.14.2/Kp; Thu, 18 Jun 2009 01:51:46 +0100 (BST) DEBUG SMTP: connected to host "mail.authsmtp.com", port: 465 EHLO jmac.local 250-mail.authsmtp.com Hello sul-pubs-3a.Stanford.EDU [171.66.201.2], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 52428800 250-AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN 250-DELIVERBY 250 HELP DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg "" DEBUG SMTP: Found extension "PIPELINING", arg "" DEBUG SMTP: Found extension "8BITMIME", arg "" DEBUG SMTP: Found extension "SIZE", arg "52428800" DEBUG SMTP: Found extension "AUTH", arg "CRAM-MD5 DIGEST-MD5 LOGIN PLAIN" DEBUG SMTP: Found extension "DELIVERBY", arg "" DEBUG SMTP: Found extension "HELP", arg "" DEBUG SMTP: Attempt to authenticate DEBUG SMTP: check mechanisms: LOGIN PLAIN DIGEST-MD5 AUTH LOGIN 334 VXNlcm5hjbt7 YWM0MDkwhi== 334 UGFzc3dvjbt7 YXV0aHNtdHAydog3 235 2.0.0 OK Authenticated DEBUG SMTP: use8bit false MAIL FROM:<[email protected]> 250 2.1.0 <[email protected]>... Sender ok RCPT TO:<[email protected]> 250 2.1.5 <[email protected]>... Recipient ok DEBUG SMTP: Verified Addresses DEBUG SMTP: Jason Thrasher <[email protected]> DATA 354 Enter mail, end with "." on a line by itself Hello World: message body here . 250 2.0.0 n5I0pkSc090137 Message accepted for delivery QUIT 221 2.0.0 mail.authsmtp.com closing connection Here's the actual mail.jsp that I'm testing war/non-war with. <%@page import="java.util.*"%> <%@page import="javax.mail.internet.*"%> <%@page import="javax.mail.*"%> <% InternetAddress from = new InternetAddress("[email protected]", "Webmaster"); InternetAddress to = new InternetAddress("[email protected]", "Jason Thrasher"); String subject = "non-Spring: Hello World"; String content = "Hello World: message body here"; final Properties props = new Properties(); props.setProperty("mail.transport.protocol", "smtp"); props.setProperty("mail.host", "mail.authsmtp.com"); props.setProperty("mail.port", "465"); props.setProperty("mail.username", "myusername"); props.setProperty("mail.password", "secret"); props.setProperty("mail.debug", "true"); props.setProperty("mail.smtp.auth", "true"); props.setProperty("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory"); props.setProperty("mail.smtp.socketFactory.fallback", "false"); Session mailSession = Session.getDefaultInstance(props); Message message = new MimeMessage(mailSession); message.setFrom(from); message.setRecipient(Message.RecipientType.TO, to); message.setSubject(subject); message.setContent(content, "text/plain;charset=UTF-8"); Transport trans = mailSession.getTransport(); trans.connect(props.getProperty("mail.host"), Integer .parseInt(props.getProperty("mail.port")), props .getProperty("mail.username"), props .getProperty("mail.password")); trans.sendMessage(message, message .getRecipients(Message.RecipientType.TO)); trans.close(); %> email was sent SOLUTION: Yes, the problem was transitive dependencies of Apache CXF 2. I had to exclude geronimo-javamail_1.4_spec from the build, and just rely on javax's mail-1.4.jar. <dependency> <groupId>org.apache.cxf</groupId> <artifactId>cxf-rt-frontend-jaxws</artifactId> <version>2.2.6</version> <exclusions> <exclusion> <groupId>org.apache.geronimo.specs</groupId> <artifactId>geronimo-javamail_1.4_spec</artifactId> </exclusion> </exclusions> </dependency> Thanks for all of the answers.

Developer IT