Gathering IP's from a complicated log

Posted by Harry on Server Fault See other posts from Server Fault or by Harry
Published on 2012-06-06T01:33:17Z Indexed on 2012/06/06 4:42 UTC
Read the original article Hit count: 366

Filed under:

logging

|

grep

|

sed

|

awk

I have a question regarding the use of some more advanced grep, awk, sed. I have a log file, for a proprietary MTA, that contains IP's in a string, delimited by [redacted]^~x.x.x.x^[redacted]. So far all of my grepping, awking, and sedding hasn't gotten me very far.

This log file has 331520 lines in it. My goal was to simply grep out the ip's, then do a for loop with sed, to sed 's/$i/redacted'. I'm including a sample of one of the log entries. If you all have any idea, I would be greatly appreciative.

Jun  4 15:21:52 host.name mta-name: 13388^~88/CC-04671-FCA0DCF4^~D^~<redactedmessageid>^~@^[email protected]^~redacted.hostname^~000.00.000.000^~port^~esmtp^~^~external_routing_nobounce^~0^~0.51^~subjectofmessage^~250 2.6.0 <redactedmessageid> [InternalId=2178458] Queued mail for delivery

© Server Fault or respective owner

Related posts about logging

log file is not getting created using JDK logging with Commons-logging

as seen on Stack Overflow - Search for 'Stack Overflow'
When I run the TestJcLLoggingService class log messages are coming to Console but no log file is created, please help me if you know the answer. two source files are pasted below. TestJcLLoggingService.java package com.amadeus.psp.pasd.logging; import org.apache.commons.logging.Log; import org… >>> More
Python: combine logging and wx so that logging stream is redirectet to stdout/stderr frame

as seen on Stack Overflow - Search for 'Stack Overflow'
Here's the thing: I'm trying to combine the logging module with wx.App()'s redirect feature. My intention is to log to a file AND to stderr. But I want stderr/stdout redirected to a separate frame as is the feature of wx.App. My test code: import logging import wx class MyFrame(wx.Frame): … >>> More
Python logging before you run logging.basicConfig?

as seen on Stack Overflow - Search for 'Stack Overflow'
It appears that if you invoke logging.info() BEFORE you run logging.basicConfig, the logging.basicConfig call doesn't have any effect. In fact, no logging occurs. Where is this behavior documented? I don't really understand. >>> More
Redirect logging output using custom logging handler

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, I'm using a module in my python app that writes a lot a of messages using the logging module. Initially I was using this in a console application and it was pretty easy to get the logging output to display on the console using a console handler. Now I've developed a GUI version of my app… >>> More
Java Logger API

as seen on Oracle Blogs - Search for 'Oracle Blogs'
This is a more like a tip rather than technical write up and serves as a quick intro for newbies. The logger API helps to diagnose application level or JDK level issues at runtime. There are 7 levels which decide the detailing in logging (SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST)… >>> More

Related posts about grep

grep is inconsistently defaulting to grep -P?

as seen on Server Fault - Search for 'Server Fault'
I have a script that does some housekeeping that works perfectly well when invoked from an interactive shell, but did nothing when invoked by cron. To troubleshoot this I started a shell with a 'blank' environment with the command: env -i /bin/bash --noprofile --norc Using this blank env I've dug… >>> More
grep pattern interpretted differently in 2 different systems with same grep version

as seen on Server Fault - Search for 'Server Fault'
We manufacture a linux appliance for data centers, and all are running fedora installed from the same kickstart process. There are different hardware versions, some with IDE hard drives and some SCSI, so the filesystems may be at /dev/sdaN or /dev/hdaN. We have a web interface into these appliances… >>> More
grep --exclude/--include syntax (do not grep through certain files)

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm looking for the string "foo=" (without quotes) in text files in a directory tree. It's on a common Linux machine, I have bash shell: grep -ircl "foo=" * In the directories are also many binary files which match "foo=". As these results are not relevant and slow down the search, I want grep… >>> More
Grep failing with Emacs (windows), and GnuWin32 Grep

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I've downloaded and installed the GnuWin32 tools, and added the grep executables to the Emacs bin. I've also, for what its worth, added the GnuWin32 bin folder to my Path variable. Problem is though, when I try and run with suggested grep commands, I always get: Grep exited abnormally with… >>> More
How to grep a line start with "*" using grep

as seen on Super User - Search for 'Super User'
Hi, How can I use 'grep' to get lines start with '* ' in my file? I tried grep "" myfile I tried grep " " myfile but returns all the lines of my file. Thank you. >>> More