awk - Remove line if field is duplicate

Posted by Kyle on Stack Overflow See other posts from Stack Overflow or by Kyle
Published on 2010-04-08T23:14:23Z Indexed on 2010/04/08 23:23 UTC
Read the original article Hit count: 316

Filed under:

awk

|

sed

Looking for an awk (or sed) one-liner to remove lines from the output if the first field matches.

An example for removing duplicate lines I've seen is:

awk 'a !~ $0; {a=$0}'

Tried using it for a basis with no luck (I thought changing the $0's to $1's would do the trick, but didn't seem to work).

© Stack Overflow or respective owner

Related posts about awk

problem with awk script

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, when I call my awk script, I keep getting an error : sam@sam-laptop:~/shell/td4$ awk -f agenda.awk -- -n Robert agenda.txt awk: agenda.awk:6: printf "Hello" awk: agenda.awk:6: ^ syntax error the script contains this : #!/usr/bin/awk BEGIN { } printf "Hello" END { } Thank you >>> More
Awk to grab colo(u)r codes from CSS files aka School me in Awk

as seen on Stack Overflow - Search for 'Stack Overflow'
Nice and (hopefully) easy. I am trying to work out how to grab the variable #XXX from a text file (css file) containing strings like hr { margin: 18px 0 17px; border-color: #ccc; } h1 a:hover, h2 a:hover, h3 a:hover { color: #001100; } Which I would like to return as ccc 777 The… >>> More
Parsing the output of "uptime" with bash

as seen on Super User - Search for 'Super User'
I would like to save the output of the uptime command into a csv file in a Bash script. Since the uptime command has different output formats based on the time since the last reboot I came up with a pretty heavy solution based on case, but there is surely a more elegant way of doing this. uptime… >>> More
AWK scripting :How to remove Field separator using awk

as seen on Stack Overflow - Search for 'Stack Overflow'
Need the following output ONGC044 ONGC043 ONGC042 ONGC041 ONGC046 ONGC047 from this input Medium Label Medium ID Free Blocks =============================================================================== [ONGC044] ECCPRDDB_FS_43 ac100076:4aed9b39:44f0:0001… >>> More
sort associative array in awk - help?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all, I have an associative array in awk that gets populated like this... chr_count[$3]++ When I try to print my chr_counts I use this: for (i in chr_count) { print i,":",chr_count[i]; } But not surprisingly, the order of i is not sorted in any way. Is there an easy way to iterate over… >>> More

Related posts about sed

sed + RE , want to ignore remarked line using sed

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to replace the a string with the word string as the following example down , but if a string exist also after comment then a string will replaced too In which way I can add to the sed command the irregular exp that ignore from the #. [root@localhost tmp]# more test a b aa bb #a #b #aa #bb [root@localhost… >>> More
script to search and replace deprecated functions

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am using the following script to search and replace the deprecated functions in a file with the newer ones. 5 for strFile in `ls deprecated_functions_search_and_replace.txt ` 6 do 7 sed "s/ereg_replace[^\(]*(\([^,]*\),/preg_replace\1('#'.\2.'#',/g" $strFile > temp_file 8 … >>> More
Footer not stretching 100% when horizontally scrolled

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I have a footer which is set to 100% width, but if i size the window smaller so a horizontal scrollbar appears, using the scrollbar shows whitespace to the right of the footer ... its not spanned 100% of the page, just the viewport. <!doctype html> <html lang="en" class="no-js"> <head> … >>> More
What does this regex mean and why

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
$ sed "s/\(^[a-z,0-9]*\)\(.*\)\( [a-z,0-9]*$\)/\1\2 \1/g" desired_file_name I apreciate it even if you only explain part of it or at lest structure it with words as in s\alphanumerical_at_start\something\alphanumerical_at_end\something_else\global Could someone explain what that means, why and… >>> More
Sed problem in a Bash script

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Hello there. I'm having a problem using the sed command . I'm trying to write a bash script that does the following : search for the line that contain :@ then save the line that contained :@ and replace it with new line as in the following: #! /bin/bash echo "Please enter the ip address… >>> More