Merging and sorting multiple files with "sort"
- by NewbiZ
Hello,
I have a bunch of text logfiles in the following format:
ID (17 characters)
Timestamp (14 characters YYYYmmddHHMMSS e.g. "20060210100040" -> 2006/02/10 10:00:40)
Random data (? characters)
end of line
The files are already sorted by timestamp.
I need to get 1 log file with all the logs from multiple logs files, sorted by timestamp. Note that the log files are really huge, around 3-4G each (and there are dozens of them)
I tried the following command:
sort -s -m -t '|' -k1n,1n +17 -o data_sort.txt *.TXT
Here is how I ended up with this command:
-s : don't bother with tie results
-m : merge all logs files
-t '|' : there is no | in my logs, so the whole line should be field 1
-k1n,1n: sort on the first field as a numeric value
+17 : the timestamp starts at index 17
-o : output file
Actually... it fails miserably. The output file data_sort.txt is just the concatenation of all files, not sorted at all :(
I would greatly appreciate if anyone could provide any help on this problem!
Thanks
