Register
It is currently Thu Dec 18, 2014 10:37 am

Help me with a file comparison script


All times are UTC - 6 hours


Post new topic Reply to topic  [ 3 posts ] 
Author Message
 PostPosted: Mon Nov 02, 2009 10:01 pm   

Joined: Mon Nov 02, 2009 9:52 pm
Posts: 1
this is probably a fairly easy task, but my scripting skills aren't up to par.

I need to take one plaintext file, and throw away any line that appears identically in any one of a series of plain text files.

The trick is that the lines will not be in the same order.

For example, an input file with the data
Code:
1
2
3
4
5
6


and two test files,
Code:
6
2
4

Code:
7
5


should return
Code:
1
3


I figure that the easiest way to write this is with one script that compares two files, then a second script to pass the output to the comparison with the next file. However, my attempts at the first piece have failed.

Everywhere I've turned shows me ways to compare two files line-by-line, which is NOT what i'm after here.

Thank you for your assistance.


Top
 Profile  
 PostPosted: Tue Nov 03, 2009 2:35 am   

Joined: Mon Nov 17, 2008 7:25 am
Posts: 221
Code:
# sort file1.txt > new_file1.txt
# sort file2.txt > new_file2.txt
# diff new_file1.txt new_file2.txt


This will work if the lines aren't supposed to be in any particular order :)
Otherwise I believe you need to tell us more about the structure of the file.

You can also use comm(1) to compare files, it might be somewhat easier to read :)

Best regards
Fredrik Eriksson


Top
 Profile  
 PostPosted: Tue Nov 03, 2009 4:14 am   
Moderator
User avatar

Joined: Thu Oct 11, 2007 7:12 am
Posts: 229
Location: London - UK
might fit your situation better;

Code:
~]# cat temp/input
1
2
3
4
5
6
~]# cat temp/first_match_file
6
2
4
~]# cat temp/second_match_file
7
5
~]# cat temp/first_match_file temp/second_match_file > temp/combined_match_file
~]# grep -v -f temp/combined_match_file temp/input
grep -v -f temp/third temp/input
1
3


* edited to put in code tags


Top
 Profile  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron


BashScripts | Promote Your Page Too
Powered by phpBB © 2011 phpBB Group
© 2003 - 2011 USA LINUX USERS GROUP