Validating lots of checksums in Linux automatically.

[lost_soul]

Let's say I have a remote server with a lot of files on it. They all reside in ONE directory, and this directory also includes an md5sums.txt file that contains checksums for every file in the directory. Let's call this file remotesums.txt.

 

Then I have a local system with SOME of the files which exist on the remote system. It is a small amount, yet still too many to verify by hand. I can write the checksums of these to a file, let's call it localsums.txt.

 

So, every line that is in localsums.txt should ALSO be in remotesums.txt, but remotesums.txt will have lots of lines that do not exist in localsums.txt because I only have some of the files and thus some of the checksums. They may also be in a different order. How do I verify this that what is in localsums.txt is correct?

 

I was thinking of something like:

 

for $line_of_text in localsums.txt

do grep $line_of_text remotesums.txt > results.txt

 

This (with correct syntax) should produce a file called results.txt which should match localsums.txt if all file checksums match up, right? Is there a better way to do this? You could also use a line counting utility to make sure all of them were found in remotesums.txt.

Edited June 4, 2012 by lost_soul

[Tels]

Use sort and diff:

 

md5sum -b * | sort >remotesum

 

md5sum -b * | sort >localsum

 

diff -u localsum remotesum

 

Also helpful might be md5sums "-c" option.

[lost_soul]

... and another way. http://unixhelp.ed.a...GI/man-cgi?comm

 

suppress all lines that are unique to remotesums.txt, dump the rest to a file called check.txt, and then run

diff check.txt localsums.txt

 

EDIT: This got me the results I was looking for. The only additional step I had to do was use sed to remove tabs that were inserted when I ran comm on the input text files. I also had to run sort on the text files before passing them to comm.

Edited June 7, 2012 by lost_soul