Program for remove exact duplicate files while caching search results

Posted by John Thomas on Super User See other posts from Super User or by John Thomas
Published on 2012-07-09T09:14:05Z Indexed on 2012/07/09 9:17 UTC
Read the original article Hit count: 300

We need a Windows 7 program to remove/check the duplicates but our situation is somewhat different than the standard one for which there are enough programs.

We have a fairly large static archive (collection) of photos spread on several disks. Let's call them Disk A..M. We have also some disks (let's call them Disk 1..9) which contain some duplicates which are to be found on disks A..M.

We want to add to our collection new disks (N, O, P... aso.) which will contain the photos from disks 1..9 but, of course, we don't want to have any photos two (or more) times.

Of course, theoretically, the task can be solved with a regular file duplicate remover but the time needed will be very big.

Ideally, AFAIS now, the real solution would be a program which will scan the disks A..M, store the file sizes/hashes of the photos in an indexed database/file(s) and will check the new disks (1..9) against this database.

However I have hard time to find such a program (if exists).

Other things to note:

  • we consider that the Disks A..M (the collection) doesn't have any duplicates on them
  • the file names might be changed
  • we aren't interested in approximated (fuzzy) comparison which can be found in some photo comparing programs. We hunt for exact duplicate files.
  • we aren't afraid of command line. :-)
  • we need to work on Win7/XP
  • we prefer (of course) to be freeware

TIA for any suggestions,

John Th.

© Super User or respective owner

Related posts about Windows

Related posts about file