Copying multiple files but want to avoid duplicates

Topic

New Reply

I have an old music server (about 1 TB) with FLAC files. The company (Olive) is now out of business. I would like to copy all the files which I can access over my LAN to my NAS . However, errors can cause the transfer (copy) to crash or interrupt. Next copy attempt ends up with additional copies.

Any suggestions for copying files to my NAS without making duplicates? Have tried searching for solutions (Google, etc) without finding good alternative. Inexpensive alternative acceptable, don’t need freeware although nice.
Thanks

Reply | Quote

Replies

I’ve used TeraCopy, in More mode, Overwrite All, has served me well. Or, you can tell it to skip any file with same name; although for the copying you want – I recommend Overwrite All and More mode.

"Take care of thy backups and thy restores shall take care of thee." Ben Franklin, revisted

Reply | Quote

I’ve used TeraCopy, in More mode, Overwrite All, has served me well. Or, you can tell it to skip any file with same name; although for the copying you want – I recommend Overwrite All and More mode.

How do I enter the whole source folder which is on a different disc into Teracopy? I can search for and find the target folder on the NAS to copy to.
The instruction are fairly sketchy.
Thanks

Reply | Quote

I use Robocopy (as do others on the Lounge). I use the following code to copy files to a backup destination. It won’t copy files that haven’t changed. The /R and /W parameters will allow retries and waits. See this website for all the parameters.

Code:

robocopy E:source \nasfolder /V /TEE /MIR /NP /XJD /R:3 /W:1 /log+:E:logsPictures.txt /XD E:folderssubfolder

Options:
/V – Produce Verbose output, showing skipped files.
/TEE – Output to console window, as well as the log file.
/E – Copy Subdirectories, including empty ones.
/MIR – MIRror a directory tree (equivalent to /E plus /PURGE).
/XD – eXclude Directories matching given names/paths.

Eliminate spare time: start programming PowerShell

Reply | Quote

I was able to get to get Teracopy to work perfectly after setting as default copy and rebooting computer. I will look into Robocopy as well. It is great to have this as a resource of ideas, so many out there if you know where to look.
Thanks.

I use Robocopy (as do others on the Lounge). I use the following code to copy files to a backup destination. It won’t copy files that haven’t changed. The /R and /W parameters will allow retries and waits. See this website for all the parameters.

Code:

robocopy E:source \nasfolder /V /TEE /MIR /NP /XJD /R:3 /W:1 /log+:E:logsPictures.txt /XD E:folderssubfolder

Options:
/V – Produce Verbose output, showing skipped files.
/TEE – Output to console window, as well as the log file.
/E – Copy Subdirectories, including empty ones.
/MIR – MIRror a directory tree (equivalent to /E plus /PURGE).
/XD – eXclude Directories matching given names/paths.

Reply | Quote

The RoboCopy suggestion is a good one. I’d make one small alteration though, the suggested /XJD switch only excludes Junction Directories. This is a good idea (to prevent recursive copies) but incomplete.

There is such a thing as a Junction File as well, and the switch to exclude those is /XJF. However there’s a better option yet again. Just use /XJ and you exclude both Junction Files and Junction Directories.

Reply | Quote

“Can enter the target file but can’t get it to recognize the source folder on another disc”
I don’t know the answer to that; maybe RoboCopy might be better in this case.

"Take care of thy backups and thy restores shall take care of thee." Ben Franklin, revisted

Reply | Quote

+1 for Robocopy, especially the Wait and Retry options.

cheers, Paul

Reply | Quote

Thanks Mr Harder. While I was looking up the parameters (to let the OP know what they meant), I realised my original list had some which were contradictory or redundant, so that helped me clean the list up a bit. There are just so many!

Eliminate spare time: start programming PowerShell

Reply | Quote

RoboCopy is one of those rare programs with almost too much power, too many options!

🙂

There’s another neat switch that I like a lot, it’s /XO. This is wonderful when you expect the destination to already have a lot of your data, but you aren’t completely sure about the versioning of all your files. /XO will decline to copy an older file from the source location, over a newer file in the destination.

Reply | Quote

I’m a sucker for GUI and transparency and being able to see what’s there and what’s going on, complete with filters and options, and I’m willing to pay for it. At the command line, I’m flying blind. I suggest you give GoodSync a try.

Reply | Quote

Of course, you could just use Windows Explorer and, when it eventually tells you it’s found duplicates, you could simply tell it to skip them all.

Or you could use Windows’ own xcopy utility…

Neither of these entails installing any 3rd-party software.

Cheers,
Paul Edstein
[Fmr MS MVP - Word]

Reply | Quote

Neither does using Robocopy – xcopy on steroids.

cheers, Paul

Reply | Quote

Something one could try before copying is to use CCleaner’s duplicate file finder to clean the source Folder. I usually create a new Folder or subFolder on an External USB drive or a NAS drive to copy the source files to then sort them out later, tends to be a lot quicker than sorting while copying since the copied files are then on the same drive as the destination Folder.

Before you wonder "Am I doing things right," ask "Am I doing the right things?"

Reply | Quote

I suggested you give GoodSync a try. I didn’t say ‘buy’.

The ‘trial’ appears to be next to nothing until you realize that you can use it to monitor the state of your source and destination files and folders, complete with date and time and the direction you may want transfers to take place. Just perform the scan. Once you have examined it, you can go back to your command line utilities and perform the operations themselves. There is no time limitation on the trial itself, so you can do this as many times as you like.

You can also tell Microsoft that Windows was a huge mistake and a terrible investment, because doing everything at the command line is so much better.

Reply | Quote

One caveat with the latest Win10 Insider Preview, the Command Prompt and Command Prompt (Admin) on the right-click of the Start button have been replaced with the equivalent Windows PowerShell and Windows PowerShell (Admin). [Omen of the future?]

Before you wonder "Am I doing things right," ask "Am I doing the right things?"

Reply | Quote

One caveat with the latest Win10 Insider Preview, the Command Prompt and Command Prompt (Admin) on the right-click of the Start button have been replaced with the equivalent Windows PowerShell and Windows PowerShell (Admin). [Omen of the future?]

Note that you can get the Command Prompt back on the right click menu via
Settings > Personalization > Taskbar > Turn off “Replace Command Prompt with Windows Powershell in the Menu when I right Click the Start Button or press Windows Key + X”

Jerry

Reply | Quote

I’ve been using FreeFileSync for several years to synchronize similar but not identical directories. It has an easy to use interface and lets you sych based on filename and date/time, or full content of files (slower that way) and doesn’t replace those that match. There is a checkbox during the install to avoid including an added junkware program.

Reply | Quote

I have an old music server (about 1 TB) with FLAC files. The company (Olive) is now out of business. I would like to copy all the files which I can access over my LAN to my NAS . However, errors can cause the transfer (copy) to crash or interrupt. Next copy attempt ends up with additional copies.

Any suggestions for copying files to my NAS without making duplicates? Have tried searching for solutions (Google, etc) without finding good alternative. Inexpensive alternative acceptable, don’t need freeware although nice.
Thanks

Open a command window and type

XCOPY /?

To see the options. I’ve never had any trouble with XCOPY.

Reply | Quote

Robocopy is better than xcopy. 🙂

cheers, Paul

Reply | Quote