Backup to network location when available/attached

Topic

New Reply

Hi. Can anyone here suggest a backup program that:

[*]can backup (not just file sync) to a network drive or FTP location
[*]wait silently if/when the target location is not available/accessible, until the location is available to perform the backup, without popping up error messages (or prepare the backup files at a temporary location and then transfer them when the location is available)?
[*]is suitable for a home networked environment (I’m happy to pay money for software licences, but don’t want to buy more hardware or convert my desktop computer to Windows Server)

Scenario: My wife is a part-time teacher, and has a personal laptop that she uses for both professional and personal purposes, but at home she might only use it 1-3 times per week not every day. We have a NAS at home (QNAP, lower end of their range) that is on 24×7 and that we use for backups of our desktop computer. I’d like to set her laptop up to do regular automatic regular backups when she’s at home (and hence can access the NAS), but I don’t want it popping up error messages if she’s using the laptop at school where our NAS is unreachable.

If it has to be manually triggered when the laptop is at home, then it’s unlikely to be done regularly or consistently.
Because the NAS is generally on 24×7 at home, so it should be ok to use “we’ve connected to the home network” as part of the trigger. Her work doesn’t offer any backup/recovery or file storage.

I’ve found one or two file-synchronisation tools that look like they’d do the job. But for a few reasons I really want this to be a proper backup solution, not just file sync. It occurred to me that I could do file-sync to an area on our desktop computer and then trigger a backup from there, but that seems inefficient.

Can anyone offer any ideas for suitable solutions? Thanks!

(If it makes any difference: wife’s laptop is Win8.1 but will move to Win10 eventually, has separate accounts for admin/work/personal use. Desktop computer is Win7. NAS is a QNAP TS-212E with network file shares and FTP capability. Home network is usual mix of gigabit ethernet and wireless.)

Reply | Quote

Replies

EDIT

I thought this was going to work for you but I can not see how to detect a network path w/o out a script 🙁

42215-Untitled

Check out task scheduler. 😀

🍻

Just because you don't know where you are going doesn't mean any road will get you there.

Reply | Quote

Schedule the backup program to run every hour with the destination set to the network path. If the path is not available the software will log the error.
You will probably have to purchase a backup program to get the scheduler.

cheers, Paul

Reply | Quote

I don’t normally recommend backup schemes that include unlimited retries on errors or unavailable resources. Bad things tend to happen when you use such tools.

The problem comes about when you have a job. Think of it as a logical unit of work, “One Backup”. If you automate this with a scheduler, you may run this perhaps once per day. OK, but let’s suppose your NAS is offline for a week. Your job scheduler has launched 7 jobs and they are all sitting there, waiting for the NAS to come online. If it comes online after a week of downtime, one of 2 things will happen. Either:

1). All 7 pending jobs run in parallel. At best this overwhelms your systems with massive amounts of I/O. At worst the jobs may write to the exact same folder locations and you may well get file locks, overwrites and lots of errors or bizarre behaviour;
2). All 7 pending jobs run in sequence. The massive I/O still happens but it happens sequentially. You will probably find that the whole string of jobs takes hours longer than it ought to. At best you notice the problem and kill all but one of the jobs. Worst case scenario, it takes so long that the next day’s backup job begins before the last of the pending jobs completes. Or you fill your NAS with redundant backups.

This is not to say that retries are bad, in fact I very much like a system that won’t halt on the first error it encounters. However keep it reasonable!

As for tools, I rather like RoboCopy. It’s very powerful. The one thing you have to watch out with RoboCopy is that by default, it retries 1 million times. IMO this is completely ridiculous under most use cases.

It’s easy to correct this behaviour though:

RoboCopy C:SomeStuff Q:Destination *.* /e /r:10

That gives you 10 retries on any failure, then RoboCopy continues on to the next object to be backed up (it does NOT halt the entire backup job).

Reply | Quote

Any backup program worth its salt will fail the backup, log an error and close, not keep running just in case.

cheers, Paul

Reply | Quote

I’ve found one or two file-synchronisation tools that look like they’d do the job. But for a few reasons I really want this to be a proper backup solution, not just file sync. It occurred to me that I could do file-sync to an area on our desktop computer and then trigger a backup from there, but that seems inefficient.

Actually, this sounds like a good way to do it:

[*]The backup would get done.
[*]You would have an additional copy of her data on the desktop.
[*]Her laptop wouldn’t be tied up while the backup is occurring, because the desktop copy of her data is what would be backed up, not the data on her laptop.

Group "L" (Linux Mint)
with Windows 10 running in a remote session on my file server

Reply | Quote

Backups haven’t tied up the machine since MS built VSS into Windows – about 10 years ago.

cheers, Paul

Reply | Quote

Backups haven’t tied up the machine since MS built VSS into Windows – about 10 years ago.

cheers, Paul

I know about VSS, because for the past 9 years my job has involved Legato Networker tape backups! But for some reason that never occurred to me.

Are the VSS snapshots continually available, or are they made on-demand? And if on-demand, how long would it take for the snapshot to be made?

Group "L" (Linux Mint)
with Windows 10 running in a remote session on my file server

Reply | Quote

They are on demand and time depends on what you are snapping, but it doesn’t affect PC use from my experience.

I haven’t used Legato for a long time, probably because I haven’t had a say in what backup app is purchased.

cheers, Paul

Reply | Quote

One additional point. Errors in a network backup can dramatically increase the usual time it takes to perform a backup. If the same logical problem causes each file to fail on the same error message, and your backup consists of thousands of files, and you have a system setup to retry each file multiple times, and there also is a pause between each retry…

Your backup is going to take forever, as in you’ll probably have to terminate the job.

The Windows Job Scheduler has an option to kill a job if it takes more than a defined total amount of time. This is a good way to handle this situation.

Reply | Quote

I use Bvckup 2 (http://www.bvckup2.com). Just $19.95 for personal use and I think it will do everything you want – and discreetly and seamlessly. At least, it does for me. Cheers!

Reply | Quote

Schedule the backup program to run every hour with the destination set to the network path. If the path is not available the software will log the error.
You will probably have to purchase a backup program to get the scheduler.

cheers, Paul

That’s interesting thinking. Certainly Task Scheduler can be set to also require being attached to a particular network (or a network in general, although this doesn’t apply to my scenario) as a condition for running the job. I’ll need to experiment with the appropriate settings to ensure that a missed or failed backup is retried every hour (assuming not still in progress!) but once a backup is successful, it waits for the desired default type/frequency (full/differential/incremental, daily/weekly/whatever) not for another hour in the future – otherwise I’m getting the disadvantages of backup combined with the disadvantages of sync. 😀 If anyone has already solved this problem, would be glad to see it – otherwise will post a follow-up if I’m successful.

If it works, it also means I can just purchase another licence of the backup software I’m already using (Backup4All, for the record) on my desktop computer.

Actually, this sounds like a good way to do it:

[*]The backup would get done.
[*]You would have an additional copy of her data on the desktop.
[*]Her laptop wouldn’t be tied up while the backup is occurring, because the desktop copy of her data is what would be backed up, not the data on her laptop.

Yes, but it introduces another dependency/point of failure – doesn’t work if the desktop computer is off/unavailable, and doubles the amount of network usage. Sort of defeats the purpose of having a NAS.

I use Bvckup 2 (http://www.bvckup2.com). Just $19.95 for personal use and I think it will do everything you want – and discreetly and seamlessly. At least, it does for me. Cheers!

I had a look at it, as far as I can tell it doesn’t meet either of my key criteria. It detects attached storage (as do many other similar programs, some free) but not network devices, and it only does file mirroring, not backup.

Still, grateful for any other suggestions.

Reply | Quote

I use Bvckup 2

I had a look at it, as far as I can tell it doesn’t meet either of my key criteria. It detects attached storage (as do many other similar programs, some free) but not network devices, and it only does file mirroring, not backup.[/QUOTE]

What sort of an amateur job it would’ve been if it couldn’t wait for a network device? 🙂
Of course, it can do that – http://i.imgur.com/M1emYy1.png

With regards to the mirroring vs. backup – this depends on what exact type of backup your are after, but the app does indeed default to mirroring. It can however be set up to archive past copies if needed.

* My apologies for barging in like this, but I just happen to be uniquely qualified to comment on this app, because I wrote it.

Reply | Quote

The next schedule will re-run the same job, but if the job is an incremental backup it won’t do much.

cheers, Paul

Reply | Quote

We use Macrium Reflect free version. This page of their user guide makes interesting reading:
http://knowledgebase.macrium.com/display/KNOW/Alternative+Locations+For+Backups

Here’s a feature comparison chart (scroll halfway down the page):
http://www.macrium.com/reflectfree.aspx

Reply | Quote

And now a follow-up post with findings…

I’ll need to experiment with the appropriate settings to ensure that a missed or failed backup is retried every hour (assuming not still in progress!) but once a backup is successful, it waits for the desired default type/frequency (full/differential/incremental, daily/weekly/whatever) not for another hour in the future – otherwise I’m getting the disadvantages of backup combined with the disadvantages of sync. 😀 If anyone has already solved this problem, would be glad to see it – otherwise will post a follow-up if I’m successful.[/quote]

I didn’t manage to work out the settings to do this as well as I’d hoped. Hampered slightly by some busyness and trial period of software expiring.

I recommend checking out CrashPlan (http://crashplan.com/). Although CrashPlan offers a paid cloud storage service, they make it a point of encouraging users to also make use of local backups. The backup software can be used for free and only requires registering a name and email address to receives status alerts for backups.

The backup software is completely silent so there are no pop-ups or other distractions. The default setting sends an email alert whenever no backup was successful for 3 days and a status report once every 3 weeks.

This seemed to be the winner for me. Crashplan reports limited support for use on a NAS, but they do report some generic and brand-specific methods to try. In the case of QNAP, the Crashplan section of the QNAP forum is a good place to start, there is a package that has been tweaked for installation on QNAP while working with the Windows graphical client (I use shortcuts to a couple of batch files to let me control whether QNAP client on my Win7 desktop works locally or connects to NAS).

If you’re only interested in backing up files/folders, why not use Windows File History? It’s built in to both Win 8.1 & Win 10. And if the destination isn’t available it will automatically wait till it is then start backing up. Have you looked into that?

I hadn’t, because I’d forgotten about that feature (Win8.1 is only on my wife’s machine, not mine). But built-in backup in Windows has a pretty lousy history of lack of forward/backward compatibility. In this case if something went wrong with the Win8.1 machine then I wouldn’t be able to restore any files unless I find another Win8.1 machine. (Or maybe a Win10 machine, but I don’t have one to test with.) That’s a show-stopper for me – backup is pointless if you can’t be confident you can restore it.

What sort of an amateur job it would’ve been if it couldn’t wait for a network device? 🙂
Of course, it can do that – http://i.imgur.com/M1emYy1.png

With regards to the mirroring vs. backup – this depends on what exact type of backup your are after, but the app does indeed default to mirroring. It can however be set up to archive past copies if needed.

It certainly wasn’t clear from the documentation how to set this up. While I agree that your program looks fantastic for mirroring and endorse your philosophy of doing one thing really well, I’m trying to do something different so a solution that does that different thing well is more appropriate to my needs.

Reply | Quote

I hadn’t, because I’d forgotten about that feature (Win8.1 is only on my wife’s machine, not mine). But built-in backup in Windows has a pretty lousy history of lack of forward/backward compatibility. In this case if something went wrong with the Win8.1 machine then I wouldn’t be able to restore any files unless I find another Win8.1 machine. (Or maybe a Win10 machine, but I don’t have one to test with.) That’s a show-stopper for me – backup is pointless if you can’t be confident you can restore it.

Actually you can restore from File History with any OS that reads the disk it’s on. It will be fine with both 8.1 and 10 PCs. The files are stored as copies of the backed up file, so can be read with Win 7 or before (or Linux etc), the only difficulty being that the filename has a date time stamp appended to it e.g. a file called Shootingdata1.txt will be saved as Shootingdata1 (2015_09_04 19_32_02 UTC).txt. Note that FH is a versioning backup; you can restore from older versions of the file (something that helped me this morning as it happens).

Obviously having the date time stamp isn’t great for just copying the file from the backup disk, but it’s a lot better than nothing and a batch file (or Powershell file) could be used to strip those elements off the filename in a worst case scenario. And in any case, the best backup regime should have multiple copies of backups, and it’s probably better if they are different methods and to different disks. (Note to self, do I do that?)

Eliminate spare time: start programming PowerShell

Reply | Quote

Hi. Can anyone here suggest a backup program that:

[*]can backup (not just file sync) to a network drive or FTP location
[*]wait silently if/when the target location is not available/accessible, until the location is available to perform the backup, without popping up error messages (or prepare the backup files at a temporary location and then transfer them when the location is available)?
[*]is suitable for a home networked environment (I’m happy to pay money for software licences, but don’t want to buy more hardware or convert my desktop computer to Windows Server)

Scenario: My wife is a part-time teacher, and has a personal laptop that she uses for both professional and personal purposes, but at home she might only use it 1-3 times per week not every day. We have a NAS at home (QNAP, lower end of their range) that is on 24×7 and that we use for backups of our desktop computer. I’d like to set her laptop up to do regular automatic regular backups when she’s at home (and hence can access the NAS), but I don’t want it popping up error messages if she’s using the laptop at school where our NAS is unreachable.

If it has to be manually triggered when the laptop is at home, then it’s unlikely to be done regularly or consistently.
Because the NAS is generally on 24×7 at home, so it should be ok to use “we’ve connected to the home network” as part of the trigger. Her work doesn’t offer any backup/recovery or file storage.

I’ve found one or two file-synchronisation tools that look like they’d do the job. But for a few reasons I really want this to be a proper backup solution, not just file sync. It occurred to me that I could do file-sync to an area on our desktop computer and then trigger a backup from there, but that seems inefficient.

Can anyone offer any ideas for suitable solutions? Thanks!

(If it makes any difference: wife’s laptop is Win8.1 but will move to Win10 eventually, has separate accounts for admin/work/personal use. Desktop computer is Win7. NAS is a QNAP TS-212E with network file shares and FTP capability. Home network is usual mix of gigabit ethernet and wireless.)

Hi Paddles,

I recommend checking out CrashPlan (http://crashplan.com/). Although CrashPlan offers a paid cloud storage service, they make it a point of encouraging users to also make use of local backups. The backup software can be used for free and only requires registering a name and email address to receives status alerts for backups.

The backup software is completely silent so there are no pop-ups or other distractions. The default setting sends an email alert whenever no backup was successful for 3 days and a status report once every 3 weeks.

On your wife’s laptop, map a network share from the QNAP and set it to auto-connect, then configure a target folder as a backup destination. There can be multiple destinations so you could backup to the QNAP, a portable USB drive and/or even the desktop computer.

Backups can be optionally encrypted and the default setting keeps multiple versions of changed files.

Chung

Reply | Quote

Hi Paddles,

If you’re only interested in backing up files/folders, why not use Windows File History? It’s built in to both Win 8.1 & Win 10. And if the destination isn’t available it will automatically wait till it is then start backing up. Have you looked into that?

Art

Reply | Quote