Worthwhile to defrag huge files (over 100 GB)?

Topic

New Reply

My backup files (from Easeus and from WIndow 7’s own backup facility) are so large that they never actually get defragged – in the case of Auslogics defrag I simply tell it to ignore those files, but I wonder if it is worthwhile or even desirable to find something that actually might succeed in defragging such files (given that there is plenty of space available on the drive for them to be placed in contiguous blocks).
Opinions?
Thanks
David

Reply | Quote

Replies

Since all that happens with backup files is that they get written once and never read (one hopes!), then you could just leave them alone.

However, if they cause problems for other files…

BATcher

Plethora means a lot to me.

Reply | Quote

Keeping the backup files on their own disk or partition is a good idea. Then “normal” files do not find themselves fragmented across multi GB sections of disk, with obvious performance implications.

cheers, Paul

Reply | Quote

I defrag mine using MyDefrag. Yes they are on a drive all to themselves. Yes it is a large drive with plenty of room. I also like to burn archive copies to DVD, and I don’t like making coasters. I don’t recall ever burning a coaster of a defragged multi-gig file.

I had a house fire in January; my external backup drive did not survive (nor did two PC”s and two laptops), but my DVD’s did.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

Thanks to all.
bbearen – are you saying that, in your experience, a fragmented large file is more likely to cause a DVD to fail (than a defragmented on) if you try to burn that file to it? If that’s the only reason for defragmenting a huge backup file I suspect it wouldn’t be worth it for files over 100 GB, since they would require ages to burn and a large number of DVDs. (?)

Reply | Quote

Thanks to all.
bbearen – are you saying that, in your experience, a fragmented large file is more likely to cause a DVD to fail (than a defragmented on) if you try to burn that file to it? If that’s the only reason for defragmenting a huge backup file I suspect it wouldn’t be worth it for files over 100 GB, since they would require ages to burn and a large number of DVDs. (?)

I guess it all depends on what one calls “worth it”…

I had a house fire in January; my external backup drive did not survive (nor did two PC”s and two laptops), but my DVD’s did.

I lost two PC”s and two laptops, and an external hard drive with several iterations of backups on it. But I didn’t lose any files, photos, music, video, etc. etc. thanks to my DVD’s. A regular on another help fourm has as his signature, “any file that you don’t have at least two backups for is a file that you don’t want to keep.”

BootIt lets me allocate whatever file size I choose. I can choose 2GB or 4GB, etc. and BootIt will parse the backups into the correct number of files, which makes it quite simple to burn to DVD. Yes, it takes several DVD’s, but in my case it is very well worth it.

If I’m doing a backup in preparation to a major update, or a significant software installation, I’ll use a single very large file; it’s not for archive, it’s just for going back in case of a foul-up. That’s what I did before installing Service Pack 1. As it turned out, that update finished without a hitch, so I didn’t really need that backup after I was sure that I had no glitches.

But the backup I did after Service Pack 1 I wanted to save as an archive, so, 4GB segments. And I have never had a problem whatsoever with defragging a BootIt backup file with MyDefrag.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

I guess it all depends on what one calls “worth it”…

I lost two PC”s and two laptops, and an external hard drive with several iterations of backups on it. But I didn’t lose any files, photos, music, video, etc. etc. thanks to my DVD’s.

Agree that multiple backups are always worth it.

In this day and age, however, I doubt that stacks of DVDs are your best “second layer” backup option, both for the time they take to create and for storage space reasons (I’m wondering how bbearren had to store his to survive a fire).

First, external HDDs are just so cheap nowadays that you can buy two for under $100. Set up a sync utility to copy your data files to the external HDD, then swap the HDD out every week or so, with the “off cycle” HDD stored somewhere else (your office, a friend’s house, in a safety deposit box, etc.).

Second, cloud storage is free in amazing quantities. Amazon lets you store up to 5GB for free. That may not be enough for your mp3s and video files, but it sure would allow you to store/sync just about all of your small files “in the cloud” (encrypted, of course, if the files contain confidential info).

The reason most people don’t do backups (let along multiple layers of backups) is that there’s just too much time and effort involved. It’s better to pick a system that’s easy to use, as you’re more likely to keep up with it.

As bbearren rightly points out, though, a backup doesn’t do you any good if it’s been incinerated.

Reply | Quote

David,
I take it your using Windows Backup. Mine are made using Acronis 2010, saved to an external hdd. I defragged them once and that made them unusable, so never did it again.

Reply | Quote

David,
I take it your using Windows Backup. Mine are made using Acronis 2010, saved to an external hdd. I defragged them once and that made them unusable, so never did it again.

If it was a branded drive, some have auto-installing software which could have been at the root of the problem.

Reply | Quote

Have you considered Blu-ray as your backup media? It’s still somewhat pricey ($100 for a drive, $50 for a 50GB disk) but I find it is worth it: I make copies of my data files about twice a year and keep them in my safety deposit box at the bank. That way I only need two physical disks for 100GB rather than bundles of CDs or DVDs. Also note that I keep COPIES rather than images created by backup software. It has happened to me in the past that I tried to recover files from a backup but was unable to do so, as newer versions of the backup software could not read the old images or else the backup software maker had gone out of business and no current versions were available. (I do use imaging software for my systems drive)
Update: Prices for 50GB BD-RE rewritable dual layer blu-ray discs have apparently dropped to about $25 since I last bought some. This technology has now been around for about 5 years and, as it gets more widely adopted, prices are expected to drop even further.

Reply | Quote

$100 for a drive, $50 for a 50GB disk

I’d buy a couple of large SATA disks and an eSata card first. That’s just silly money for new and unproven technology.

cheers, Paul

Reply | Quote

Hi Roderunner
Yes Windows and Easeus. Your experience confirms my suspicions. I think I’ll leave well alone.

Reply | Quote

Hi TheAncient
Last time I asked I was told recordable BluRay disks were not yet reliable enough, but maybe improvements have been made since then….
I’ll look into it anyway, thanks.
David

Reply | Quote

Hmm, has anyone suggested defragging your Windows setup (C: drive) before you create a backup image?

Reply | Quote

Hmm, has anyone suggested defragging your Windows setup (C: drive) before you create a backup image?

Always; that’s my standard practice, cleanup and defrag before imaging.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

One might also invest in a couple of 1 TB passport drives for secure data storage and move them offsite.
If you have just copied data to the drives & filled them, then there would be no need to defragment, ever.

DVDs are highly reliable in my opinion, and I use both +(above). But if your starting from scratch and have 2 or 3 TB worth of files needing backup, your gonna be burning DVDs untill doomsday.

To the original poster; If your laying down new large files to a drive you rarely manipulate data on, defraging should rarely be needed, if at all.
But defragmenting will not be detrimental either. It does not matter if files that are heavily fragmented on a hard drive are burned to DVDs.
Fragmentation does not get translated to “burned DVDs” they would just be reorganized to fill the DVD.

Reply | Quote

Fragmentation does not get translated to “burned DVDs” they would just be reorganized to fill the DVD.

No, that’s not the issue, but severely fragmented files have been known to delay the burn process enough to result in a coaster. It’s not as much of a problem with newer drives and burning software, but not everyone has newer drives and the latest software.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

I only do 3 or 4 sets of DVD archives a year. My frequent backups are to external drives, which are also the source for burning my DVD’s. Using dual-layer DVD’s cuts the number in half.

But an external drive is, after all, a hard disk drive. Failure is not a matter of if, it’s a matter of when. A cheap dead drive is still a dead drive. I feel more comfortable having some backups on static, inert media.

The DVD’s survived because they were stored at floor level in a corner. Some others stored elsewhere didn’t survive.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

DVD and Cloud Storage are impractical for Full Image Backups. Too many disks to manage in a recovery operation, and Cloud Storage does not begin to fulfill the needs of most serious backup archives. Too slow to upload, and in the event of a computer failure, no Internet connection is the norm. Data can live in the Cloud, but true system backups are best handled locally.

And as posted earlier in this thread, if you really want to mess up a backup archive, just try to defragment it. Backup Archives are written sequentially, and if they are defragmented, the exact sequencing and arrangements of the fragments into specific blocks and sectors on the backup drive usually will be disrupted in unpredictable ways. The end result has been for me, archives which would not pass Validation, and which could not be successfully mounted for recovery operations.

-- rc primak

Reply | Quote

And as posted earlier in this thread, if you really want to mess up a backup archive, just try to defragment it. Backup Archives are written sequentially, and if they are defragmented, the exact sequencing and arrangements of the fragments into specific blocks and sectors on the backup drive usually will be disrupted in unpredictable ways. The end result has been for me, archives which would not pass Validation, and which could not be successfully mounted for recovery operations.

Yes, this is what I suspected and was partly the reason for starting this thread. Thanks for confirmation and explanation

Reply | Quote

And as posted earlier in this thread, if you really want to mess up a backup archive, just try to defragment it. Backup Archives are written sequentially, and if they are defragmented, the exact sequencing and arrangements of the fragments into specific blocks and sectors on the backup drive usually will be disrupted in unpredictable ways. The end result has been for me, archives which would not pass Validation, and which could not be successfully mounted for recovery operations.

I would say it depends entirely on the backup software. BootIt is, in my experience, totally immune to defragmenting of its image files. I defragment all, I’ve never had a failure of validation, I’ve never had an issue with mounting an image.

I prefer bullet-proof, not fragile and finicky.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

I would say it depends entirely on the backup software. BootIt is, in my experience, totally immune to defragmenting of its image files. I defragment all, I’ve never had a failure of validation, I’ve never had an issue with mounting an image.

I prefer bullet-proof, not fragile and finicky.

I prefer backup software which can be accessed from within Windows Normal Mode, Windows Safe Mode, or from a Rescue CD/DVD. This is why I choose Acronis, which is a complete backup and recovery product, and offers the best options of any such software I’ve ever seen. Add to it Disk Director (also available from the Rescue Media) and even a munged file system or mangled MBR can often be repaired instead of reformatting and starting over.

Bootable archives are nice for full restore operations, but they usually are less useful for recovering files or folders (as in a virus cleanup where only some System Files need to be restored).

My point, in answer to the OP, was about the wisdom of defragmenting backup archives stored on external hard drives. Due to their size and their character, these files should not be defragmented. That is what was being discussed, not how “bullet-proof” the archives are.

And I find that once you learn the interface, True Image Home (TIH) is not “finicky”. It just works, time after time. As long as the archives are not more than two TIH versions old, and as long as the drive is not too crowded, and as long as no defragmenting is attempted, I have only had two archives not pass Validation in four version upgrades of TIH.

-- rc primak

Reply | Quote

As long as the archives are not more than two TIH versions old, and as long as the drive is not too crowded, and as long as no defragmenting is attempted, I have only had two archives not pass Validation in four version upgrades of TIH.

Too many “as long as” there for me. I much prefer no problems any time over a couple of problems sometimes, and image files that aren’t fragile or sensitive, but just work.

And I haven’t reformatted since Windows 95 OSR2.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote

Here, Here Bob. I use both Acronis TIH 2010 (wife’s PC) and Acronis TIH 2011 (my PC). as you state, learn the interface and it works!

Reply | Quote

Except when it doesn’t (on my particular system.) Then you use Macrium, learn the interface, and it works.

Dick

Reply | Quote

OK, folks, let’s end this debate over whose backup method is the best. Obviously, we all have our own operating environments and our own preferences. Just as long as we are all happy with the backups we do, we are all on the right side of the issue, I think. Whatever works best for us, at least we do back up and test our methods from time to time. That’s what really matters. And yes, if one backup method isn’t working the way you want it to work, by all means try something else.

-- rc primak

Reply | Quote

Assuming this is a major manual backup and you can prepare matters beforehand, and assuming it is hard drive to hard drive, then you should defrag both the destination drive and the source drive before the fact. If the destination is dedicated to backups, then there should be no need to defrag it more than once, and ‘never’ if it is dedicated from the start. Once it is full, you might reallocate it to general storage as you grandfather the backups, and switch to a different drive for backups. Or, as one post suggested, alternate between a pair of drives (or strictly grandfather them among three).

Two distinctions are useful: one is the state of the system when you back it up – ideally you want everything perfect or with as few unresolved problems as possible, which means that when everything is really humming, then it’s time to drop all else and make a backup, instead of slavishly making scheduled backups that capture buggy system states.
The second is that data is irreplaceable, and backing up data should take precedence over backing up systems. Systems are replaceable, data is not.

Reply | Quote

Assuming this is a major manual backup and you can prepare matters beforehand, and assuming it is hard drive to hard drive, then you should defrag both the destination drive and the source drive before the fact. If the destination is dedicated to backups, then there should be no need to defrag it more than once, and ‘never’ if it is dedicated from the start. Once it is full, you might reallocate it to general storage as you grandfather the backups, and switch to a different drive for backups. Or, as one post suggested, alternate between a pair of drives (or strictly grandfather them among three).

Two distinctions are useful: one is the state of the system when you back it up – ideally you want everything perfect or with as few unresolved problems as possible, which means that when everything is really humming, then it’s time to drop all else and make a backup, instead of slavishly making scheduled backups that capture buggy system states.
The second is that data is irreplaceable, and backing up data should take precedence over backing up systems. Systems are replaceable, data is not.

Right-on about data backups vs. system images. This is why I simply do copy/paste data backups whenever I make changes to something important. This is entirely different from a system image, and takes place much more frequently.

-- rc primak

Reply | Quote

There are a few simple questions one needs to answer regarding backups, if backups are deemed important.

Is your backup routine easy enough that it doesn’t feel like a dreaded chore? Is your backup routine/method absolutely trustworthy? If you can’t answer the question, perhaps you should do some testing. Is your backup media absolutely trustworthy? (If the only backup media you use is magnetic, the answer to that one is “no”. ) Can your backups survive a disaster? In other words, can you restore your complete system and all your data to bare metal (a new PC)? Can you do such a restoration quickly and easily?

This is still relevant.

Always create a fresh drive image before making system changes/Windows updates; you may need to start over!

We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.

We were all once "Average Users".

Computer Specs

Reply | Quote