Long-term storage media recommendations

Topic

New Reply

I would like a recommendation for what medium to use for LONG-term storage. I have some on a set of CDs and was advised to use USB thumb drive instead “because CDs won’t be readable for too much longer”. I want to know what medium is recommended for long-term “off site” storage of information which might be needed after a disaster, or needed by my successor trustees when I’m no longer around to access the information.

Thanks

J.Conklin

Reply | Quote

Replies

I would like a recommendation for what medium to use for LONG-term storage. I have some on a set of CDs and was advised to use USB thumb drive instead “because CDs won’t be readable for too much longer”. I want to know what medium is recommended for long-term “off site” storage of information which might be needed after a disaster, or needed by my successor trustees when I’m no longer around to access the information.

Good question.

I don’t think there will soon come a time when the CD format itself is obsolete, but studies on the lifespan of optical media have raised serious concerns about the shelf life of inexpensive CD-R discs. E.g., (1) Langa Letter: Time To Check Your CDRs — Fred Langa — InformationWeek; (2) Lounge: Media Data Storage Life.

The flash memory in a USB “thumb drive” should be very stable, but the electronics that connect the flash memory chips to the PC may be a bit more fragile.

I still think of hard drives in a standard format (e.g., FAT32 or NTFS) as the gold standard.

Reply | Quote

Any media you decide to use will need to be periodically checked and verified for data integrity, and not just
left and forgotten until needed. You must actively preserve it.
I would use more than one type of media; CD/DVD disks, USB thumb drives, and even a hard drive.
If it’s a hard drive, make sure it is stored in some sort of protected and sealed antistatic & antishock bag.
CDs & DVDs are not to be discounted, irregardless of what you have been told. Chances are, if properly
stored, they will out live you quite nicely.
Technology changes, hence the need to periodically re-evaluate media in light of new tech and
to verify the integrity of your data.

So a combination of media that is pre verified and periodically monitored for integrity over the years will
likely prove to be your best bet.

The storage location is important too. If the storage location is climate controled that is better still.
Bank safety deposit boxes are ideal for CD/DVDs, USB thumb drives, even an SSD but basically small items.
If the storage location is outside you can expect the local climate to play a roll in degradation;
Extremes in temperature vs humidity, sea side environments/desert etc. All can have their own adverse effects.

Key words:
Properly stored
Periodically verrified
Multiple types of media
Multiple copies in different locations

Reply | Quote

I use both a USB Ext HD (Seagate Go 1 TB) and a networked desktop PC (not used for much else anymore). I’m not sure about the “Long Term” storage of these 2 media, but the HD on my desktop is more than 7 years old and has not suffered any degredation as far as I can tell. As stated though, for the last year it’s been used sparingly. Good luck on what ever media you use. As Clint stated, checking periodically is a great policy.

Reply | Quote

You’ve heard of the Rosetta stone and the Dead Sea Scrolls? Rock and papyrus, I’m telling ya, that’s where its at!

But, accepting any feasibility study results on employing rock and scrolls as a backup medium, the above recommendations will have to do.

Reply | Quote

Magnetic tape is the gold standard, but it’s way too expensive for small business / home use.
Depending on the volume of data and your recovery requirements, you have 2 choices.
1. Large volume in one backup: SATA hard disk. This is very good value for storage capacity and should last 5 to 10 years without too many problems.
2. Small volumes backed up often: CD. Good quality CDs stored correctly should last 5 to 10 years.

As Clint said, store them properly, have more than one copy and check them regularly.

cheers, Paul

Reply | Quote

Magnetic tape is the gold standard, but it’s way too expensive for small business / home use.
cheers, Paul

Ummm, no. Not true about being the gold standard for backups. Mag tapes deteriorate over time and the magnetic iron oxide coating can actually flake off or wear thin, rendering the media useless. It’s why we who were in the mainframe and server management business for decades always knew to throw out tapes that were more than two years old. Better safe than find you have (literally) flaky spots on your tapes. What to do then when you discover your backups are unreadable? Update your resume and look for a new job!

Reply | Quote

I’ve had ordinary aluminum-based CDs simply vanish for no obvious reason. Some fluids can attack the medium, reducing portions to clear plastic. Ordinary rubbing alcohol will destroy some of them.

So CDs are not uniformly reliable. Web information yields a consensus that aluminum-based CDs manufactured in mass quantities will last a max of ten years. CDs written by drives on ordinary computers are not “burned” as deeply as the mass-produced variety. So caution is mandatory for information that simply cannot be lost. Data gleaned from a careful experiment is an example. That novel you’re working on but have never printed to hard copy is another.

There is a partial solution. Kodak manufactured gold-based recordable CDs. None of these has failed in my experience. The problems are two.

Last I checked, Kodak is no longer making these CDRs, and they are expensive. Most retailers will sell only in lots of 100, for more than a US dollar each. My judgment is that they’re worth the cost.

Searching the Web using the phrase “gold CD” will yield sellers, and some information about manufacturers other than Kodak.

I hope other Lounge Lizards know more than I do about all this.

Reply | Quote

drkurtz is right. There are special “archival CDs” and DVDs, and they are gold-based. And very expensive! But if the data to be preserved are important enough, several manufacturers have these archival media available. When stored properly, these could last in excess of ten years.

Thumb drives are the least reliable backup media, and have a high failure rate. Also, some recovery programs may not recognize them. Other mechanical hard drives are pretty reliable, but they still do degrade over time, and if not used periodically, they can freeze up and become useless, Cloud-based backup storage has not been mentioned yet in this thread, and its main drawback is that you have to trust that the company which hosts the backup service will remain in business for as long as you will need access to your backups. And there is also the problem in recovery of having to download all your backed up data in order to perform a restoration.

Personally, I use external USB hard drives. They have never failed, but have filled up with backup images, so I have retained nothing for extended periods. My OEM backups are all on DVDs, which I should check more often than I do, but I think they have lasted five or six years for my Winbook laptop. I would call that reliable enough for my purposes. These are not special Archival Media.

-- rc primak

Reply | Quote

If you are considering off-site storage, an additional method you might look into is the Amazon S3 storage server. It costs about 15 cents per GB per month – you can store 25GB for about $45/year, about the same price as a safety deposit box without all the hassle. It will be ‘safe in the cloud’.

Once you set up an account, the easiest way to access is using the S3Fox plugin for Firefox browsers. I believe there’s also a plugin for Chrome browsers or there is desktop software available to connect you to S3.

Reply | Quote

It seems the general consensus is on USB EHDD’s (Universal Serial Bus External Hard Disk Drives) and with that I’d have to agree. As has been mentioned flash drives are not very reliable for long term storage, also CD’s are iffy at best today. If you’re storing small amounts of data then a CD may well keep, especially the non-rewritable varieties (less susceptible to ESD (Electro Static Discharge) and magnets) but, there are questions as you’ve no doubt heard about how much longer CD’s will last as a well supported format. For distribution of software, especially Operating Systems, CD/DVD’s are still mainstream but, it seems that in the media marketplace they are being replaced quite quickly by direct download methods.

Also as has been mentioned ANY media you use should be checked periodically. Flash technology and Hard Disk technology rely on a stored charge which will last quite a while. For instance I accessed a hard drive which hasn’t been used in probably 5 or more years and still retained all of it’s original data but, that doesn’t mean that will always be the case.

I would also suggest if the data is that important that you use double backups.

Reply | Quote

If you’re storing small amounts of data then a CD may well keep, especially the non-rewritable varieties (less susceptible to ESD (Electro Static Discharge) and magnets)

CD recording is a mechanical process, not electronic of magnetic.

Flash technology and Hard Disk technology rely on a stored charge which will last quite a while.

Hard disks use magnetic recording, not stored charge.

Mag tapes deteriorate over time and the magnetic iron oxide coating can actually flake off or wear thin, rendering the media useless. It’s why we who were in the mainframe and server management business for decades always knew to throw out tapes that were more than two years old.

Quality magnetic tape stored properly will last decades. It has always been the gold standard for data storage.
http://www.clir.org/pubs/reports/pub54/4life_expectancy.html
http://www.imagepermanenceinstitute.org/shtml_sub/NEHTapeFinalReport.pdf

cheers, Paul

Reply | Quote

Also as has been mentioned ANY media you use should be checked periodically.

Several people have stressed the importance of checking periodically, but I wonder if this serves any useful purpose. When you check and discover the data is no longer available you are stuck.

It would probably be safer to make fresh copies of the backups from time to time, perhaps keeping the original and checking occasionally just to determine how long your backups remain usable, and make sure you make new ones well before this period has elapsed.

Reply | Quote

I should have added something the original poster may not (or may) have considered. If you choose the right media your backups may last longer than you, but the same is not true of backup/restore software. Another discipline we old dinosaurs acquired during mainframe days was periodically checking our old backups with current versions of the restoring software. You’d be surprised that few people read the fine print in the newest-greatest-most-wonderful-version-ever of their backup/restore software. Many times it’s only backward compatible by one or two versions. Beyond that, you may be out of luck and nobody will be able to read or restore your backups, no matter how well preserved they are.

Reap some lessons from solid I.T./I.S. experience and always use your current software to make newer copies of your older backup media. Externally label your backups with the company, name and version of the software in addition to the dates of backups. Never assume that one backup is adequate — your media can always fail for one reason or another. If this is critical data, always make two copies and store them in separate locations. If it can go wrong, it will.

Now you know what I do, and I spent 40+ years learning it the hard way!

Reply | Quote

Paul, hello.

>>> CD recording is a mechanical process, not electronic of magnetic. ( you might mean “or” magnetic. )

I do not know where you got this information but get a sheild up as the rounds will be coming in. The CD recording is done with a small magnetic head, the reading too. There is NO noise coming from a CD-R device beside the spinning motor. Mechanical devices are noise producing.

Tee ! hee ! Jean.

Reply | Quote

The CD devices I’ve seen, both playback and recording, are optical drives. So are DVDs. That’s what the little laser is all about.

Reply | Quote

Paul, the laugh is on me perhaps.

>>> >>> CD recording is a mechanical process, not electronic of magnetic.

I did a bit of reading and you might be closer to the solution than I. They speak of valleys and peaks in recording a CD. I guess that the laser is in fact, “burning” them in thus it could be as you wrote, a mechanical process. Either the peaks or valleys are 1 or 0 when read. Is this closer to your opinion ?

See, I am not too proud to admit a defeat ! Jean.

Reply | Quote

I don’t even know if its a mechanical process, I’ve always thought of it as more of a chemical or thermal process since its the laser that heats an organic layer in the disc and forms the bumps and pits that are later read by a differential wavelength sensing laser. Mechanical in that the disc is spinning I guess?

Reply | Quote

Byron, you got it. You da man !

>>> thermal process since its the laser that heats an organic layer in the disc and forms the bumps and pits

Be good. Jean.

Reply | Quote

Several people have stressed the importance of checking periodically, but I wonder if this serves any useful purpose. When you check and discover the data is no longer available you are stuck.

This is true on the surface of things but in conjunction or in the context that this is a backup or archived which should hopefully mean to most that there are still two copies of the data, possibly on two different types of media, there is still one or more good copies of the data when the other tanks for whatever reason.
Of course this doesn’t rule out the possibly sound advice of periodically renewing the backup to fresh and shiny media on a recurring basis instead of waiting until one copy fails.

Reply | Quote

This is going to be perilously close to topic-drift encouragement.

BUT, I’m going ahead with it.

I”m about to buy an external enclosure with a fan. It will accommodate two 3.5″ drives, and will connect with an SATA port on the back of the machine it’s going to be attached to. It will also have USB cable ports.

Does anyone have any experience or other information regarding the differences between USB 2.0, Firewire, and external SATA? In particular I’m curious about reliability as much as I’m interested in speed. The ASUS M2N-Sli deluxe mainboard being used has all three abilities. (I’m sticking with XP Pro until I get used to Ubuntu.)

Reply | Quote

I’ve used USB and firewire previously with external drives, but eSATA will probably be the faster of the three,
with USB being the slowest.
I’d look for something with more than one option though, whatever you decide to use.

Irregardless of what you go with you may be required to use the “safely remove” hardware device option prior to unplugging or removing them. Others in this forum have had issues with this.

Reply | Quote

The best place for archive data is on a server that is backed up regularly. This data store is then moved forward to the replacement server hardware and moved forward in the next replacement indefinitely. This data should be backed up with the yearly backup. As such it will always be fresh and not subject to obsolete hardware devices, etc.

In the home setting a partition on a desktop harddrive can serve this same purpose. Back it up periodically and move it forward to your next computer when you replace this one.

Reply | Quote

Go to MAM-A. They sell gold thrmal disks – not cheap, but good, quality stuff. Store in Safety deposit box – pull every few years and test. Make secondary back up at 5 years.

Reply | Quote

SATA is the fastest and the cheapest for the speed. USB3 is as fast, but there isn’t much of it about and eSATA is always going to be the choice for hard disks – native interface, no USB to SATA converter required.

cheers, Paul

Reply | Quote

Years ago, much information was committed to IBM type punch cards.
I worked on that equipment. That information is still as good today as it was then, if the cards were stored in a cool, dry place. But…..find a machine that could read those cards today. Maybe the Smithsonian might have a punch card reader, but then you’d need a compatible computer to take the data flow from the card reader. It just gets real complicated, trying to keep your valuable data on a media that won’t be soon obsolete. If I handed any of yous guys, a 5.25″ floppy disk and told you the greatest program ever was on the disk, would you know how, or have the hardware to read it? One out of a hundred might.

One thing I’ve done since PC’s came out with 3.5″ floppy disks, is transfer my old data from 5.25″ disks to the newer media.
Now I have to transfer that data to DVD’s if I intend to keep it, or just let it wither away.

In time, the cosmic rays that pass through us every day, will level the magnetic bubbles on all magnetic media, hard drives included.
I have many old floppy disks that have been summarily erased and are no longer readable with currently available drives.

The big deal a few years back was the GOLD CD from Sony that would presumably remember its data for 30 years, or more. Truth, or just Hype?
But the real question is, ‘in 30 years who will have a CD drive that could read those CD’s??? Not me!

As an electronics tech, I seriously doubt that any flash drive will still be readable in 30 years. Oxidation/Corrosion will surely take their toll.
I’ve seen at least a dozen or more flash drives less than three years old that have already shot craps. They are actually very fragile and can be destroyed by static electricity and the oils and salts in a person’s hand.
I put nothing on any flash media that I’m not willing to loose.
Permanent storage? Forgetaboutit!!!

Just a thought,
The Doctor

Reply | Quote

Hey Doc,

I remember those punch cards. My first programming class in college was Fortran using punch cards. What a hoot! Thanks for making me get my brain cell working to remember them. An early job had a Compac PC with dual 5 1/4 inch floppy drives as there was minimal HD or RAM (was RAM even invented at that time???) Hear the clanking as my brain cell kicks into high gear!

I don’t remember what the original question was, but thinking of those young years (both the PC and me) was a lot of fun. What do you mean off topic!!!
Cheers, Ted

Reply | Quote

Verbatim makes a line of archival CDs and DVDs under the brand name Ultralife. I found “interesting” reviews at amazon.com. Some buyers reported that not all the disks would burn successfully.

Here’s a link to Verbatim’s description of a 50-pack spindle of DVD’s:

http://www.verbatim.com/products/detail.cfm?product_id=6A0D4031-1143-3415-5F916B32947AE822&cat_id=7CA8481A-AA65-482E-991CD8C4156EA2D3

Also, Verbatim Medidisc is marketed as suitable for archiving medical records:

http://www.amazon.com/gp/product/B0003QIFIK/ref=wms_ohs_product

HTH,
Bob

Reply | Quote