Windows file systems

Topic

New Reply

ISSUE 21.49 • 2024-12-02 WINDOWS 11 By Ed Tittel When Windows first made its debut in November 1985 — coming up on 40 years ago — it included support
[See the full post at: Windows file systems]

17 users thanked author for this post.

rc primak, jburk07, Fred, wosully, TechTango, mcbsys, fisher75, AlexEiffel, schmootoo, GeeBeeDee, DLivesInTexas, windbg, Casey S, lmacri, Norio, Alex5723, WSRolandJS

Reply | Quote

Replies

You skipped WinFS? You know, the all new file system to replace NTFS in Windows Vista. And the main reason why Vista was delayed, delayed, delayed and in the end hit the market without WinFS. At the time, WinFS was the next best thing according to Microsoft. But for some reason, they didn’t put it on the back burner to be introduced in a later version of Windows – it was simply scrapped? I wonder why, after spending so much money…

Reply | Quote

I not only skipped WinFS, I completely forgot about the Windows Future Storage system. I hope I can be forgiven, though, because that project was never released for public consumption. They canceled it in 2006, according to what I just read online.

Thanks for the reminder, tho. For the incurably curious, here’s a Wikipedia article on that topic. Also: thanks for posting.

Cheers!

–Ed–

2 users thanked author for this post.

Cybertooth, rc primak

Reply | Quote

Ed! How are you?!? I use still a couple of Ye Olde File Systems, FAT and NTFS. When I was substitute teaching, I still remember subbing for a class in which your son was one of the students! Please tell him I said hello to him and to all of you/family.

"Take care of thy backups and thy restores shall take care of thee." Ben Franklin, revisted

Reply | Quote

Thanks for your comment. Greetings will be conveyed the next time Gregory calls in from Emerson College, where he’s a junior this year. Didn’t we meet for lunch one time a few years back? I dimly recall something along those lines.

FWIW, I used four different file systems on Windows 10 and 11 regularly FAT32, exFAT, NTFS and ReFS.

Nice to hear from you,

–Ed–

Reply | Quote

Don’t remember right now, we can still do so again, since I’m retired, name some sample times.  Quick, write down my email [removed email address], I will remove it later.

Moderator Edit: Please do NOT post personal information in the Forums.

Reply | Quote

Recently, I researched the subject of ReFS again, as I have been using it to do mirrored drives protected from corruption and bit rot since Windows 8 for many people and me and I wanted to make sure it was good enough for my new PC as I’ve came across scary stories about it over the last few years.

It is hard to find good information about ReFS. It seems more tested on the cloud and used mostly on Windows Server. The fact that Microsoft removed the ability to use ReFS on mirrored drives in the Pro version of Windows 10 and restrict it to the obscure and expensive Workstation version during a feature update a long time ago probably didn’t help, although mirrored drives will probably always be niche. I’ve bought the workstation version just to run a mirrored raid ReFS drive, because I could, but I don’t think Microsoft should ask normal users to pay for that privilege.

Two times, Windows updates that were clearly not well tested broke some ReFS drives. I was lucky I never ran ReFS the way it was required to be affected and I always delay Windows feature updates the longest I can before it looses support for security, so I didn’t suffer from this, but it showed me clearly that Microsoft didn’t do their homework with what should be rock solid feature.

I’ve read very concerning information that seemed to indicate that after all those years, Microsoft still hasn’t succeeded at making ReFS a more resilient file system than NTFS, but in fact it still offers you more risk of loosing your data than if you run NTFS.

I’ve had some success with Intel Raid for mirroring, I saw it saved a few computers over the years. I also seen it 2 or 3 times disable itself and offer 2 identical drives to the user, without that user noticing quickly, so it is not fun to try to determine on which drive you worked for a while and if it was always the same drive that got written to. Anyway, ReFS was protecting against bit rot, which Intel Raid didn’t do, so that was the main reason for me to switch.

I’ve looked into AMD’s raid solution and let’s simply say that it doesn’t look like a viable option at all.

So, I found out that some guy wrote some code to test ReFS and he found very concerning bugs. It seems someone from the ReFS team reached out to him and thanked him for finding these things and they would work on it, but then, no news. This has been many months but those issues were not that old. It seemed for example that sometimes the system would use the data from the bad disk to “fix” a file if it found corruption instead of the good one if the corruption didn’t happen on the “good” disk (I know)! It seemed the system, when enabling the feature that autocorrect corruption could actually corrupt your drive.

Also of concern, with the difficulty finding any detailed information about what they fixed on the different versions, the team said it would be very difficult to port back the latest versions of ReFS to Windows 10 that is still stuck at a much older version. Does that imply that critical bugs are not corrected on Windows 10? There have been a few versions since the one Windows 10 is still on and that version predates the bugs identified. This is also scary.

Also, someone upgraded its computer to Windows 11, his disks got upgraded to the latest version and then he wasn’t able to read them anymore (maybe trying to read them back on Windows 10, I guess). This is not great although understandable if Windows 10 is stuck at an older version.

I’ve also seen numerous people report that everything worked fine and then suddenly they woke up with a raw disk for no reason, and it didn’t necessarily seemed tied to the Windows updates bugs. This is scary.

I’m skipping all the issues that the terrible GUI can cause because it doesn’t work well and you have to use powershell to avoid some issues with the configuration of the ReFS drives in Storage Spaces.

Frankly, after all this, I was very disappointed that Microsoft doesn’t provide what I consider basic tools for a reliable, professional OS, just like when they announced they would deprecate the Windows imaging solution and suggest to us that we use third party tools to create a backup image of the result of our lengthy post-install process to dedumbify Windows.

Since I have an AMD computer, I decided I would run a NTFS Storage Spaces mirrored drive, hoping if one physical disk fails, I will be able to read it on another computer. This puts my dream of protecting my data against bit rot to storage until much later. The only alternative that seemed much more close to the ideal scenario was a NAS, either running ZFS with Linux or a store bought product (although some of those had issues too over the years). I didn’t want to go there and I think for many people I help, they don’t want a second expensive tool to safeguard the data inside their only computer. Now I just hope bit rot won’t be noticeable and I accept the fact that I might loose some files but I would be quite unlucky to loose an important file without noticing.

I also think it can be a good idea to burn 100GB M-disks of data that doesn’t change. This is probably the safest bet for archiving. But I’m diverging from the ReFS subject.

Yes, I know, RAID is not a backup, but I love that you don’t waste much time when it protects you from a physical hard drive failure. I also know that despite best intentions, a lot of people don’t do their backups often enough and RAID can save them in the case of a physical disk failure (not ransomware or accidental deletion). It is a very nice additional protection.

RAID on Windows should work as good as hardware RAID in terms of reliability and protection from corruption in 2024. I can’t count how many times I had hard disk fails on servers and I just popped the drive out to replace it with another while running the server and nobody noticed anything.

So, for me, right now, ReFS is a big fail. It might be useful now on a dev drive for performance reasons, but I would not trust it for the scenario outline above, until Microsoft can show more transparency and a much clearer commitment to it. Right now, it is clear they didn’t test it properly over the years and anything they add to it, just like the new features of Windows, can seem to raise the risk of issues on such a fundamental feature that should not be fragile.

So, I hope that my experience helps anyone here who is a data hoarder or interested in preserving their precious data.

Reply | Quote

Wow! Thanks for sharing. I’d steered clear of ReFS for the very reasons you mentioned in your post, but when MS made it public in a big way — for both Win10 and Win11 — through the Dev Home app (preview release: May 30, 2023; public release Sept 26, 2023 in a Windows 11 update; MS Store release for Windows 10 on Jan 10, 2024), I jumped back on it. I’m not using it for anything important but I do target some backups to an external ReFS drive. Knock on wood: so far, no problems. I also made sure Macrium Reflect could read/write ReFS backups before I took that plunge, though (they’ve supported it since July 2021, in fact).

If you had a question in your post, I missed it. Again: thanks for your extensive commentary.

Best wishes,

–Ed–

2 users thanked author for this post.

rc primak, AlexEiffel

Reply | Quote

No question, I was just sharing my experience and research to add to your great article so people think twice and research the subject themselves well before using it for the purpose I would like to use it.

I had the same reflex as you thinking oh Microsoft is making it mainstream, it is better… maybe it will be better, not sure it is right now and it probably isn’t on Windows 10 since it is stuck to an older version. I wonder if version 3.4 even supports what makes it useful for a Dev drive. For now, on the desktop, it might be good for developers only as one ReFS drive doesn’t have all the same potential issues as a RAID one. You can also not enable the automatic repair of corruption, which seems safer, paradoxically.

I want to point out I also read many issues although not all of them happened when using thin provisioned drives.

Another problem is that to warn the user of corruption, it seems easy to miss it, think regular event viewer consultation might be better because you might miss the notification and also some reported that Windows was saying the disk was fine in some places and it was not fine at all in other…

I think I read something along the lines of using it for development makes sense because it handles performance much better for the kind of file operations developers do. Maybe it has something to do with the support of copy on write, where instead of copying multiples copies of the same block for files that have blocks in common, it links to the block already written (not sure, I go from memory and didn’t pay too much attention to this aspect). This would make sense if you have multiple similar versions of compiled code.

I’ve been running ReFS on version 1.2 moved from Windows 8 to 10 since many many years with no issues except some files can’t be copied (very rarely) to it, but I didn’t get that extensive real-time automatic protection and recovery from corruption that appeared later and caused some of the recent issues.

Some might tell me I read anecdotal evidence of issues. Yes, maybe in some cases, but sometimes we only have induction as the best method for taking a decision and I have not seen people jump in and say hey I had similar issues with NTFS too on Storage Space! Considering the number of people running ReFS on the desktop vs NTFS, it indicates an imbalance in the ratio of issues/use and those issues were really bad. Maybe some would say there is not that much mirrored NTFS with Storage Space, too. I guess. Also, the fact that you could run a script and observe in real time the bugs or the erratic behavior of ReFS is pretty telling. At the end of the day, since information about ReFS is scarce and there is not much about how great it works, or acknowledgment and documentation of fixed issues easy to find, it still looks like a side project on the desktop. Maybe Microsoft hopes that developers will find their bugs and help them make it more mainstream for a future version of Windows? I think they mostly  focused on the cloud and didn’t test it enough on the desktop.

Last, REFS shouldn’t bother you too much. An uppercase here instead of a lowercase, it’s just like having a 0 instead of a 1 somewhere, it’s a sign that Microsoft is focusing on the big picture, not sweat the small minor details, uh? It’s just sad that ReFS doesn’t deserve the love and talent that some people at Microsoft definitely have. I guess round corners and taking 10 years to create a complete new settings dashboard is more important.

It is sad that Microsoft doesn’t seem to understand the difference between trying something like widgets or Paint3D only to deprecate them later and providing a new filesystem where you can’t afford to beta test it on users. Having the GUI so dysfunctional is a symptom of the lack of care for this feature and doesn’t look good. If I was in charge at Microsoft, I would use Windows as a marketing platform. Since not many people can avoid it, I would make it so good at the basic level, making power users happy and able to quickly tune it to how they like it and make them able to easily help others, so that respect for the professionalism of Microsoft would be high and they would be the no nonsense go to company for everything else where the money is. Maybe they don’t feel they need that, maybe rightly so, but I think they are just lucky to have a monopoly by pragmatism. As for me, I avoid everything Microsoft when I can since Windows 10 and I am sure I am not alone.

 

Reply | Quote

And again, thanks for all the detail. Once you get into the weeds on any Windows topic, it’s amazing how big the crop comes in. I learned some useful bits from each of your comments. Keep up the good work!

–Ed–

1 user thanked author for this post.

AlexEiffel

Reply | Quote

Interestingly, I was given an old Toshiba laptop today by a friend and asked to retrieve the pictures on the drive.

Machine booted fine, unfortunately it wanted a password, NTPasswordRecovery to the rescue.

Once into the machine I attached a 1Tb NTFS 2.5″ HDD in an external case to the device. It would not assign an drive letter. It recognized that a USB device was attached but no drive letter. I checked if XP supported NTFS and yes it does, up to 2Tb.

I popped in a 32GB USB stick formatted as FAT32 and it assigned a drive letter right off and I got to copying the pictures.

May the Forces of good computing be with you!

RG

PowerShell & VBA Rule!
Computer Specs

Reply | Quote

You prove once again that where Windows is concerned: “Where there’s a will, there’s often a way.” Now, if only it didn’t sometimes take a long, long time to figure things out. Thanks for your comment. I hope it helps some other stuck individual someday. Indeed, I’ve filed your info for future reference should I ever find myself in that particular boat.

Thanks for your comment. Best wishes,

–Ed–

Reply | Quote

Very good article.

The following is from the article:

“The biggest hard disks and SSDs for 2024 are 32TB and 100TB, respectively.”

Should that be in reverse?

Joe

Reply | Quote

Biggest hard disk 2024: 24TB WD Gold. Source: Tom’s Hardware.

Biggest SSD 2024: 100 TB Nimbus Data ExaDrive DC100. Source: PCMag.

Counterintuitive, ain’t it? But no, I don’t have it backward. Thanks for asking, though: I was struck at the time that it seemed topsy-turvy, too.

Best wishes,

–Ed–

Reply | Quote

Joseph Pontecorvo wrote:

“The biggest hard disks and SSDs for 2024 are 32TB and 100TB, respectively.”

The largest SSD is Nimbus ExaDrive DC 100: 100TB SSD, priced at $40,000
The largest HDD is Western Digital Ultrastar DC HC690: 32TB HDD

1 user thanked author for this post.

Ed Tittel

Reply | Quote

Funny thing, I can find the home page for the product, but I can’t find a price for this 32TB behemoth online.  Even the “Buy direct” item on that page isn’t live. I’m guessing it will cost somewhere north of $1,000 given that the 24TB drives go for ~$550. Somebody share a link to this device if you can find it, unlike yours truly.

Thanks for posting,

–Ed–

 

Reply | Quote

WD introduced 128TB QLC NVMe SSD

Samsung, Huawei and Phison 128TB SSDs

Reply | Quote

I had a Win10 23H2 machine happily managing a secondary ReFS drive. After upgrading to Windows 11 24H2, the ReFS drive was no longer accessible. Apparently Windows 11 can’t read ReFS version 3.4, although Windows 2022 can, and will update it to version 3.7.

I booted from a Win10 ISO to retrieve the data (RoboCopy to another drive), then reformatted the “bad” ReFS drive as NTFS.

More details:

https://techcommunity.microsoft.com/discussions/windows11/refs-volume-inaccessible-after-update-from-windows-10-22h2-to-windows-11-23h2/3999414

Reply | Quote

Dear mcbsys: I can’t say that I blame you. Right now, I use ReFS only for non-essential stuff myself, and make sure to have an integrity checked backup handy. I believe MS intends Windows 10 and 11 users to run ReFS in the context of a developer environment (e.g. GitHub and or Visual Studio). It’s a wonder that this file system is available in Disk Management as a format option for bigger drives though it shows as REFS not ReFS. Note: the screencap below shows only NTFS, but if you explore all options, you’ll see “REFS” as one of them. Sigh.

Thanks for sharing. Readers should be careful about what they put on ReFS drives, and be sure to maintain a usable backup “just in case.” Sigh again.

–Ed–

Reply | Quote

Great article, and thank you for the detail and history!

One thing, Windows 11 24H2 (HOME) now comes with BitLocker. Previously BitLocker was not supported on home systems but that appears to have changed. I purchased the Co-Pilot + PC with Win 11 24H2 Home, and there was BitLocker.

Reply | Quote

Home doesn’t get the File Encryption mentioned in the article, but has got Automatic Device Encryption for more than eight years since Windows 8.1. It’s only activated if you sign into Windows as admin with a Microsoft Account.

1 user thanked author for this post.

rc primak

Reply | Quote

Thanks for sharing the items about Windows 11 Home 24H2. I don’t run that OS if I can help it, because I have 8 PCs in my office and use RDP to get from my primary desktop to all the other PCs (and 3 more elsewhere in the house). Home doesn’t support RDP.

Happy holidays, too!

–Ed–

PS: this item is a response to new items for today about Windows 11 Home from both wosully and b.

Reply | Quote

This topic has been very informative!

"Take care of thy backups and thy restores shall take care of thee." Ben Franklin, revisted

1 user thanked author for this post.

Ed Tittel

Reply | Quote

“resistant to corruption and damage, and able to maintain data integrity despite such errors.”

The ability to recover from these errors is also called fault tolerance. Resilience gives more emphasis to resisting corruption in the first place. Both terms apply to modern, advanced file systems like ReFS, btrfs and ZFS. Chromebooks also make use of dual Root file systems, each one backing up and updating the other. A companion term is “immutable”, meaning that the working Root file system is never changed until its companion is updated. These techniques improve resiliency and fault tolerance.

Likewise, native NTFS drivers (NTFS-3G and NTFS3) are available for Linux, but with varying degrees of kernel compatibility and support, depending on specific distributions in use.

I noticed this issue of varying degrees of Linux kernel support for NTFS when Mint was upgrade from Victoria (21) to Wilma (22). Like Ubuntu in their 24.04 release, Mint Wilma upgraded to a much more recent Linux kernel than its predecessor. Suddenly, USB interfaced NTFS partitions on external SSDs couldn’t be trimmed, unmapped or discarded the way they could be under previous kernels. This is referred to in Linux as a regression. The same NTFS-3G FUSE utilities and commands were available. Such regressions are different from deprecations, such as the recent removal of ReiserFS support from the Linux kernel. (BTW, I prefer to let Windows do the trim operations for NTFS partitions, because Windows developers actively support all the features of NTFS.)

MiniTool Partition Wizard (MTPW) is also one of my favorite tools for dealing with partitions, especiallly in mixed systems where there are both Linux ext4 partitions and Windows NTFS partitions. It pairs nicely with Macrium Reflect for backup and cloning of partitions. Much faster than most Linux tools like Clonezilla and gParted.

-- rc primak

1 user thanked author for this post.

AlexEiffel

Reply | Quote

Dear RC Primak:

Thanks for your insightful and thoughtful comments. I like MTPW enough that I run paid-up licenses on production PCs (their data recovery tools have also saved my tuchus on three or more occasions). I’m all in on Macrium Reflect 8 now, because it’s by far the best way to back up and restore ARM-based Windows 11 PCs.

It’s also true that “foreign support” for OS-specific file systems — Linux support for NTFS or ReFS via special drivers and tools — can be a mixed bag. The same kinds of deprecations you mention for NTFS in Wilma seem more or less inevitable for ReFS items as well, as the changing tides of development shift and over as they always do over time.

Thanks again,

–Ed–

1 user thanked author for this post.

TechTango

Reply | Quote