How do I address Windows' file system fragmentation on SSDs?

gremlinkurst · Jul 12, 2023

Back when I was using analog drives, I regularly defragged my drives, kept them optimized, and consolidated free space. Now that I'm using solid state drives, I can't do that anymore, so there's a problem: Windows file systems still fragment, and defragging SSDs damages them. So, what can Windows users under these circumstances do to combat fragmentation safely?

NavyLCDR · Jul 12, 2023

SSDs are not affected by fragmentation because they do not have mechanical read/write heads.

gremlinkurst · Jul 12, 2023

NavyLCDR said:
SSDs are not affected by fragmentation because they do not have mechanical read/write heads.

Now, that is puzzling (thanks, by the way); they didn't teach me that when I was studying computer science at uni after I got out of the Navy. So, what do you reckon it is that made Windows fragment file systems on mechanical drives while Linux didn't (doesn't for Linux users still running moving parts hardware)? Back before solid state drives were manufactured and marketed to the average consumer, I learned how to operate, configure, and administer many operating systems, and I've never found a platform more rife with problems and difficulties than Windows. However, if what you said is true, it makes me question what I had been taught, the motives of those that taught me. When it comes to science, personal bias of a non-rational nature has no authoritative place, neither ethically nor rationally.

pparks1 · Jul 12, 2023

It's not that linux doesn't have fragmentation, but the design of the file system makes it far less regular than with Windows based file systems. When a Linux filesystem writes a file, it leaves substantial space before and after the file. This allows modifications to these files in this extra space. If insufficient space exists here on future writes, it will attempt to rewrite the entire file elsewhere on the disk.

pparks1 · Jul 12, 2023

The reason fragmentation isn't as much of an issue with SSD's, is the fact that being solid state, the drive controllers abilty to get to any spot on the drive is pretty much instantaneous. Therefore, if a fragmented file is scattered in 5 different locations on your SSD, the drive can get to all 5 locations in a fraction of a millisecond to gather the entire file.

Back in the mechanical drive days, the drive heads would have to physically move to another location to pick up all the fragments of the file, and f it takes 12+ms per fragment, a file spread out in 5 locations would take 60.0ms to be found...where as the SSD would do it i like 0.15ms. So, once a mechanical hard drive is fragmented enough, you want to defrag and get as much of the drive contiguous as you can. However, doing this on an SSD uses up write cycles for what amounts to a completely negligible time penalty, so the "juice isn't worth the squeeze".

MisterEd · Jul 12, 2023

pparks1 said:
The reason fragmentation isn't as much of an issue with SSD's, is the fact that being solid state, the drive controllers abilty to get to any spot on the drive is pretty much instantaneous. Therefore, if a fragmented file is scattered in 5 different locations on your SSD, the drive can get to all 5 locations in a fraction of a millisecond to gather the entire file.

Back in the mechanical drive days, the drive heads would have to physically move to another location to pick up all the fragments of the file, and f it takes 12+ms per fragment, a file spread out in 5 locations would take 60.0ms to be found...where as the SSD would do it i like 0.15ms. So, once a mechanical hard drive is fragmented enough, you want to defrag and get as much of the drive contiguous as you can. However, doing this on an SSD uses up write cycles for what amounts to a completely negligible time penalty, so the "juice isn't worth the squeeze".

I would like to add that a SSDs life is limited to a fixed amount of writes to them. Defragmenting does unnessary writes that could reduce the life of an SSD.

pparks1 · Jul 12, 2023

MisterEd said:
I would like to add that a SSDs life is limited to a fixed amount of writes to them. Defragmenting does unnessary writes that could reduce the life of an SSD.

Yup, hence my last sentence: However, doing this on an SSD uses up write cycles for what amounts to a completely negligible time penalty, so the "juice isn't worth the squeeze".

NavyLCDR · Jul 12, 2023

gremlinkurst said:
Now, that is puzzling (thanks, by the way); they didn't teach me that when I was studying computer science at uni after I got out of the Navy. So, what do you reckon it is that made Windows fragment file systems on mechanical drives while Linux didn't (doesn't for Linux users still running moving parts hardware)? Back before solid state drives were manufactured and marketed to the average consumer, I learned how to operate, configure, and administer many operating systems, and I've never found a platform more rife with problems and difficulties than Windows. However, if what you said is true, it makes me question what I had been taught, the motives of those that taught me. When it comes to science, personal bias of a non-rational nature has no authoritative place, neither ethically nor rationally.

Sounds like the "uni" that you studied computer science at wasn't very high quality. It is very basic knowledge that file fragmentation does not affect SSDs because SSDs do not use mechanical read/write heads. The problem with file fragmentation on hard drives is that the mechanical read/write heads must move from one area of the physical platters to other areas in order to assemble the files. Solid State Drives do not have that issue.

zbook · Jul 13, 2023

This administrative PS command may be useful for drive analysis:

Optimize-Volume -DriveLetter H -Analyze -Verbose

(modify H to the applicable drive letter)

This is from a Microsoft link (updated 02/03/2023) with comments on SSD defrg

defrag

Reference article for the defrag command, which locates and consolidates fragmented files on local volumes to improve system performance.

learn.microsoft.com

When run from the scheduled task, defrag uses the below policy guidelines for SSDs:

Code:

Scheduled task
The defragmentation process runs scheduled task as a maintenance task, which typically runs every week.
As an Administrator, you can change the how often the task runs by using the Optimize Drives app.

When run from the scheduled task, defrag uses the below policy guidelines for SSDs:

Traditional optimization processes. Includes traditional defragmentation, for example moving files to make
them reasonably contiguous and retrim. This is done once per month. However, if both traditional defragmentation
and retrim are skipped, then analysis isn't run. Changing the frequency of the scheduled task does not affect the
once per month cadence for the SSDs.

If you manually run traditional defragmentation on a SSD, between your normally scheduled runs, the next scheduled
task run performs analysis and retrim, but skips traditional defragmentation on that SSD.

If you skip analysis, you won't see an updated Last run time in the Optimize Drives app. 
Because of that, the Last run time can be up to a month old.

You might find that scheduled task hasn't defragmented all volumes. This is typically because:

The process won't wake the computer to run.

The computer isn't plugged in. The process won't run if the computer is running on battery power.

The computer started back up (resumed from idle).

MisterEd · Jul 13, 2023

gremlinkurst said:
Now, that is puzzling (thanks, by the way); they didn't teach me that when I was studying computer science at uni after I got out of the Navy. So, what do you reckon it is that made Windows fragment file systems on mechanical drives while Linux didn't (doesn't for Linux users still running moving parts hardware)? Back before solid state drives were manufactured and marketed to the average consumer, I learned how to operate, configure, and administer many operating systems, and I've never found a platform more rife with problems and difficulties than Windows. However, if what you said is true, it makes me question what I had been taught, the motives of those that taught me. When it comes to science, personal bias of a non-rational nature has no authoritative place, neither ethically nor rationally.

Linux didn't need to defragment hard drives because it is more intelligent than Windows about how it places files on the hard drive. However, eventually even a hard drive used with Linux may need to be fragmented. Even then it is way less often than Windows.

I suppose you could find a way to defragment even an SSD. But why would you want to do something that has no benefit and reduces the life of the drive?

BTW, I used my first computer in 1973 and bought my first computer in 1979. A lot has changed since then. You have to keep up with technology as it changes and not assume things that were true before are still are.

How Windows File Systems Work

Microsoft’s old FAT file system — last seen by default on Windows 98 and ME, although it’s still in use on USB flash drives today — doesn’t attempt to arrange files intelligently. When you save a file to a FAT file system, it saves it as close to the start of the disk as possible. When you save a second file, it saves it right after the first file — and so on. When the original files grow in size, they will always become fragmented. There’s no nearby room for them to grow into.

Microsoft’s newer NTFS file system, which made its way onto consumer PCs with Windows XP and 2000, tries to be a bit smarter. It allocates more “buffer” free space around files on the drive, although, as any Windows user can tell you, NTFS file systems still become fragmented over time.

Because of the way these file systems work, they need to be defragmented to stay at peak performance. Microsoft has alleviated this problem by running the defragmentation process in the background on the latest versions of Windows.

How Linux File Systems Work

Linux’s ext2, ext3, and ext4 file systems — ext4 being the file system used by Ubuntu and most other current Linux distributions — allocates files in a more intelligent way. Instead of placing multiple files near each other on the hard disk, Linux file systems scatter different files all over the disk, leaving a large amount of free space between them. When a file is edited and needs to grow, there’s usually plenty of free space for the file to grow into. If fragmentation does occur, the file system will attempt to move the files around to reduce fragmentation in normal use, without the need for a defragmentation utility.

Of course, this is different for solid state drives, which don’t have moving parts and shouldn’t be defragmented — defragmenting an SSD will actually reduce its life. And, on the latest versions of Windows, you don’t really need to worry about defragmenting your file systems — Windows does this automatically for you. For more information on best practices for defragmenting, read this article:

Why Linux Doesn’t Need Defragmenting

Why Linux Doesn’t Need Defragmenting

If you’re a Linux user, you’ve probably heard that you don’t need to defragment your Linux file systems. You’ll also notice that Linux distributions don’t come with disk-defragmenting utilities. But why is that?

www.howtogeek.com

TechnoMage2021 · Jul 13, 2023

For 26 years, I've used the same ol' Backup/Restore program. (Ghost)
And I'm still using the last version, that ran in DOS, Ghost 11.5. I can boot up any system that will boot from a CD, Flash Drive, SD Memory Card or even a Floppy Disk, with my Ghost boot disk and back it up, or restore a Backup Image file, to it. That gives me flexibility not available with Widows based programs.

Back in the days when I still ran Spinners as my everyday HD's, I did a weekly massive HD cleanup, ending with a full Defrag. I found out that when I did my Ghost Backup, and while the program was still running, I could RESTORE the backup I had just made, and Ghost would overwrite the HD, putting the files down on the HD in sequential order, end to end, with no spaces between files and NO Fragmentation! It's what I called the "Perfect Defrag".
And, the Ghost Restore actually took less time than the average Defrag. *

* Because the Windows Defrag ran from within Windows, there were some files that it could not defrag or move.
So no defrag was Perfect. Using "Ghost Restore" to defrag my HD's didn't have that problem. **

** The Ghost Backup/Restore program that I use, runs in DOS, from a DOS boot disk (or Flash Drive) so it doesn't see an OS or data, just ones and zeros. It doesn't care what the files are, it just backs up what it sees. It will back up any version of DOS, Windows, Linux, Server, or even a data disk with no OS at all. Or even a corrupt Windows disk that cannot even boot up a system.

But, nowadays, with the SSD's, I can eliminate that (Restore) step.
Just like I don't worry about fragmentation on a Flash Drive, I don't worry about fragmentation on my SSD either.
SSD's do use a program called "Trim" to optimize the SSD. It may be worth Googling "Trim" for a full explanation.
I wouldn't even attempt that here.

Some of my Flash Drives, get a real workout, with reads, writes, deletions, file updates, etc. So every so often I copy all the files off of a Flash Drive, reformat it, and then copy my files back to it. Without going into the details, it just seems like a "Thing to do".

Cheers Mates (interesting thread)

TM

MilesAhead · Jul 13, 2023

TechnoMage2021 said:
I could RESTORE the backup I had just made, and Ghost would overwrite the HD, putting the files down on the HD in sequential order, end to end, with no spaces between files and NO Fragmentation! It's what I called the "Perfect Defrag".
And, the Ghost Restore actually took less time than the average Defrag. *

My older brother did Mini-computer repair and maintenance for DEC. The way they defragged the HD was to make a backup, then restore it. No other software needed. I went with him on a repair call one time. He took the cover off the top of a HD to show me. The drive head was moving all around and open to the air. Evidently being in an air conditioned room was enough to keep it happy. No hermetically sealed drives. It kind of reminded me of a stack of 45s on an old record player.

I noticed doing Macrium incremental backups to external SSD one drive wrote much faster if I used the Tools in Explorer context menu to defrag it. The defrag only took about 10 seconds. A larger SSD of the same make has no option to optimize the drive and does not seem to need it. After writing the incremental backup I let it sit until the access LED goes out. Usually less that 30 seconds. It must have some automatic trim command in the device driver is my guess.

TechnoMage2021 · Jul 13, 2023

Speaking of old hard drives... the first one I ever worked on was a single disk, open to the room air, and the read and write heads were mounted in a solid aluminum plate just above the platter surface, and had to be adjusted manually using an O'scope. Loads of fun, and many hours sitting on the floor.

Many years later, I was the key operator on an NCR Mainframe, that had a single (sealed) hard drive in its Central Processing Unit. (CPU)
When I began working there (the county IT dept) I saw a lot of garbage on that old HD. And, in ten years, it had never been cleaned up, restored, or Defragmented. It was a freakin' mess!!!
After some conversation with the IT manager, I was able to clean all the garbage files off of that HD, then after work one day, when the computer was shut down for the night, we backed it up to Disk Packs and Tape, then reformatted it and restored all the files.
The next morning, when the data input clerks came to work, I got a call, asking me what I'd done to the Computer, because it was running faster than they had ever seen, in ten years.

MilesAhead · Jul 13, 2023

TechnoMage2021 said:
Speaking of old hard drives... the first one I ever worked on was a single disk, open to the room air, and the read and write heads were mounted in a solid aluminum plate just above the platter surface, and had to be adjusted manually using an O'scope. Loads of fun, and many hours sitting on the floor.

My brother actually rewrote the procedure for calibrating the tape systems used on DEC mini-computers. He always lugged around a small oscilloscope because he didn't believe a multimeter always told the whole story. I believe these tape backup units had three capstans. The procedure in the manual he used had this complex four hour method. Apparently every time you adjusted one capstan it threw off the other two. Finally he said there must be an easier way. He tried calibrating them in order three times rather than using the complex pattern. He could demonstrate that using this technique the system always ended up in spec. Time required, forty-five minutes. When he showed his results to the tech management they said "write it up."

SIW2 · Jul 13, 2023

The answer is in the link zbook provided.

gremlinkurst · Jul 19, 2023

NavyLCDR said:
Sounds like the "uni" that you studied computer science at wasn't very high quality. It is very basic knowledge that file fragmentation does not affect SSDs because SSDs do not use mechanical read/write heads. The problem with file fragmentation on hard drives is that the mechanical read/write heads must move from one area of the physical platters to other areas in order to assemble the files. Solid State Drives do not have that issue.

The "uni" that I attended was not of inferior quality...it's just that SSDs had not yet been invented...and neither had the internet.

MilesAhead · Jul 19, 2023

I have AData external SSDs and thumb drive so I noticed this freeware ADATA SSD ToolBox

I optimized 2 AData external SSDs with it. It seemed to do no harm. It does a Trim command on the free space.

It also offered system optimization for my laptop's internal SSD. I haven't tried that yet. But I would say it is worth a look. With any optimizing software make sure you can restore the current state if it should scramble things.

SIW2 · Jul 19, 2023

or this

In addition

If anybody wants to secure erase an ssd , the solidigm synergy says it works for all disks not just their own brand.

hsehestedt · Jul 19, 2023

MilesAhead said:
I have AData external SSDs and thumb drive so I noticed this freeware ADATA SSD ToolBox

I optimized 2 AData external SSDs with it. It seemed to do no harm. It does a Trim command on the free space.

It also offered system optimization for my laptop's internal SSD. I haven't tried that yet. But I would say it is worth a look. With any optimizing software make sure you can restore the current state if it should scramble things.

All the optimization for an SSD does is to send TRIM "hints" to the SSD. The OS will tell the SSD to perform a TRIM when blocks are deleted so an SSD should automatically perform a TRIM operation in the background. However, if power is lost before the SSD completes TRIM operations, it may not get all blocks trimmed. As a result, when Windows performs the regularly scheduled optimization, it simply sends TRIM "hints" telling the SSD "these are all the unoccupied blocks" and the drive will then check to see what blocks are not already TRIMmed.

MisterEd · Jul 19, 2023

gremlinkurst said:
The "uni" that I attended was not of inferior quality...it's just that SSDs had not yet been invented...and neither had the internet.

I graduated with a 2-year AAS degree 49 years ago and a 4-year BS degree 40 years ago. There were no SSDs or Internet back then either.

I guess you missed the most important skill you were taught in college which was "How to learn". With that skill learning never stops. Over the years I have learned a lot about computers including SSDs and how they are different than mechanical drives. Of course if you are unwilling or unable to learn new things after you get out of school you will never realize that what you learned then may be no longer true or relevant.