Macrium Free. Restored image is fine but......

Having a play with Macrium Free on my older laptop where I have just been reinstalling W11.

I noticed on the restored image that Windows Indexing has reset the index and is going through indexing all the drive from scratch. Is that normal behaviour for a restore, not just with Macrium but any similar program? Maybe it is and I've just never noticed before.

I wanted to prove this to myself. I've just imaged my C drive which has around 30k of indexed files. I have done a full backup and then a restore. The index has been set back to zero and this will now take 'hours' to rebuild.

Has anyone noticed this before? and/or is it something that perhaps only affects the free version?

I beleive that the index database may well be one of the items that are excluded from images, along with .tmp files and the like, in order to reduce the image size. This makes sense as the index can be rebuilt and is not essential for Windows to run.
I am sure someone on the Macrium user forums must have asked about this before, although as a free customer I don't think you can post questions you can view the forums, so may be worth a look.

Thanks. I might try again with the VSS option set to 'exclude VSS writers'.

I'm still waiting for the index to rebuild. Its only half way there at the moment. To me these are pretty major 'issues' that should be documented so users know what to expect.

DigitalGoat said:

although as a free customer I don't think you can post questions you can view the forums, so may be worth a look.

Click to expand...

I've asked. I have a license for the paid version which I use on my main setup.

Index is still rebuilding after 3 hours. It's still barely more than halfway through.

You could speed the index rebuild up by not using the computer or by disabling Indexer backoff [the facility that slows indexing down while you are using the computer].
Enable or Disable Indexer Backoff - ElevenForumTutorials

Best of luck,
Denis

Thanks Denis.

I have just had a brilliant reply off Macrium. All explained here...

That was very informative. I suspect that the involvement of the VSS means that other system imaging utilities also have this 'limitation'.
- I alternate Macrium with a free sawn-off version of AcronisTI, 'Seagate Discwizard'.
Now I understand what the position is, I shouldn't think of it as a 'limitation' at all but merely as a feature - because enabling IndexerBackoff allows me to carry on working while the index is being rebuilt [and I don't notice any slowdown in system responses during the rebuild].

What a very interesting thread,
All the best,
Denis

You can also speed up the indexing by setting your indexer to only look for needed and prefered files types, spend some time in the indexer options customising it, I managed to get the index database down from around 1.4GB to merely 250MB by excluding locations and file types I have no interest in indexing, as you can imagine this index completes in seconds on a rebuild. Your mileage may vary as not everyone wants the same data indexed of course.

Try3 said:

That was very informative. I suspect that the involvement of the VSS means that other system imaging utilities also have this 'limitation'.
- I alternate Macrium with a free sawn-off version of AcronisTI, 'Seagate Discwizard'.

Click to expand...

I suspect so yes. If the VSS do-dah is a Windows feature then all these backup utilities will see the same (I imagine so anyway)

I still don't fully understand it all because I was under the impression that once you hit the image button all these files and processes are taken as a snapshot and backed up as they stand at that moment. If anything subsequently changes then that doesn't make it to the image file but the PC still sees those changes in real time as you use it. When the backup is finished the PC is released back to normal.

Is the risk that something like indexing might be running at many items a second if it happened to be running at that time (if you were adding or deleting stuff for example) and that in the short interval when the snapshot is taken that the index and files might end up out of step.

I really don't know on that.

Try3 said:

You could speed the index rebuild up by not using the computer or by disabling Indexer backoff [the facility that slows indexing down while you are using the computer].

Click to expand...

Thanks, its useful (to me anyway) to note all this down so its handy if needed.

DigitalGoat said:

You can also speed up the indexing by setting your indexer to only look for needed and prefered files types,

Click to expand...

That's something I might look at. One of the programs I use has zillions of very small text files of data and that is where a lot of 30k of indexed files comes from.

Those files are slow to copy to a flash drive as well. When it reaches these the free space on the drive barely alters as they all copy across. The transfer speed goes from 30Mb or whatever down to a few kB for all these. These are what fill the index up I think.

The majority of flash drives use pretty terrible memory (slow) and controllers, they are cheap for a reason.
Spending more can get you better memory/ controllers that can actually write at decent speeds, especially when writes are large files (so sustained writes) but small files will always cause a slowdown simply because of the way the file system works combined with how flash memory works.
If you are copying lots of small files regularly and they arn't needed often from the flash drive then consider zipping them into a folder to copy, even with no compression the resulting zip file should copy quicker, relatively.

Mooly said:

I still don't fully understand it all because I was under the impression that once you hit the image button all these files and processes are taken as a snapshot and backed up as they stand at that moment. If anything subsequently changes then that doesn't make it to the image file but the PC still sees those changes in real time as you use it. When the backup is finished the PC is released back to normal.

Click to expand...

Yes, but VSS [as you know] can omit specific files.

Mooly said:

Is the risk that something like indexing might be running at many items a second if it happened to be running at that time (if you were adding or deleting stuff for example) and that in the short interval when the snapshot is taken that the index and files might end up out of step.

Click to expand...

It's not so much a risk as a decision that has been taken to avoid certain files.
VSS could cope with including the index file(s), as that MS article explained, but they've chosen not to.

Mooly said:

One of the programs I use has zillions of very small text files of data and that is where a lot of 30k of indexed files comes from.

Click to expand...

Does 30KB matter to you?

Mooly said:

Those files are slow to copy to a flash drive as well.

Click to expand...

Is that a regular task? Consider using the built-in command RoboCopy to speed the job up - you can use it to copy across only changed files. Just as an example, I use RoboCopy to copy files between my computers.
If you're interested then I can post some useful links.

All the best,
Denis

DigitalGoat said:

If you are copying lots of small files regularly and they arn't needed often from the flash drive then consider zipping them into a folder to copy, even with no compression the resulting zip file should copy quicker, relatively.

Click to expand...

The program that uses all these files is a bit of an oddball in the way it works/installs. These files are part of the programs actual install in c:\Program files. The files are duplicated in a folder in Documents (all by design) which allows the user to use and alter them without corrupting the original install... bit weird isn't it.

The programs installer warns not to use the OneDrive Documents folder as OneDrive is known to complain at all the small files.

Try3 said:

Is that a regular task? Consider using the built-in command RoboCopy to speed the job up - you can use it to copy across only changed files. Just as an example, I use RoboCopy to copy files between my computers.
If you're interested then I can post some useful links.

Click to expand...

As well as image backups (which are fine and quick) I sometimes flirt all the Documents contents over to Flash Drive as a secondary backup of my own stuff. Lately I have started being more selective and having selected 'everything' I go back and unselect these folders.

Copying to traditional drives is way faster than flash memory I have found, and this is not so much an issue with these.

This is an example file, you can see how small it is. They are really just text files but with a different extension. There are many thousands like this. Probably half of the 38k here are these type of file. In total they occupy only a tens of MB

Mooly said:

The program that uses all these files is a bit of an oddball in the way it works/installs. These files are part of the programs actual install in c:\Program files. The files are duplicated in a folder in Documents (all by design) which allows the user to use and alter them without corrupting the original install... bit weird isn't it.

Click to expand...

If they are just copies of files in the installed folder, perhaps their inclusion in your system images is the only backup that you need for them?
I do backup to an SD card in addition to my proper backups [the SD card slot just sits there so I just decided to make use of it] so I do a rough equivalent of your flirt backups. I use RoboCopy to do this so I can just re-copy the changed files. The whole lot is about 300GB because it includes my audio files but it normally takes less than a couple of minutes because it only bothers with the changed ones [I just timed one routine run - 1 min 29 secs].

I limit my indices more then you do. I do not index the whole of the C:\Users folders. I select specific folders so that I can omit unwanted Documents sub-folders created by applications & filled with unwanted rubbish of their choosing.

By the way, I have looked further into VSS & excluding folders/files from imaging. I came across this Macrium Reflect KB article
How to exclude files from a Disk Image - MRKB
which refers to another MR Registry Sub-Key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumImage
and I also found it interesting to look through the other Sub_keys of
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore
which, I think, affect other snapshots & images.



All the best,
Denis

Try3 said:

If they are just copies of files in the installed folder, perhaps their inclusion in your system images is the only backup that you need for them?

Click to expand...

Disc imaging of the whole C drive is very quick, they seem to have no impact that I'm aware of on that. I guess same for restores as they are really quick as well.

Copying my user files in their native form to a traditional USB 3 1TB spinner is pretty quick, it flies through them. Do the same with the USB 3 (Sandisk) flash drives and its around 30 minutes of a job.

I've actually very little in terms of size to back up. 22GB of documents many of which are pdf's No music or vids on this setup and around 4Gb of pictures.

Thanks for the link... it's interesting having at least an idea what is going on. I might try that Macrium reg key at some point on my test setup. I did try a full image and restore with the single tick box option to disable VSS writers and it all seemed perfect. Index was present and correct and SFC reported no issues on the restored image.

I've tried many Backup/Restore programs over the years, but as stated above, they can get very detailed and complicated.
And, loaded with H&D (hate and discontent).

Since 1997, I've been using a Backup/Restore program simply called "GHOST", as my everyday backup program.
I start my weekly backup by thoroughly cleaning up my Windows OS (any Windows from 98 to 11.)
Once my PC is clean and properly shut down, I boot up the PC with my GHOST CD or FlashDrive. When the Ghost program comes up with its menu, for my weekly backup, I select the Partition to Image option, then specify where to put the backup Image file and tell it to Start the backup.

During the backup, every bit of info (ones and zeros) in the C: partition is read, compressed and written into the Backup Image File, and written where I have designated. Ghost doesn't care what's in the designated partition. It could be an active OS, or even a damaged OS, or just data. Whatever is there, will be copied into the Backup Image File.

I've used the DOS version of Ghost, (ver. 11.5) for over 24 years, with every OS since Windows 98. It's been 100% Safe and Effective, since all the files in the C: partition are closed and inactive, when I do the backups, from a DOS boot disk.

Symantec Corp. that bought the program from its original authors, has since then abandoned it. But, it has survived, floating around out there in the WWW as an ISO for the boot disk. I have that bootable ISO burned to CD's, Flash Drives, and even an SD Memory card.

It just WORKS, with no muss or fuss. Doing a Ghost backup of a C: partition, followed by a Restore, re-writes the entire C: partition, file by file with NO Fragmentation and no dead space between files, for the Perfect DISK DEFRAG.

Happy Holidays, Mates!
TM