Macrium Reflect image backup with exclusions

Hey, So how long do you keep your full backups? 90 days? 180 days? Why do you say one-time? Why do you say temproary?
What if you want to roll back a service pack? Or malware / crypto lockers that has been present for 1-2 months? Or perhaps prior to deletion of data? Or perhaps (quite common) roll back to a prior accounting package update? Or restore and bring a system online that has been decommissioned? I can go on!

I don't know anyone who needs retention policies of one backup. Just like the "delete before backup" default for retention policies - worst idea ever! What if it fails? What if it is not noticed for 2-3 weeks (2-3 full backups) later? You've lost a lot of data.

But at the end of the day, what is data protection worth? Time spent learning a new system, dealing with teething issues, documenting for others, migrating clients, etc. Vendors / Dev's know there is resistance to change and it works in their favour for client to stay...

hsehestedt said:

Since this backup would be a one-time temporary backup and will only require space until you either restore that backup or you know that all is good and you no longer need it, does that change things for you? You have enough storage space available that you can make a temporary backup without having to exclude anything?

Click to expand...

Flipmode said:

Why do you say temporary?

Click to expand...

That was an error on my part. I was mixing up two different posts. In the other post was a discussion regarding doing a backup prior to attempting an upgrade to Windows 11. The backup would have served as a recovery mechanism in case the upgrade went absolutely wrong and would no longer be needed once that was successfully completed.

Apologies for the mix-up.

I am not a fan of excluding non temporary folders from a backup.

With VX (at least), it is not understood by many that when you do create a folder placeholder with no files in it.

So if you do a restore, it restores a blank folder AND wipes any existing files on that folder.

Excluded folders should be considered to be temporary only.

If you do not want to backup a folder, it is safer to simply have that folder in a separate partition that is not backed up.
Alternartively, backup the folder anyway if you do not want to risk losing them.

So do some tests in any version to make sure the restore behaviour is as you expect.

cereberus said:

With VX (at least), it is not understood by many that when you do create a folder placeholder with no files in it.

Click to expand...

I think you forgot to finish the sentence?
Wouldn't you perform a selective restore? Empty folders (AGAIN) should be an option to exclude, just like numerous Sync software from 2000's have as options under their VSS settings (like SyncBack SE/Pro).

If it is FREEWARE then I understand not having such "advanced" functionality, but I don't think you understand this is expensive subscription software now. It is time more enterprise-grade functionality and options became available.

If you exclude paths from a full-backup, it simply increases the speed / performance of backups, requires fewer resources etc. VSS snapshots the disk, copying that data from the shadow copy to a repository and excluding paths creates minimal extra CPU load but dramatically reduces the I/O.
It is so beneficial for low-spec machines, for performing more regular backups of multi drive systems. Plus there is no point backing up 400GB of OneDrive when it is already backed up to Onedrive with version control (no point filling up your Repo with this on each computer!). Plus backing up Exchange mailboxes, databases, etc is pointless when the whole objective is to backup the major component and apps running the machine, and recovering those components should there be a failure - bad sectors, failed arrays, viruses, malware, accidents, etc etc etc.

You are MUCH wiser to backup the system at a set frequency, but *critical* data be backed up at a different interval with greater version control.
I love how everyone has an opinion about NOT being able to specify exclusions when LITERALLY EVERY OTHER S/W suite offers this ability!

What I want to see from Macrium is a centrally managed interface! Where you can push out jobs, create groups, adjust schedules, et al. This is what made GFI Backup v4 absolutely spectacular, while also being web-based (unfortunately partially ioncube encoded) so you could make changes. Anyway... Installing a MR SAAS on a VM (vSphere etc) to manage all devices on the network is the avenue Macrium should be looking at - whether homes with a few devices, small businesses, charities, government and enterprises. That would really have them exceed all competition!

Flipmode said:

I assume all they have done in version 10 is finally map this registry path to the application, as they had already done in version 8 for folder backups.

Click to expand...

No, they have done a lot more than that for Reflect X.

Macrium said:

We've created a new Macrium VSS writer that enables exclusion of unwanted files from VSS Snapshots. Using this filter, you can exclude various standard file categories, specific files, or apply wildcard filters recursively to selected folders.

Click to expand...

Those registry paths you quoted are not the one Reflect 8 uses for excluding files/folders from images. It's this:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumImage

Reflect 8 relies on the Microsoft VSS Optimization Writer, and that had some lmitations.

Macrium said:

Microsoft VSS includes the ability to exclude file patterns from snapshots. Although this can be useful in removing unnecessary files from images and clones the functionality isn't 100% reliable. Removing files is reliant on the VSS Optimization Writer and the if the files are not removed before the VSS timeout time has elapsed then snapshots may not exclude all the desired files....

Click to expand...

---edit---

original pages no longer available, links updated to archived versons.

Flipmode said:

Registry Hive Configuration
Created 3x Multi-String Value entries under:

HKLM\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumFile\​

HKLM\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumImage\​

Click to expand...

I am confused? They are the exactly the registry paths I had specified. One for File/Folder backups and one for Disk Images.
I also specified which paths were not being excluded or ''processed'' in blue.

Can you please try excluding those same paths, and see if they are still not included in your snapshots / backups??
Thanks

Bree said:

Those registry paths you quoted are not the one Reflect 8 uses for excluding files/folders from images. It's this:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumImage

Click to expand...

No, it is an expression. Obviously they've done "more" in regards to coding / features etc. But as this topic specifically discusses one specific issue, therefore YES, all they have (finally) done in v10 is map the VSS registry paths and link them to the same menu but under disk backup, to get the functionality to work properly.

Maybe I will install MR 10 (or "X") in Sandbox and then see what it does when I enter my exclusions.

I was really hoping someone could actually answer my query. I documented all settings to make it easy, but I am confused why you're telling me I am using the wrong path?

cereberus said:

If you do not want to backup a folder, it is safer to simply have that folder in a separate partition that is not backed up.
Alternartively, backup the folder anyway if you do not want to risk losing them.

Click to expand...

Thanks Cerberus. Will test restore. Just come back to MR issues as another wave of clients wanting to follow the exodus from Acronis.

Bree said:

.....

Click to expand...

That's why I showed the TEMPORARY paths that failed in my post. Did you understand it? Maybe I wasn't clear enough?
These are all temporary pathsI I am trying to exclude yet are still being backed up. These numbers are for a NEW PC! Imagine a 3-4yo PC?

Flipmode said:

It is so beneficial for low-spec machines, to perform more regular backups. Plus there is no point backing up 400GB of OneDrive 100GB of Outlook etc (when Doc's is already backed up to Onedrive with version control (no point filling up your Repo with this on each computer!)). Plus backing up Exchange mailboxes, Lotus Notes / SQL and other databases, etc is pointless when the whole objective is to backup the major OS component, drivers, app data and apps running the PC, and recovering those components should there be a failure - bad sectors, failed arrays, viruses, malware, accidents, etc etc etc.

Is is wiser to backup the system / OS at a less frequent frequency, but *critical* data be backed up at a different interval with greater version control.

Click to expand...

To everyone,
For example: OST and PST files. Exchange files. Databases. Accounting packages. It's feasible to repartition or reinstall S/W packages & DBs to other partitions. I should not have to partition 10 or 100 computers, and then regularly monitor & juggle free space for OS or data. Who is going to pay for my time?? Client won't want to - it's not broke! Certainly not government clients. And my not-for-profit's and community charities pay nothing - I do it all free-of-charge. They get s/w discounted or free too.

MR is paid software. They didn't release v9 thus should be implementing these stock standard features in v8. I know they write 100s of times in the forums to update to v10 but then what was the point of paying a premium for lifetime licences if functionally deficient??

I should be able to keep each User's Data backup that collects all roaming data, some local data paths (bad Dev's putting config/data in wrong locations), photos, media, documents, Outlook databases and other databases. Not have it duplicated and fill up backup repositories much quicker.
Having a properly functioning exclusions / inclusion filter means you do not have to shift every existing computer's setup over to another partition. Plus you are only spinning the barrel of the gun - having to re-partition a dozen computers, I guarantee one will have issues or inconsistencies and destroy your plans for the day....
Being able to backup 5-10 GB PST files in a separate backup profile, eg. hourly, is MUCH more beneficial than attempting to do a full PC backup every hour. Do you guys work in IT or just home PCs? Do you guys not understand network congestion? And why would you want 5x 10 GB PST plus 2-3 10GB archive PSTs choking up your daily backups, and being backed up at an unnecessary frequency?? I give up...

I do not understand resistance from other members. I am asking for help on a specific issue that other people have raised (this topic).

Can everyone please start sharing their MR hacks, hints and tips?
Thank you

cereberus said:

I am not a fan of excluding non temporary folders from a backup.

With VX (at least), it is not understood by many that when you do create a folder placeholder with no files in it.
So if you do a restore, it restores a blank folder AND wipes any existing files on that folder.

Excluded folders should be considered to be temporary only.

If you do not want to backup a folder, it is safer to simply have that folder in a separate partition that is not backed up.
Alternartively, backup the folder anyway if you do not want to risk losing them.

So do some tests in any version to make sure the restore behaviour is as you expect.

Click to expand...

Yes, I only exclude temporary paths in my examples, other than OneDrive.

Does MR have any intention to fix this destructive issue in version 10???
There is absolutely no need to destroy folder contents, if selectively restoring, when there is nothing to overwrite that folder with. If it's an empty folder, it's an empty folder. Does that mean any new folders created when restoring will be deleted?
Or are you talking about a full restore?

It is almost like they forgot a switch or option? Perform destructive restore (Default)? Or Restore and Merge (most common default in S/W I have seen that support partial/selective restore).
Has this issue been reported and any knowledge of a fix in v10?

I might do 3 backups on a VM — test with the 1) in-app exc/inc, 2) registry paths, and 3) untested/undocumented registry paths I posted at bottom of last page.
Cheers Cereberus (isn't it spelled Cerberus?)

Flipmode said:

Yes, I only exclude temporary paths in my examples, other than OneDrive.

Does MR have any intention to fix this destructive issue in version 10???

Click to expand...

White text tends to be unreadable when in light mode. I shouldn't have to select it to be able to read it

Bree said:

Click to expand...

Huh? I can't see what you said...

Just as a follow-up, in Macrium Reflect X 10.0.8750 you can't configure any Exclusion at the time of performing each backup (unlike you can do with Compression, for example). You have to go to Defaults in Settings and change the exclusion list each time before each backup, in case you want to change something (you don't need to set it each time from defaults since settings are saved). You can exclude specific files or folders recursively.

When I make an image, I want everything.

I find that exclusions are important to me on laptops that have only a single drive. I used to divide up my SSD into my Windows partition as well as a separate data partition. But the problem with that scheme is that my needs change over time so invariably I end up having to make the data partition larger or smaller. Even then, I always feel like I am wasting space putting too little or too much space in one of the partitions.

One such systems, I really like just having one great big partition and then excluding the stuff I don't need to backup. For example, I always have a lot of VMs configured purely for testing. I don't need to back those up and they account for a large amount of space. In addition, my VMs folder changes every single day so my daily incremental backups would be huge if I didn't exclude those backups.

As for being able to set a different exclusion for each backup, I don't know if this would work, I would have to test it, but it may be worth a shot: You can save your backup definition as a DOS batch file, PowerShell script, or a VB Script. I wonder if exclusions are saved in those definition files at all. I'll try testing this later when in front of another machine.

EDIT: I checked the docs and it does not appear that there is any provision for specifying exclusions at the command line, so I guess that idea was a bad one. Maybe in version 11 .

I have used the macrium reflect free for several years. Discovered this registry key recently "\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshotMacriumImage"
My experiment with this key is kind of hit and miss. In my multip string values, I add these folder location on my c drive:
C:\ProgramData\Microsoft\Windows\Containers\*.* /s
C:\Users\homeuser\.minikube\*.* /s
C:\Users\homeuser\.lmstudio\*.* /s
BTW, I am doing disk image back of my whole drive. The result image file still have these 3 folder and most of their folders, while "C:\ProgramData\Microsoft\Windows\Containers" still have all its contents including subfolders, i.e. same size. The other 2 have their size reduce greatly. I can't figure it why it won't work on the "C:\ProgramData\Microsoft\Windows\Containers" directory. I even tried to move the order of these 3, and the same results.

A little update. I can't get MRF v8 to exclude the "C:\ProgramData\Microsoft\Windows\Containers\*.* /s". I tried to have it in its own multiple string values, put it at the first/last of a group other folders, which all worked to certain extend except this directory. My guess is MS VSS just refuse to exclude this directory.

I give up on MRF v8 and will try a different free tool next week.

Everywhere I read states that deleting the "Containers" folders is a dangerous thing to do (failed apps and System instability)... and that's exactly what you'll do if you ever restore the image that no longer contains that folder. That's why VSS may be keeping you from doing it.

@Froggie

Andy Devine, "Plunk your magic twanger, Froggie!". You must have been a kid in the early '50's to know that.

BTW, when I click on your location "The Pond - USA" it takes me to The Duck Pond in the city of Atlanta, GA, USA.

I just reread the whole thread. A lot has changed since my first post. I'm now using Macrium Reflect X. When I make my image backups, I back up everything. I see no reason to exclude anything.

I keep everything on my 1TB C: drive. All my data and photos are kept in my OneDriver folder, and I keep Files On Demand turned OFF so that my cloud OneDrive and my local OneDrive are always in real-time sync. That way when I make my nightly Macrium Reflect X image backups I'm including all data and photos in my image. Reflect X is much faster, and my backup images now take only about 3 minutes to make.

I rotate complete backup images on two 4TB Samsung T9 portable SSD drives. This gives me enough local storage for a month of rotating daily backup images.

My System One and System Two both log in to the same Microsoft 365 account and the same 1TB of cloud OneDrive. This allows me to transparently go back and forth between using either computer while always maintaining two identical local copies of my data, and very importantly, maintaining a complete offsite real-time copy of all my data and photos in cloud OneDrive.

That's my 3-2-1 Backup Strategy.
3 copies of my data using
2 different media types with
1 copy always kept offsite.

I think there is a bit of a misunderstanding by some people about how file and folder exclusions work, and it is important to understand how it works.

Suppose you exclude a folder from an image backup e.g. to prevent image backups being too large.

f you restore that image backup for some reason, the folder that was not backed up will be deleted!

This is basically because an image backup restore restores sectors of data rather than specifically restoring files individually. By excluding files, the sectors containing those files are marked as blank in effect, so upon a restore, anything held in that sector gets wiped.

Note the above is a simplification but shows the principle.

In the end, files and folders should really only be used for temporary files/folders (default in VX).

If you really want to exclude permanent file/folders in image backups, then make AT LEAST one separate backup of the file/folders to be excluded as a safety precaution. Indeed use File and Folder backup for those file/folders is a pragmatic solution.

In all honesty, it is simply easier to keep such files on a separate drive (still noting that files should be backed up anyway separately in case drive fails).