Reliable Free Backup Software

johnlgalt said:

You should pay a bit more attention. You're using articles written 14 years ago. and apply to Windows XP and Server 2003.



Perhaps more relevant would be the more current articles on VSS and relevant services.

Volume Shadow Copy Service

Learn more about: Volume Shadow Copy Service

learn.microsoft.com

Volume Shadow Copy Service - Win32 apps

Back up volumes while applications on a system continue to write to the volumes. Minimize application downtime by quickly creating a snapshot (shadow copy) of a volume that duplicates all data. Perform multivolume backup.

learn.microsoft.com

VSS Application Compatibility - Win32 apps

When developing your own VSS application, you should observe the following guidelines and restrictions.

learn.microsoft.com

And finally

Using the Volume Shadow Copy Service - Win32 apps

VSS backup and restore operations each use a protocol for the interaction of the systems that use mass storage (writers) and those that back it up (requesters).

learn.microsoft.com

Which links to several more.

Even for XP and 2003, there is an updated version of the VSCS dated this year available for download:

Download Volume Shadow Copy Service SDK 7.2 from Official Microsoft Download Center

Software development kit for the Volume Shadow Copy Service.

www.microsoft.com

And how to configure it in the modern computing era:

Configuring VSS - Win32 apps

By using VSS configuration tools&\#8212;including setting the shadow copy context, choosing a restore method, and changing a restore target&\#8212;writers and requesters control how a backup or restore operation takes place:

learn.microsoft.com

With different configuration for both backup and restore:

VSS Backup Configurations - Win32 apps

There are a number of conventionally supported backup types&\#8212;incremental, differential, and full&\#8212;that VSS is aware of, as well as a backup configuration peculiar to VSS.

learn.microsoft.com

VSS Restore Configurations - Win32 apps

File restoration on a running system can be problematic. It is important that running applications (writers) indicate what to do when difficulties arise during restores, for instance, if the file being restored is currently in use.

learn.microsoft.com

And of course, backup strategies, including how it all works, on modern computer OSs:

Overview of Processing a Backup Under VSS - Win32 apps

In processing a backup, requester and writers coordinate to provide a stable system image from which to back up data (the shadow copied volume), to group files together on the basis of their usage, and to store information on the saved data.

learn.microsoft.com

It's great you've seen it fail - hell, any backup solution has the ability to fail. But referencing 14 year old information to prove a point when the difference between then and now relating to VSS is almost night and day is hardly a viable argument for what you're saying.

Sorry, but I still disagree, and particularly when there is still no evidence of a Macrium backup showing unreliability due to inherent software issues (versus misconfiguration).

Click to expand...

When a VSS backup operation is conducted without the involvement of a writer, the shadow copy creation can still occur.

As noted elsewhere (see Default Shadow Copy State), the result of this type of shadow copy is a volume reflecting the state of a disk at the time of the shadow copy: data on the shadow-copied volume may reflect incomplete or partial I/O operations and is described as being in a crash-consistent state.

There are several situations that will require a backup application to work with crash consistent shadow copied data:

• Data is managed by VSS-unaware applications
  Almost every system will have some applications—text editors, mail readers, word processors, and so forth—that are VSS unaware, so it is always likely that some of the data on a shadow copy will need to be thought of as being in a crash consistent state.
  This sort of data is not typically system- or service-critical, so backing it up should not be problematic, or at least no more problematic than during a conventional backup.
  As with preparations for conventional backups, if possible, backup operators should attempt to suspend or terminate such applications prior to starting a VSS backup job.

{...}
So, while this sort of data is not typically system- critical or service-critical, it still potentially can be. Now, let's suppose you have the IT admin skills to verify that none of the applications and background processes that are still actively running on the system can still write this sort of data on the volume in question. What will happen next is, you still have to keep in mind that the shadow copy that results will not be application consistent, which can tend to be problematic to applications in the possible event that this sort of data includes application-critical data like I said.

Finally what happened is that, basically, Microsoft repeated almost every word from 14 years ago in this much more recent article. As you can see, the first link in that one is this: Shadow Copies and Shadow Copy Sets - Win32 apps
Even though the file system flushes all I/O buffers prior to creating a shadow copy, this will not ensure that incomplete I/O is properly handled.

Therefore, assuming that the system has no VSS-enabled applications, the data in a shadow copy is said to be in a crash-consistent state. A shadow copy in a crash-consistent state contains an image of the disk that is the same as that which would exist following a catastrophic system shutdown. All files that were open will still exist on the volume, but they are not guaranteed to be free of incomplete I/O operations or data corruption.

While the crash-consistent state does not fully deal with all the issues associated with defining a stable backup set (see Common Volume Backup Issues),
{...}
Again, this boils down to the same words from 14 years ago, all simply because these words are still very much relevent today, which is fairly obvious. In an old thread (here on ElevenForum or on TenForums, I don't remember which) several months ago someone asked if making a hot backup of the Windows system partition (by using the same or similar type of strategy/method that most people here and on TenForums typically use to do it) is essentially equivalent to forcing a hard reset (or pulling the power cord out of the power supply unit while Windows is still actively running on the PC) followed by making a cold backup of it, when it comes to the reliability of this sort of backups. The only person who replied yes was me.

Simply almost no one else cares to just search and read up on a lot of stuff anymore or it sure looks that way sometimes. Sometimes often. Right now I'm having troubles with my ISP because I called customer service (twice) only to find out that they consistently (...) refuse to understand their own hardware that they provide. You don't have to convince me that Microsoft could still be wrong about their VSS, though... after all, they're the ones who invented the mickey so, that company will likely always remain full of surprises. lol Microsoft screwups (plus the jack-in-a-box double-click speed tester, IMO the latter was reliable...) were among main reasons why, a very long time ago, I had switched my career, from writing data-centric desktop applications with Visual Studio 2000 in Visual Basic 6 using things like ADO 2.5 on Windows 2000 Pro, to Eclipse with J2EE so, currently I am a Jakarta EE software developer.

And when you're making a backup of an offline system, issues can still occur - power fluctuations, noise, and more. SO, should ever home user also have backup generators to prevent power outages, and voltage regulation equipment as well?

You need enterprise level backups? Great. Others here do as well.

Not everyone needs it. And there is not a single backup system in the world that has not failed at one time or another. Just because it may not have failed for youdoesn't mean it has not failed.

When you're making a backup of an offline system (what they call a "cold" backup), it means that after the backup preparation starts no data will continue to be written to the OS system partition (that will be included in the backup). This is due to the OS (that has been installed on the partition) having been shut down first, i.e. as the first preparation step. This continues to be true all the way until after the backup verification went successful, after which the OS may be started up again because the backup was successful and also because the verification the purpose of which was to guarantee that the data contained in the backup matches the data stored on the disk volume(s) was successful. Some exceptions can be made to this, e.g. by choosing to let the backup be created in a destination folder that is located on a partition that will be included in the backup, which isn't a recommended choice, but nevertheless it is an option of choice that the backup tool software might be able to give.

Similarly, the verification step is also optional so the user can decide, but if your first goal is for backups to be reliable, then of course you want to use that option. With the Acronis, it reads all the data again, which it does the same way as when you do a restore, so it matches the data contained in the backup to the data already present on the disk as opposed to writing the data from the backup to the disk. Due to the fact that this is the only difference between the verification step and doing an actual restore, this verification method is able to provide the kind of reliability that is freed, as much as would technically be possible, from unknown risk factors that could potentially interfere. The chance of a read error going twice undetected on a storage device is really very small.

To mitigate the potential risks of hardware failures, power outages, etc. you could always decide to combine additional protections with the proven concept of making backups in such a way that you'll know you'll have a handle on how reliable they are going to be as opposed to persistent cyclic human error from the likes of not reading the docs from Microsoft while putting all the blame on those who did read them to weed out all the guesswork that some people keep propelling (like, till hell freezes over twice). The key purpose of a backup is not to prevent getting hit by lightning, but to prevent losing data permanently e.g. as a result of a whole RAID array going up in thin smoke with just a single spark (which looks kind of cool, but also looks kind of expensive...). But if, for example, the backup and/or restore performs so terribly slow that this introduces a whole new set of other important problems, then obviously, it can make sense to improve the speed, and it can make sense to combine backups with the concept of making the kind of snapshots that can't replace backups. In the latter case it can also make sense to, in addition, make (short-term) backups of the kind of snapshots that can't replace backups. But the average home user doesn't usually know, or need to know anything much about that particular subject. For the average home user, the least unconvenient way to avoid pitfalling as a result of inconsistent backups is to make reliable cold backups first and foremost. It helps to assure in a profound manner that the backups will be consistent, as much as possible without requiring to use more complex skills for validation and testing that are associated heavily with the the sort of IT admin skills that, clearly, you do not possess.

Some say having to boot from a USB flash drive and waiting for the backup to finish before booting back into Windows isn't very practical. But then, they typically never say anything about running the potential risk of having the sort of application-inconsistent data in backups that initially appears not to be problematic, but that can still nevertheless start to wreak havoc to an application, some time after doing a restore. Rather, they tend to base their decisions on guesswork, and they continue to rely on guesswork even after the company that built the VSS wrote articles about it, and they still continue to rely on guesswork even after some guy on the internet already explained the articles to them. Reliability usually starts when people stop touting their own guesswork as fact and they actually spend the extra time and effort to correctly investigate what's causing their own reliability (and that of the backups that they love to make) to potentially be flawed when flaws could have easily been avoided by learning to understand the risks, also including the added risk of falling victim to misinformation/desinformation from those who clearly never bothered to look for these articles, let alone read them.

Hi everyone, my first post here.
I've been using Disk2vhd(x) for ages, before being acquired by Microsoft. Now version 2.02. And about two years ago I found a problem with it that's closer to be a bigger one as time passes:
VHDX files can be 64 TiB maximum, right. But when using Disk2VHDX to create an image of just one partition (up to 64 TiB, so doable) BUT inside a Physical disk bigger than 64TiB, the resulting VHDX is made but not valid (can't be mounted because "it's too big"). The reason is that although you select one partition smaller than 64TiB, disk2vhdx takes into account the rest of the partitions (although doesnt copy them and leaves them empty) so the resulting VHDX is bigger than 64TiB and not valid.
In other words Disk2vhdx copies the structure of the whole physical disk, but just the data of the partition/s you select. So if your physical disk is over 64TiB, the resulting VHDX complete structure, is not valid even if the size of the partition you're interested were just 1KB.
I love disk2vhd and would not like to rely on third party tools as the OP said, unless they are the only option. Also because time for me to worry about this is not yet too close.

Thanks

mfuentes said:

Hi everyone, my first post here.
I've been using Disk2vhd(x) for ages, before being acquired by Microsoft. Now version 2.02. And about two years ago I found a problem with it that's closer to be a bigger one as time passes:
VHDX files can be 64 TiB maximum, right. But when using Disk2VHDX to create an image of just one partition (up to 64 TiB, so doable) BUT inside a Physical disk bigger than 64TiB, the resulting VHDX is made but not valid (can't be mounted because "it's too big"). The reason is that although you select one partition smaller than 64TiB, disk2vhdx takes into account the rest of the partitions (although doesnt copy them and leaves them empty) so the resulting VHDX is bigger than 64TiB and not valid.
In other words Disk2vhdx copies the structure of the whole physical disk, but just the data of the partition/s you select. So if your physical disk is over 64TiB, the resulting VHDX complete structure, is not valid even if the size of the partition you're interested were just 1KB.
I love disk2vhd and would not like to rely on third party tools as the OP said, unless they are the only option. Also because time for me to worry about this is not yet too close.

Thanks

Click to expand...

For files up to 64TB why not simply use within windows in diskpart

create vdisk file=<x:\path to file> type=<fixed / expandable> maximum=<max size in MB>
then attach vdisk
select disk xxx
then format it how you want, can create efi partition if required etc etc.

you can just back this up with file explorer or whatever.

Not sure why you'd want single disks of 64TB though - but thats not my call.

I run loads of different windows systems from vhdx files too. For example.

Also with vhdx files you can run the OS'es easily as Windows to Go devices i.e from an external device.


cheers
jimbo

mfuentes said:

Hi everyone, my first post here.

Click to expand...

Welcome to Eleven Forum.

mfuentes said:

VHDX files can be 64 TiB maximum, right. But when using Disk2VHDX to create an image of just one partition (up to 64 TiB, so doable) BUT inside a Physical disk bigger than 64TiB, the resulting VHDX is made but not valid (can't be mounted because "it's too big"). The reason is that although you select one partition smaller than 64TiB, disk2vhdx takes into account the rest of the partitions (although doesnt copy them and leaves them empty) so the resulting VHDX is bigger than 64TiB and not valid.

Click to expand...

It's called Disk2VHD (without the 'X') - an easy typo to make as we hardly ever use a .VHD these days.

You are correct, Disk2VHD creates a dynamic .vhdx that contains the layout of all the partitions from the source disk, but only populates those partitions you have selected for copying.

That's a situation that few (if any) can reproduce and test. I suspect not many of us have a physical disk that's larger than 64TiB (65,536GB). Nor for that matter are many of us running Windows Server 2019. But as you are, then there's one possibility you could try.

You should have Hyper-V Manager available to you. You could try using Edit Disk... from it's Actions pane. It may be able to resize the virtual disk to something smaller that 64TiB. I am in no position to test this for you, but it's worth a try.

Bree said:

Welcome to Eleven Forum.

It's called Disk2VHD (without the 'X') - an easy typo to make as we hardly ever use a .VHD these days.

You are correct, Disk2VHD creates a dynamic .vhdx that contains the layout of all the partitions from the source disk, but only populates those partitions you have selected for copying.

That's a situation that few (if any) can reproduce and test. I suspect not many of us have a physical disk that's larger than 64TiB (65,536GB). Nor for that matter are many of us running Windows Server 2019. But as you are, then there's one possibility you could try.

You should have Hyper-V Manager available to you. You could try using Edit Disk... from it's Actions pane. It may be able to resize the virtual disk to something smaller that 64TiB. I am in no position to test this for you, but it's worth a try.

View attachment 70727

View attachment 70726

Click to expand...

Thanks Bree

I know it was originally without X, just that I don't use VHD anymore, always VHD X: Elon Musk told me so
Even I call now my 2015 Nissan Leaf, Nissan X .
And you are correct too: I also use HyperV as a tool for VHDX, but I obtain the same error as mounting the VHDX with DiskManager below ("Not Valid" or "too big")

Maybe this screenshot makes thigs more clear

Disk1 (64TiB) No problem whatsoever with Disk2vhd and partitions D: F:. I can mount and use those resultant VHDX.
Disk2 (>64TiB) I can create a VHDX for partition G: (completed successfully) with just 10GB, but then cannot mount or use such VHDX. Although tiny, It was made when it was inside of a physical disk too big for the VHDX format to handle.

This is done in a new server I'm testing. The previous one is much smaller but has a 64 TiB disk with 40 TiB of data, so I must use block backups (which I quickly upload to other servers or the cloud) and not file backup for that (about 100 Million files). I know most people can't simulate my issue (for now).

Thanks again
Manuel F.

Anyone try Hasleo Backup Suite Free ???

mfuentes said:

And you are correct too: I also use HyperV as a tool for VHDX, but I obtain the same error as mounting the VHDX with DiskManager below ("Not Valid" or "too big")

Click to expand...

Don't try to mount it. Leave it unmounted and see if you can edit the vhdx using Hyper-V Manager to shrink it to less than 64TiB. If that works (something I can't test for you) then it will be mountable.

KWEB said:

Anyone try Hasleo Backup Suite Free ???

Click to expand...

@mfuentes doesn't want to use any 3rd party tools, just those tools provided by Microsoft. I haven't tried Hasleo, but some others have tried it by now, cereberus for one...

so I must use block backups

Click to expand...

Have you tried using wbadmin to create the vhd(x) ?

SIW2 said:

Have you tried using wbadmin to create the vhd(x) ?

Click to expand...

As far as I know the limit for WindowsBackup Client is just 16TiB, too small for my case. Is Wbadmin any different?
Funnily I use it for V2P: I found a workaround and I can restore VHDX to Physical Disk in partitions (up to 64TiB obviously)

Bree said:

Don't try to mount it. Leave it unmounted and see if you can edit the vhdx using Hyper-V Manager to shrink it to less than 64TiB. If that works (something I can't test for you) then it will be mountable.

Click to expand...

I succeeded doing expand/shrink o viceversa in other circumstances (apparent corruption), but already tried unsuccessfuly. This is now:

mfuentes said:

I succeeded doing expand/shrink o viceversa in other circumstances (apparent corruption), but already tried unsuccessfuly.

Click to expand...

What you can't do in the GUI may be possible in Powershell.

One partial win with an idea thanks to your suggestions:

I abandoned disk2vhd for a moment and returned to WindowsBackup client (or wbadmin) in that +64TiB "problematic" physical disk, BUT this time I chose a <16TiB PARTITION (its known limit) and ... it worked. I obtained a valid VHDX (rather a Windows Backup structure with the VHDX inside).
I had to say it's half as fast as disk2vhd, but it's valid as it has just that partition inside the VHDX.

Now I just wish I could do the same with partitions bigger than 16TiB where I continue to get the infamous "Volumes larger than 16777216 megabytes cannot be protected"

Thanks to all of you

So my hope is that the next Diskvhd version will not create the size of the VHDX according to the physical size of the disk involved, but according to the sizes of the partitions you want to backup: That's what WindowsBackup does (with the apparently unnecessary limit of 16 TiB)

mfuentes said:

So my hope is that the next Diskvhd version will not create the size of the VHDX according to the physical size of the disk involved, but according to the sizes of the partitions you want to backup...

Click to expand...

I doubt that it ever will. The stated purpose of Disk2vhd is to clone a physical disk to a vhdx so that it can be used as a VM. That even includes the vhdx having the same disk signature as the original. It was never intended as a way to back up individual partitions.

SysInternals said:

Disk2vhd is a utility that creates VHD (Virtual Hard Disk - Microsoft'sVirtual Machine disk format) versions of physical disks for use inMicrosoft Virtual PC or Microsoft Hyper-V virtual machines (VMs)....

Click to expand...

Disk2vhd - Sysinternals

mfuentes said:

...That's what WindowsBackup does.....

Click to expand...

It always puzzled me what exactly was the type of disk Windows Backup used to store each of the partitions it had backed up, so I mounted all three of the vhdx it had created from an old backup of mine, a Legacy/MBR machine with the original 10240 release of Windows 10. That's the System partition, the C: partition, and the Recovery partition.

Apparently, regardless of whether the source disk was MBR or GPT, it creates a GPT disk (complete with its own MSR partition) as a dynamically expanding vhdx. The size of each disk is set to be just large enough to contain the partition.

In a situation where the windows system image had been deleted, in a few seconds I found the changed block copy in a shadowcopy and restored perfectly.

There are several ways. It can be mounted directly from the shadowcopy volume or it can be copied out depending on the preferred method of restore.

hdmi said:

First, download and install Ventoy on a USB flash drive... if you haven't already done that. Next, download this ISO file and just copy it to the USB flash drive:

http://dl.acronis.com/s/AcronisTrueImage2021_39287.iso

Finally, boot from the USB flash drive and go from there. To be able to do this, you might need to disable Secure Boot. Also make sure that BitLocker is turned off on the drive that holds the partitions you want to be included in your image. If it's taking a bit long before it moves to the next menu screen after you click, then simply unplugging the network cable from the back of the PC before you continue to boot from the USB flash drive should fix that problem.

The only reliable way to make a backup of your Windows system partition is to make sure that the instance of Windows, that has been installed on this partition, is shut down first. It means that, when Windows is actively running on the system, you can still make a reliable backup of a different Windows partition, if you have one. Otherwise you simply can't. That's just because snaphots were never designed to be reliable backups. Instead, they're just data. Sure, you can make reliable backup copies of any data you might have, but that doesn't magically turn your data into reliable backups. In addition to this, just because you are able to restore from an image, doesn't necessarily always mean that the data that is contained in the image is free from errors. Inconsistencies may have crept in at the time when the snapshot was still being created. Each time when the VSS service that is running on Windows creates a new snapshot, it informs other processes actively running on the system, that they should finish up on doing whatever it is that they are doing to the filesystem that could potentially cause these inconsistencies. In no way does this guarantee that all processes will take the appropriate actions and finish operation tasks, both successfully and in a timely fashion. There exists no verification method to this. Just because you are still able to boot into your Windows normally after you restore from image and all your apps still seem to be working OK, doesn't make it a reliable method in any way, shape or form. As most people never have troubles with it, most people continue to imagine that it is a reliable method. As a result, I call it imagining software. If you mean what I get... lol

Click to expand...

Creating a "straw man" and then cutting it down doesn't really prove anything. I completely disagree with the premise of the poster's discussion. I particularly enjoyed the statement "Just because you are still able to boot into your Windows normally after you restore from image and all your apps still seem to be working OK, doesn't make it a reliable method in any way, shape or form." From where I sit this seems like this IS a valid description of "reliable." And I must disagree with this quote, "The only reliable way to make a backup of your Windows system partition is to make sure that the instance of Windows, that has been installed on this partition, is shut down first." Most of us have been using Reflect for many years, using default settings and making images of partitions and disks, with complete success. We load Marcruim Reflect, tell it to make an image of a drive or partition, and start the process. We don't shut down Secure Boot, we don't touch Bitlocker, we just start the process and let it conclude. If our systems get hose and don't permit booting we just plug in the rescue disk we made long ago and restore the last image. I can't "imagine" what he's talking about so I must be missing something this poster is trying to convey as his digression doesn't make any sense to me. I've tried other programs (Acronis, AOME, etc.) and none are as simple to use. I'll stick with Reflect free until an operating system is made that is not compatible with the free version.