Native boot vhdx files and pagefiles

cereberus · May 15, 2022

I have some vhdx files which I native boot i.e. not in a virtual machine.

I have come across an issue which I cannot fully understand.

The vhdx file is on my physical 2nd drive (E drive).

When I boot the vhdx file with the pagefile set to E drive, it boot fines.

If I set it to the VHDX C drive (inside virtual hard drive) it boot ok but complains about a pagefile issue, and creates a temporary pagefile.
Ok, if the pagefile is in vhdx file, it is still actually on the E drive really.

This above is not really an issue, as I want the pagefile on my 2nd nvme drive.

Where things get complicated is when I boot the vhdx in hyper-v, as it cannot directly see the E drive (in enhanced mode, you can see the e drive, but it is via tclient i.e. effectively a network drive. So the vm then complains as no E drive and creates a temporary pagefile. If I create a page file on the C drive in the vhdx file, it boots fine and creates a pagefile inside the vhdx file as you would expect.

So I have to create two page files, one on C (inside vhdx) and one on E (on 2nd drive). When I boot the vhdx natively, it seems to ignore the C drive and uses the E drive. When I boot the vhdx in a vm, it uses the pagefile created inside VHDX (C) and does not even see the E pagefile.

This works fine but I do not fully understand why. It seems there is a limitation that pagefiles need to be external to vhdx file when natively booting vhdx files.

jimbo45 · May 15, 2022

cereberus said:
I have some vhdx files which I native boot i.e. not in a virtual machine.

I have come across an issue which I cannot fully understand.

The vhdx file is on my physical 2nd drive (E drive).

When I boot the vhdx file with the pagefile set to E drive, it boot fines.

If I set it to the VHDX C drive (inside virtual hard drive) it boot ok but complains about a pagefile issue, and creates a temporary pagefile.
Ok, if the pagefile is in vhdx file, it is still actually on the E drive really.

This above is not really an issue, as I want the pagefile on my 2nd nvme drive.

Where things get complicated is when I boot the vhdx in hyper-v, as it cannot directly see the E drive (in enhanced mode, you can see the e drive, but it is via tclient i.e. effectively a network drive. So the vm then complains as no E drive and creates a temporary pagefile. If I create a page file on the C drive in the vhdx file, it boots fine and creates a pagefile inside the vhdx file as you would expect.

So I have to create two page files, one on C (inside vhdx) and one on E (on 2nd drive). When I boot the vhdx natively, it seems to ignore the C drive and uses the E drive. When I boot the vhdx in a vm, it uses the pagefile created inside VHDX (C) and does not even see the E pagefile.

This works fine but I do not fully understand why. It seems there is a limitation that pagefiles need to be external to vhdx file when natively booting vhdx files.

That happens to me as well if I am using an external device and booting Physical vhdx files i.e not within nor part of a VM. I suppose the external device is equivalent to your "2nd" device in the scenario you outlined.

It's probably something to do with the "Pre-Os load" and initial disk allocation from the EFI boot and before attaching the "VHDX" files for finally loading the Windows kernel and GUI. Remember when the boot menu is presented you are effectively in a Win PE situation. There's very little if any documentation (outside Ms I suppose) that goes through how this bootstrap loader works and the "Minimal" GUI is presented before attaching the main OS and passing control to it.

Inside a VM all the disks are "Pre allocated" by the VM before any "internal" GUEST bootloader starts, so after VM (HOST) loads its own bootloader there shouldn't be any problems with the GUEST bootloader and Windows finding Disks.

This doesn't happen if I create VHDX files from within a Virtual machine environment and have the VM boot from those. as you have also showed even though you are using Windows as a HOST and I'm using Linux as Host machine.

I agree the popup is a bit annoying but it's "liveable" with.

For example INSIDE the VM "D" (my external disk is Disk 0 which the VM loader thinks is actually an "Internal Disk" ) and has designation D -- page file etc on it (there's also a load of other vhdx files on it).

Further down is the loaded "vhdx" file which is the relevant Windows install - as the "C" drive - but it's already been picked up from "D" in booting in the VM so the paging file is already available to "C".

Disk 1 is "In Progress" as I'm moving the VM's from External to Internal HDD (SSD) on to my Linux laptop,

My analysis might be faulty -- not a Windows internals guy but I suspect the logic is on the right lines. Perhaps a Microsoft Internals developer could elaborate more perhaps. !!!

Cheers
jimbo

cereberus · May 15, 2022

jimbo45 said:
That happens to me as well if I am using an external device and booting Physical vhdx files i.e not within nor part of a VM. I suppose the external device is equivalent to your "2nd" device in the scenario you outlined.

It's probably something to do with the "Pre-Os load" and initial disk allocation from the EFI boot and before attaching the "VHDX" files for finally loading the Windows kernel and GUI. Remember when the boot menu is presented you are effectively in a Win PE situation. There's very little if any documentation (outside Ms I suppose) that goes through how this bootstrap loader works and the "Minimal" GUI is presented before attaching the main OS and passing control to it.

Inside a VM all the disks are "Pre allocated" by the VM before any "internal" GUEST bootloader starts, so after VM (HOST) loads its own bootloader there shouldn't be any problems with the GUEST bootloader and Windows finding Disks.

This doesn't happen if I create VHDX files from within a Virtual machine environment and have the VM boot from those. as you have also showed even though you are using Windows as a HOST and I'm using Linux as Host machine.

I agree the popup is a bit annoying but it's "liveable" with.

For example INSIDE the VM "D" (my external disk is Disk 0 which the VM loader thinks is actually an "Internal Disk" ) and has designation D -- page file etc on it (there's also a load of other vhdx files on it).

Further down is the loaded "vhdx" file which is the relevant Windows install - as the "C" drive - but it's already been picked up from "D" in booting in the VM so the paging file is already available to "C".

View attachment 29213

Disk 1 is "In Progress" as I'm moving the VM's from External to Internal HDD (SSD) on to my Linux laptop,

My analysis might be faulty -- not a Windows internals guy but I suspect the logic is on the right lines. Perhaps a Microsoft Internals developer could elaborate more perhaps. !!!

Cheers
jimbo

I have a solution as I said.

In vhdx whilst native booting I set a page file on C drive.

This is how I set it up from native host. As you can see, it sets up pagefile on E drive and ignores the C drive entry.

Then when I boot vhdx inside a vm, it looks like below.
So it uses the C drive entry from above and ignores E drive (as no attached E drive).

Doing it like this works fine, and no "temporary pagefile" error.

Note: if you get a temporary pagefile error, delete the registry key TempPageFile before rebooting to a new pagefile configuration as sometimes it seems to ignore changes if key is still set. It took me a while to find this - lots of googling.

jimbo45 · May 15, 2022

cereberus said:
I have a solution as I said.

In vhdx whilst native booting I set a page file on C drivq

This is how I set it up from native host. As you can see, it sets up pagefile on E drive and ignores the C drive entry.

View attachment 29215

View attachment 29214

Then when I boot vhdx inside a vm, it looks like below.
So it uses the C drive entry from above and ignores E drive (as no attached E drive).

Doing it like this works fine, and no "temporary pagefile" error.

Note: you might have to delete a

View attachment 29218

View attachment 29216

Note: if you get a temporary pagefile error, delete the registry key TempPageFile before rebooting to a new pagefile configuration as sometimes it seems to ignore changes if key is still set. It took me a while to find this - lots of googling.

View attachment 29219

Great stuff
however I was just suggesting why this happened in the ist place.

Will definitely try your suggestion.

And IT WORKS !!! even on my Linux Host.

I'm still moving the VM's though to Internal laptop storage as there's 1TB of SSD / (actually NVME) space - very fast. !!

With this and no 20% wallet busting VAT very reasonable -- chucked out the 256 GB module the laptop came with -- why do even high spec laptops skimp on storage !!!!! -

As an old fashioned Engineer getting "harrassed" in my younger days by great Ship building Engineers from the Scottish Clyde as a "F...ng f...oooo .rrrrr....iiii....nnn...eer...rr" and not even deemed worthy of being called a "Sassenach" I actually learnet a lot --and we did actually drink in pubs as friends with them afterwards as they were only "having a Go" !!! as they thought the "Sassenachs" weren't capable of "answering back " !! and replacing the NVME module was a real easy job with the right tools.

Approx 90 EUR here (VAT on this stuff ZERO - "Educational Material" !!!!). A small shipping charge and still ZERO customs duties as stuff is sourced from outside UK - even though delivery is fulfilled by Amazon UK from East Midlands Airport.

Thanks !!!

Cheers
jimbo

cereberus · May 16, 2022

I did a bit more googling. It seems it is a design limitation of vhdx files that when natively booting a vhdx file, the pagefile is OUTSIDE the vhdx file, but when booting vhdx file in a vm, pagefile is INSIDE the vm (which confirms my observations).

extract from link

The local disk partition that contains the VHDX file must have enough free disk space for expanding a dynamic VHDX to its maximum size and for the page file created when booting the VHD. The page file is created outside the VHDX file, unlike with a virtual machine where the page file is contained inside the VHD.

Deploy Windows with a VHDX (Native Boot)

docs.microsoft.com

So when I set vhdx (when natively booting it) up with 2 pagefiles (C (inside vhdx) and E (outside vhdx), this has enabled me to satisfy the pagefile requirements when natively booting or vm booting.

When natively booting, it selects the E drive external drive pagefile, ignoring C drive.

When booting in a vm, the C drive pagefile is selected.

----------------------------------------------------------------------------------------------

Edit.

I have found an easier way of setting up pagefiles if you want it automatic, but it is a bit counterintuitive.

1) native boot into vhdx
2) uncheck automatically manage paging
3) set all pagefiles for each drive to none
4) recheck automatically manage paging
5) save setting

Upon native reboot, it looks like this - the pagefile is now in the 21H2 drive which is the host OS directory.

When you boot into the vhdx in a vm, it looks like this:

Here the pagefile is inside the VM.

The only minor limitation is that the native boot pagefile is on the physical drive partition containing Host OS.

If you want to put the native boot pagefile on a different partition/drive, you have to use the more complicated method I showed first.

jimbo45 · May 16, 2022

cereberus said:
I did a bit more googling. It seems it is a design limitation of vhdx files that when natively booting a vhdx file, the pagefile is OUTSIDE the vhdx file, but when booting vhdx file in a vm, pagefile is INSIDE the vm (which confirms my observations).

extract from link

The local disk partition that contains the VHDX file must have enough free disk space for expanding a dynamic VHDX to its maximum size and for the page file created when booting the VHD. The page file is created outside the VHDX file, unlike with a virtual machine where the page file is contained inside the VHD.

Deploy Windows with a VHDX (Native Boot)

Deploy Windows with a VHDX (Native Boot)

docs.microsoft.com

So when I set vhdx (when natively booting it) up with 2 pagefiles (C (inside vhdx) and E (outside vhdx), this has enabled me to satisfy the pagefile requirements when natively booting or vm booting.

When natively booting, it selects the E drive external drive pagefile, ignoring C drive.

When booting in a vm, the C drive pagefile is selected.

----------------------------------------------------------------------------------------------

Edit.

I have found an easier way of setting up pagefiles if you want it automatic, but it is a bit counterintuitive.

1) native boot into vhdx
2) uncheck automatically manage paging
3) set all pagefiles for each drive to none
4) recheck automatically manage paging
5) save setting

View attachment 29254

Upon native reboot, it looks like this - the pagefile is now in the 21H2 drive which is the host OS directory.

View attachment 29255

When you boot into the vhdx in a vm, it looks like this:

Here the pagefile is inside the VM.

View attachment 29256

The only minor limitation is that the native boot pagefile is on the physical drive partition containing Host OS.

If you want to put the native boot pagefile on a different partition/drive, you have to use the more complicated method I showed first.

I use these scenarios
On laptop Internal disk is Linux EFI and xfs formatted so Windows just ignores.

on boot from external disk with Real vhdx files :

DISK D with EFI, msr, and rest of disk space one big ntfs partition containing a load of vhdx files
Bootloader installed to EFI partition for the 3 Windows systems (each via bcboot) . All those Windows systems when booted (note real systems so can't be run concurrently unlike as VM's) have C: as windows disk - request for paging as per the original post.

ON laptop -- inside a VM - have the same set up -- virtual "Disk" as per Disk "D above with vhdx files - - now when starting the VM I get the same boot menu as the physical external D one - select a VM and after that boots no request for paging files !!!

Note this method is probably not very common as people usually create discrete VM's for each OS - but I find this the best way for having several windows builds that have identical "Virtual hardware" so it saves messing about with multiple VM's.

I find it a lot quicker simply to mess around with vhdx files and install relevant windows systems via DISM.EXE /Apply-Image and update the boot loader. Of course the disadvantage of having this multi boot WITHIN the VM is that these systems can't run concurrently either but I usually don't want that anyway. Backups are simple - just backup the vhdx files in either case. On restore just update the boot loader if the host disks have been moved / altered.

Cheers
jimbo

cereberus · May 16, 2022

cereberus said:
The only minor limitation is that the native boot pagefile is on the physical drive partition containing Host OS.

If you want to put the native boot pagefile on a different partition/drive, you have to use the more complicated method I showed first.

I had a light bulb moment - I created native boot vhdx paging as follows:

and then I created a second VHDX file (16 GB in my case), formatted it as NTFS, and then attached that to VM as a second drive.

I booted into vhdx file in vm and changed pagefiledrive letter to E, rebooted vm, and boom - page file on second drive (it changed drive letter to D as no drive D in vm - no big deal).

Trick was to have a second drive in vm like native boot vm sees.

This is a much simpler solution for what I want i.e. pagefile on secondary NVME drive, not on primary NVME Optane drive (write resilience of second drive is much higher than primary drive).

jimbo45 · May 16, 2022

cereberus said:
I had a light bulb moment - I created native boot vhdx paging as follows:

View attachment 29261

and then I created a second VHDX file (16 GB in my case), formatted it as NTFS, and then attached that to VM as a second drive.

View attachment 29262
I booted into vhdx file in vm and changed pagefiledrive letter to E, rebooted vm, and boom - page file on second drive (it changed drive letter to D as no drive D in vm - no big deal).

Trick was to have a second drive in vm like native boot vm sees.

This is a much simpler solution for what I want i.e. pagefile on secondary NVME drive, not on primary NVME Optane drive (write resilience of second drive is much higher than primary drive).

From WITHIN a VM I don't need to do any of that -- It just works perfectly -- at least on my HOST system : Even when I have a choice of "Multiple OS'es" from the Windows boot menu from WITHIN the VM. Both OSes here are VM's BTW. English and Icelandic versions. Same hardware (real and Virtual), and the "Virtual disk" is defined as a single disk with one efi partition, one msr partition and an NTFS area on which some vhdx files exist containing the relevant OS'es.

Cheers
jimbo

jimbo45 · May 21, 2022

cereberus said:
I did a bit more googling. It seems it is a design limitation of vhdx files that when natively booting a vhdx file, the pagefile is OUTSIDE the vhdx file, but when booting vhdx file in a vm, pagefile is INSIDE the vm (which confirms my observations).

extract from link

The local disk partition that contains the VHDX file must have enough free disk space for expanding a dynamic VHDX to its maximum size and for the page file created when booting the VHD. The page file is created outside the VHDX file, unlike with a virtual machine where the page file is contained inside the VHD.

Deploy Windows with a VHDX (Native Boot)

Deploy Windows with a VHDX (Native Boot)

docs.microsoft.com

So when I set vhdx (when natively booting it) up with 2 pagefiles (C (inside vhdx) and E (outside vhdx), this has enabled me to satisfy the pagefile requirements when natively booting or vm booting.

When natively booting, it selects the E drive external drive pagefile, ignoring C drive.

When booting in a vm, the C drive pagefile is selected.

----------------------------------------------------------------------------------------------

Edit.

I have found an easier way of setting up pagefiles if you want it automatic, but it is a bit counterintuitive.

1) native boot into vhdx
2) uncheck automatically manage paging
3) set all pagefiles for each drive to none
4) recheck automatically manage paging
5) save setting

View attachment 29254

Upon native reboot, it looks like this - the pagefile is now in the 21H2 drive which is the host OS directory.

View attachment 29255

When you boot into the vhdx in a vm, it looks like this:

Here the pagefile is inside the VM.

View attachment 29256

The only minor limitation is that the native boot pagefile is on the physical drive partition containing Host OS.

If you want to put the native boot pagefile on a different partition/drive, you have to use the more complicated method I showed first.

Hi there

If you are native booting a physical vhdx file from an external drive (as a Windows 2 Go type of system) and your normal internal drive is a Linux disk then your method here works as Windows won't have access to the computers internal internal drive. So you allocate as described in your post the system managed paging on both the "C" (the windows drive from within the vhdx) and on the disk that the vhdx resides on e.g Q or whatever. Then reboot and it all works perfectly.

What is noticeable though even on a system with enough RAM after you do this the system managed paging files seem to improve the system performance (even with small paging files) compared with Zero paging files or "System created temporary pagefile" with that popup message when starting the system

Cheers
jimbo

jolly · Jul 21, 2022

cereberus said:
This is a much simpler solution for what I want i.e. pagefile on secondary NVME drive, not on primary NVME Optane drive (write resilience of second drive is much higher than primary drive).

What drive do you have that has better resilience than an optane? You really want the pagefile on the optane, since that has lightning fast latency. I love optane drives btw.

cereberus · Jul 22, 2022

jolly said:
What drive do you have that has better resilience than an optane? You really want the pagefile on the optane, since that has lightning fast latency. I love optane drives btw.

For the same amount of writes, my WD NVME has only used 1% of its life expectancy, whereas the Optane drive has used 4%.
Since I swapped page file over and disabled hibernation, the life expectancy rate of Optane has slowed down.

I am not concerned about latency on optane drive, as I have 16 GB RAM and do not pagefile much (it was the hibernation files and vhd drives on optane thrashing it).

In the end, of course, life expectancy figures are garbage really, as manufacturer often set them artificially low to avoid ever having to pay a warranty claim.

jolly · Jul 22, 2022

I suspect whatever you are using to measure life expectancy is giving you wrong info.

A Optane 905p is rated for 10 full drive writes a day. The latest gen p5800x optanes are an even more insane 100 DWPD.
A top of the line WD black SN850 is rated at:
ENDURANCE4 (TBW): over 5years
2TB: 1,200
1TB: 600
500GB: 300

If I'm doing the math right, for the 2TB one thats 1200/(365*5)= 0.657534246575342 TB per day, then 0.657534246575342 /2 =
0.328767123287671 DWPD for the WD.
It's practically impossible for a consumer to wear out an optane.

(I do like the WD drives, I've got a few of those, but Optanes are a whole different class of devices)
Which optane drive do you have? The older gen will be slower for bulk transfer, but usually end up being better for anything thats not sequential workloads.

Anyways, thanks for the thread/forum, it's been quite useful in my VHDX native boot adventures today. I've now got a vhdx OS that's almost identical to my main, with a lot of the directories symlinked through so it's almost the same as my main os, but I can play around with it.

cereberus · Jul 22, 2022

jolly said:
I suspect whatever you are using to measure life expectancy is giving you wrong info.

A Optane 905p is rated for 10 full drive writes a day. The latest gen p5800x optanes are an even more insane 100 DWPD.
A top of the line WD black SN850 is rated at:
ENDURANCE4 (TBW): over 5years
2TB: 1,200
1TB: 600
500GB: 300

If I'm doing the math right, for the 2TB one thats 1200/(365*5)= 0.657534246575342 TB per day, then 0.657534246575342 /2 =
0.328767123287671 DWPD for the WD.
It's practically impossible for a consumer to wear out an optane.

(I do like the WD drives, I've got a few of those, but Optanes are a whole different class of devices)
Which optane drive do you have? The older gen will be slower for bulk transfer, but usually end up being better for anything thats not sequential workloads.

Anyways, thanks for the thread/forum, it's been quite useful in my VHDX native boot adventures today. I've now got a vhdx OS that's almost identical to my main, with a lot of the directories symlinked through so it's almost the same as my main os, but I can play around with it.

Nah - I understand life expectancy. My optane drive is quoted at 300 TB if I remember and basesd on decline, and amount I have written, that is accurate However, the official life is probably garbage as I said. Equally my other nvme drive is quoted at 600 TB (2x as much) and again figures stack up with amount of writes, and indeed I am exceeding nominal guarantee rates somewhat.

You say it is practically impossible to wear out an optane drive but I write far more than average person.

In the end, quoted figures are meaningless really.

I just see no performance loss set up way I am, and I do test it.

Native boot vhdx files and pagefiles

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Well-known member

My Computer

Active member

My Computer

Well-known member

My Computer

Active member

My Computer

Well-known member

My Computer

Similar threads