This is the way that I came up to work around it. It's not pretty, but it works for me:
Add to pass 4 specialize > Windows-Deployment > RunSynchronous > RunSynchronousCommand with the following values:
Order: 1 (if 1 is in use, use next available number)
Path: powershell.exe -Command "Get-NetAdapter | Disable-NetAdapter -Confirm:$false"
This will disable all network adapters which will also cause bypassing of Windows updates during setup.
Next, we need to re-enable network adapters at the end of setup. To do so, add this to your autounattend.xml answer file:
Add to pass 7 oobe System > Windows-Shell-Setup > FirstLogonCommands > RunSynchronous > SynchronousCommand with the following values:
Order: 1 (if 1 is in use, use next available number)
Path: powershell.exe -Command "Get-NetAdapter | Enable-NetAdapter -Confirm:$false"