Implementing VMDK relative raw disk partitions on Windows

[regedit]

This bug serves to track progress of implementing the -relative option for the VBoxManage internalcommands createrawvmdk -filename ... -rawdisk ... -partitions ... command on Windows hosts.

See section 9.9.1 of the docs regarding using physical partitions as a Virtual Drive for a VM. The -relative option for the above command was only available on Linux hosts.

[regedit]

Some additional background: Although a raw disk VMDK could be created on windows hosts, even with specific -partitions selected, and such a VM could even be started up and booted, however the VM soon crashes with rc=VERR_ACCESS_DENIED presumably as soon as VirtualBox tries writing to these partitions (​VBox.log and ​VBoxStartup.log) apparently due to newer restrictions/security measures introduced in Windows Vista+ / Server 2008+.

Initial changesets (Thanks Klaus!)

• https://www.virtualbox.org/changeset/56910/vbox - somewhat related cleanup
• https://www.virtualbox.org/changeset/56912/vbox - bulk of the implementation
• https://www.virtualbox.org/changeset/56918/vbox - windows MBR numbering scheme

[regedit]

This fix *seems* to be correct/OK, however there's a

Problem on UEFI system

Scenario
UEFI enabled system with dual boot Windows 8.1 and Kubuntu 15.04 both installed (in UEFI mode) to different partitions on same GPT HDD, trying to virtualize the Linux as guest within the Windows as host.

Short description
Error VERR_SHARING_VIOLATION occurs trying to use the new -relative option to make a VMDK of the Linux partition(s) with the EFI partition included.

If the VMDK is made without the EFI partition, the VM can successfully turn on but lands at an EFI shell with no filesystems mounted, only BLK devices. There doesn't seem to be a 'mount' command, which might not help either way since the EFI partition would not be readable as per the VMDK.

Addtional Info

Diskpart info (with partition descriptions added in parentheses)

DISKPART> select disk 1

Disk 1 is now the selected disk.

DISKPART> list partition

  Partition ###  Type              Size     Offset
  -------------  ----------------  -------  -------
  Partition 1    Recovery           300 MB  1024 KB   (Recovery)
  Partition 2    System             100 MB   301 MB   (EFI Partition)
  Partition 3    Reserved           128 MB   401 MB   (Windows Internal/Hidden)
  Partition 4    Primary            232 GB   529 MB   (Windows)
  Partition 6    Unknown            229 GB   232 GB   (Linux)
  Partition 5    Unknown           3905 MB   461 GB   (Linux Swap)

VBox info (with partition descriptions added in parentheses)

prompt> VBoxManage.exe internalcommands listpartitions -rawdisk \\.\PhysicalDrive1
Number  Type   StartCHS       EndCHS      Size (MiB)  Start (Sect)
1       0x00  0   /0  /0   0   /0  /0            300         2048   (Recovery)
2       0x00  0   /0  /0   0   /0  /0            100       616448   (EFI Partition)
3       0x00  0   /0  /0   0   /0  /0            128       821248   (Windows Internal/Hidden)
4       0x00  0   /0  /0   0   /0  /0         237941      1083392   (Windows)
6       0x00  0   /0  /0   0   /0  /0         234564    488386560   (Linux)
5       0x00  0   /0  /0   0   /0  /0           3905    968773632   (Linux Swap)

So I need to enable EFI in the VM system settings, and I'll need to expose the EFI partition to the VMDK, thus partitions 2 (EFI) 5 (Linux) and 6 (Linux swap). Note that the order was reversed for the last two during Linux setup, but shouldn't matter.

prompt> VBoxManage internalcommands createrawvmdk -filename C:\Users\Regedit\Desktop\VM\test1.vmdk -rawdisk \\.\PhysicalDrive1 -partitions 2,5,6 -relative
VBoxManage.exe: error: VMDK: could not open raw partition file '\\.\Harddisk1Partition2'
VBoxManage.exe: error: Error code VERR_SHARING_VIOLATION at F:\tinderbox\win-rel\src\VBox\Storage\VMDK.cpp(3545) in function int __cdecl vmdkCreateRawImage(struct VMDKIMAGE *,struct VBOXHDDRAW *const ,unsigned __int64)
VBoxManage.exe: error: Cannot create the raw disk VMDK: VERR_SHARING_VIOLATION
VBoxManage.exe: error: The raw disk vmdk file was not created

Perhaps it cannot gain access to the EFI Partition because it is considered "in use" by windows? Maybe it should be read-only access?

I will attach VMDK files successfully generated by omitting the -relative option, as well as by omitting partition 2.

[regedit]

successfully created VMDKs

[Klaus Espenlaub]

Looks like this needs proper readonly partition support (applies both to -relative and without), which doesn't exist yet. Needs a new option (plus documentation), and the per-partition flag needs to be passed to the VMDK code, marking the partition as readonly.

Also, the VMDK code doesn't handle readonly extents correctly, if it needs to open the extent it will use the same writable flags as for the main VMDK. That's what causes the error (during raw partition VMDK create), which would repeat itself later at VM runtime, as the same issue exists when opening an existing VMDK. There simply wasn't any need to implement RDONLY beyond the check in the write operation.

[regedit]

Workaround for the UEFI issue:

Make a VDI out of the physical disk's EFI partition.

• Use dd or ddrescue to make a disk image out of the EFI partition (which can be found via fdisk -l on Linux or diskpart/Disk Management on Windows)
• VBoxManage convertfromraw the_disk_image/path/and/filename destination_vdi_file/path/and/filename.vdi
• Add the VDI to the VM (might need to specifically be the first disk)

Note that depending on your BIOS/UEFI boot setup you may need to manually boot the OS either from EFI shell or from the BIOS's various options.

For those familiar with the VMDK spec, you can specify references to the EFI disk image file itself in the vmdk, for a single virtual disk solution. (Thanks Klaus)

[aeichner]

This is only slightly related to the ticket description but it was mentioned in the first comment. Writing to volumes is not possible on Windows while it is mounted on the host and Windows has the habit to mount usable volumes automatically under certain circumstances which will make writing to it fail during VM runtime. The next maintenance release tries to fix this by unmounting any volume on the host when starting the VM and preventing any mount attempt by taking the volume offline so VBox can access it exclusively. I uploaded a testbuild to https://www.virtualbox.org/download/testcase/VirtualBox-5.0.1-101957-Win.exe, it would be great if you could try it out and report any regressions, thanks!

[Klaus Espenlaub]

Note that with 5.0.2 there was a regression in raw disk/partition access (and even flat vmdk support), as everything became readonly. Use the latest test build for working code.

[regedit]

Problems booting VM on 5.0.4r102546 using -relative 2r,5,6

[tukan]

I can confirm this bug in the VM 5.0.6 r103037 using windows 7 x64.

A raw disk vmdk creation fails:

The command used: VBoxManage internalcommands createrawvmdk -filename "C:\Users\googre\VirtualBox VMs\oBSD\oBSD.vmdk" -rawdisk "\\.\PhysicalDrive1"

The error shown: VBoxManage.exe: error: Cannot open the raw disk '\\.\PhysicalDrive1': VERR_SHARING_VIOLATION VBoxManage.exe: error: The raw disk vmdk file was not created

The disk layout:

DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          238 GB      0 B
* Disk 1    Online           14 GB    14 GB
  Disk 2    No Media           0 B      0 B
  Disk 3    No Media           0 B      0 B
  Disk 4    No Media           0 B      0 B

DISKPART> select disk 1

Disk 1 is now the selected disk.

DISKPART> list partition

There are no partitions on this disk to show.

DISKPART>