Optional entries in the <MediaRegistry> section may prevent VM registration

[Socratis]

The <MediaRegistry> section of the .vbox file contains a registry of active and previously used media; HDs and DVDs, a.k.a. the Most Recently Used (MRU). The MRU list is nothing but a convenience for the end user.

Problem: If there's an MRU entry that conflicts with an existing registered medium, the import/registration of a VM fails.

Example: Registered VM1 has an entry in the MediaRegistry section for a previously used DVD:

<MediaRegistry>
  <DVDImages>
    <Image uuid="{11111111-1111-1111-1111-111111111111}" location="Test.iso"/>
  </DVDImages>
</MediaRegistry>

A second VM2 has the following in the MediaRegistry section for a previously used DVD:

<MediaRegistry>
  <DVDImages>
    <Image uuid="{22222222-2222-2222-2222-222222222222}" location="Test.iso"/>
  </DVDImages>
</MediaRegistry>

The names of the DVDs are the same, the UUIDs are different (it happens more often than you think). Trying to register VM2 results in the following error:

Failed to open virtual machine located in /Users/Shared/Test2.vbox.

Cannot register the DVD image '/Users/Shared/Test.iso' {22222222-2222-2222-2222-222222222222} because a CD/DVD image '/Users/Shared/Test.iso' with UUID {11111111-1111-1111-1111-111111111111} already exists.

Result Code: NS_ERROR_INVALID_ARG (0x80070057)
Component: VirtualBoxWrap
Interface: IVirtualBox {9570b9d5-f1a1-448a-10c5-e12f5285adad}

and the import fails. Not good.

Workaround: The only workaround so far is to manually edit the .vbox file and remove the optional entry for the "offending" medium. Remember, this entry is purely optional, the DVD is not actively attached.

Solution: If an entry in the MediaRegistry section is solely used so that it can be displayed in the MRU, ignore that entry upon registration and delete that entry from the MediaRegistry section, as to not cause conflicts.

[Socratis]

Reproducing the problem

Register either of the attached VMs. Then try to register the other one. Kaboom...

[Socratis]

It's actually far worse than I first realized. We're talking about a potential loss of a VM definition, the ".vbox" file under common scenarios... :o

Scenario

• You have your VMs all set up and you decide to restore a VM from a backup.

• The backup is from a slightly different version, different enough to warrant a different UUID for the "VBoxGuestAdditions.iso".

• Shutdown the VM and VirtualBox, restore the contents from "<Backup>\VictiM" overwriting the existing "<VirtualBox VMs\VictiM" folder. That can include previous Snapshots and Logs, but it's of no importance.

• Unfortunately for the VictiM, when it was backed up, it contained a section with "known" media:

  <MediaRegistry>
    <HardDisks>
      <HardDisk uuid="{990cfa7b-3d35-401d-b566-f61396c7dd45}" location="Mint 19.vdi" format="VDI" type="Normal">
        <HardDisk uuid="{7e1fe780-294d-48e6-950c-9e8e88a2ab28}" location="Snapshots/{7e1fe780-294d-48e6-950c-9e8e88a2ab28}.vdi" format="VDI"/>
      </HardDisk>
    </HardDisks>
    <DVDImages>
      <Image uuid="{6589fa73-3ebb-4f1c-aee2-f6d550b6cc58}" location="/Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso"/>
    </DVDImages>
  </MediaRegistry>
  

• Note that the <DVDImages> part is simply in the "MRU list" and is not actively attached to the VM (see "VictiM.vbox").

• Upon a simple launch of the VirtualBox Manager (just launch it, nothing more), the "VictiM.vbox" gets mutilated and back to a really bare-bones skeleton of a VM. See attached "VictiM-after.vbox".

• If someone edits the "VictiM.vbox" file before launching !VirtualBox and simply removes the following section, there is no problem whatsoever!

  <DVDImages>
    <Image uuid="{6589fa73-3ebb-4f1c-aee2-f6d550b6cc58}" location="/Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso"/>
  </DVDImages>
  

Adding the VM

You could also try to "Remove the VM" even deleting its files, and you restore from backup, and try to add "VictiM.vbox", you end up with the original error of this ticket:

Cannot register the DVD image '/Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso' {6589fa73-3ebb-4f1c-aee2-f6d550b6cc58} because a CD/DVD image '/Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso' with UUID {7d63cdf3-f12a-49b2-8c83-8cd9ad2fed69} already exists.

[Socratis]

Scenario

Newer, simpler scenario...

1. Clean start. I moved my "VirtualBox" directory to make sure no signs of VMs/settings are there.

2. Start VirtualBox, create two VMs, all defaults. They're guinea pigs after all...

3. Launch VM1, let it fail to boot. Attach the GAs ISO. Eject it. Power off the VM.

4. Launch VM2, let it fail to boot. Attach the GAs ISO. Eject it. Power off the VM.

5. Backup the whole VM1 folder for safekeeping. Copy it on your Desktop for example.

6. Go to the MediaManager » Optical Disks » Remove the "VBoxGuestAdditions.iso" from there.

7. Launch VM2, let it fail to boot. Attach the GAs ISO. Eject it. Power off the VM.

8. Close VirtualBox, make sure VBoxSVC is done.

9. Restore VM1 from the backup. The one from the desktop?

10. Launch VirtualBox, see your VM2 becoming a skeleton. No other action is needed on your part, you just ruined your day...

Explanation

When you first inject the "VBoxGuestAdditions.iso" (could be any other disk), it gets a new UUID. That UUID is also stored for convenience in the MRU of the VM(s).

When you backup a VM, you take along with the .vbox, its MRU and its UUIDs.

Cleaning your media from the MediaManager means forgetting about these MRUs and the UUIDs. The next time that you're going to insert the "VBoxGuestAdditions.iso" it gets a new UUID.

Restoring a VM from a backup, that happens to have a different UUID? Bad news. You have the same referenced file sporting two different UUIDs. Not good. VBoxSVC goes berzerk and kills one (or more) VMs down to a skeleton.

I originally found this on OSX, I came with the steps on a Win7-64 host, so it's cross-platform, why wouldn't it?

[Klaus Espenlaub]

If only I could reproduce... I get VM2 being inaccessible, with trunk on Linux host. Expected behavior of the API code. Guess I need to try the same on Windows and/or macOS.

[Klaus Espenlaub]

Of course it's reproducible. On all platforms, as it should. The reason why I couldn't reproduce on Linux was a "slight shortcut" I took, namely not creating a hard disk for the VMs. Turns out that this is a key ingredient to the reproduction. The hard disk is first in the medium registry (and thus get processed correctly), and the DVD image comes afterwards (and with the reproduction it is forced to have a UUID difference).

The root cause of the issue is that this pattern triggers saving of the (initially correctly recognized as "Inaccessible") config file of VM2, which for inaccessible VMs however has only meaningless defaults in the in-memory state. So that wipes out the content of the config file of VM2. This sequence isn't visible in the GUI (it is however visible with "VBoxManage list vms --long"), because it's relatively aggressive with attempts to refresh an inaccessible VM. This isn't a bug as such, but hides the development of the API bug, in so far as the refresh will find the (previously saved) config file of VM2 which now all of a sudden (due to the loss of the original file content) is valid.

From a quick look at the code history it seems that the code which triggers all this has been added during development of 4.1.0, a bit over 7 years ago. Not sure if the bug was "active" straight away, but it certainly was for many years. Extremely few people must have used the sequence of actions to trigger the bug. It only happens if a user relies on moving VMs from/to the control of VirtualBox, e.g. using such an approach for backups or similar purposes.

[Socratis]

Thanks @Klaus, I won't be losing VMs when restoring from backup! Fix in https://www.virtualbox.org/changeset/78202/vbox

But still that fix shows the VM as inaccessible. Still not an "acceptable" solution in my head, and still a valid concern when I send you my VM which includes a <MediaRegistry>\<DVDImages> section that doesn't agree with what you might already have.

It doesn't make sense to have an inaccessible VM for "past regressions". I load the GAs on my VM, I send you my VM (the whole folder) and it's ... inaccessible? For what reason exactly?

This section should be completely discarded in my opinion *if* there's the smallest of conflicts. We're not talking about actively attached DVDs, we're talking about a

"(VM speaking): At some point I mounted that thing and I'm now inaccessible, someone has to *manually* edit my .vbox to fix it, they better know where and what do delete!"

situation...

[Klaus Espenlaub]

Sure, this is just an intermediate step. First goal is always making sure that there's no data loss. What still needs doing (and before that it needs having a good idea how to give the user a way to make the right decisions) is to actually implement a way to get UUID problems fixed.

You provoke a relatively special one (different GA ISO UUID), and it's not the first time that someone runs into this one. The more tricky case (which also happens surprisingly often with people who have multiple machines) is to handle duplicate hard disk UUIDs, caused by copying the VM elsewhere, making modifications and after copying it back to the original place expecting to use it alongside its ancestor.

[Socratis]

I'm afraid I have to re-open this one. The original example with Test1.vbox ans Test2.vbox still fails to register "VM2". Try it...

[Klaus Espenlaub]

Phew. Thought that something is still wrong with the current logic, still losing data. But no, the reopening was just a reminder that the "behaves as designed" isn't yet close to "behaves in a sensible way".

It's 'only' to request to have a way to fix such VM configs (for DVDs we might do it earlier, because the stakes are not as high than for hard disk images - getting those wrong can mean data loss) as part of registering a VM config with conflicting media registry entries.

[Carlos D.]

Hi, it's good news the cause has been found and will be fixed, eventually!

I'll link to my ticket here that I opened three years(!) ago, dealing with the very same problem.

https://www.virtualbox.org/ticket/15373

[Michael Thayer]

Since the fix should be in published releases, could you please confirm it? Your "will be fixed" suggested that it is not yet fixed for you.

[Carlos D.]

The immediate problem (data loss) should be fixed, although I couldn't test it, so far. I should have rather said "fixed completely".

I'm referring to the comment #7 by @klaus, who mentioned the current situation is only an intermediate step. Preventing data loss (I lost VM settings many times and have backups of all of them, for years) is always important, that has been done. I'll test as soon as I can, but my VBox is usually in use by BOINC. I'm currently using 5.x Testbuilds.

BOINC tasks are what introduced the original issue for me. These tasks are controlled by a wrapper, using VBoxManage to create, update, maintain and, finally, destroy VMs on the fly. They usually never shut down and only use Save states. Worst of all, they all leave their copy of the VBoxGuestAdditions ISO attached!

The problem occurred at first start, after updating VBox. This changes the VBoxGuestAdditions version and often (or always?) assigned a new GUID for the new copy. But the BOINC task's GUIDs still were the old one, left untouched. Old and new GUIDs collided.

But, you'll surely admit that automatically managed VMs becoming "Inaccessible" is not the optimal solution. The wrapper does not know how to deal with them and task failure rates will go up.

Hope I could kill some confusion!

[Socratis]

Repeating myself here:

I'm afraid I have to re-open this one. The original example with Test1.vbox ans Test2.vbox still fails to register "VM2". Try it...