VERR_CFGM_INVALID_NODE_PATH starting VM that had restored snapshot => Fixed in SVN

[Jacob Klein]

I'm having trouble getting a VM to restart.

The error is: VERR_CFGM_INVALID_NODE_PATH

This VM has only ever seen 4.3.6, and was setup to use VT-x/AMD-V. The relevant log information is below. Any ideas?

VirtualBox VM 4.3.6 r91406 win.amd64 (Dec 18 2013 17:12:34) release log
00:00:00.223057 Log opened 2014-01-03T14:30:44.400956300Z
00:00:00.223058 Build Type: release
00:00:00.223060 OS Product: Windows 8
00:00:00.223060 OS Release: 6.2.9200
00:00:00.223061 OS Service Pack: 
00:00:00.223065 Host RAM: 12278MB total, 9521MB available
00:00:00.223068 Executable: C:\Program Files\Oracle\VirtualBox\VirtualBox.exe
00:00:00.223068 Process ID: 1884
00:00:00.223068 Package type: WINDOWS_64BITS_GENERIC
00:00:00.223889 Installed Extension Packs:
00:00:00.223906   Oracle VM VirtualBox Extension Pack (Version: 4.3.6 r91406; VRDE Module: VBoxVRDP)
00:00:00.252311 VRDP: Statistics created: [full], enabled: 0.
00:00:00.252602 VRDP: VRDP: VD: Frames=10 MinMS=15 MaxMS=300 HistoryMS=2000 VideoMS=300
00:00:00.252658 UIMediumEnumerator: Medium-enumeration finished!
00:00:00.253857 VRDP: TCP server listening on port 49443 (IPv4 and IPv6).
00:00:00.254010 VRDP: Audio: rate correction mode 0x3.
00:00:00.254027 VRDE: loaded version 4 of the server.
00:00:00.254035 VRDE: [IMAGE]
00:00:00.254039 VRDE: [MOUSEPTR]
00:00:00.254042 VRDE: [SCARD]
00:00:00.254045 VRDE: [TSMFRAW]
00:00:00.254047 VRDE: [VIDEOIN]
00:00:00.254049 VRDE: [VRDE::INPUT]
00:00:00.273337 SUP: Loaded VMMR0.r0 (C:\Program Files\Oracle\VirtualBox\VMMR0.r0) at 0xfffff80004c79000 - ModuleInit at fffff80004c98220 and ModuleTerm at fffff80004c98530 using the native ring-0 loader
00:00:00.273356 SUP: VMMR0EntryEx located at fffff80004c99760, VMMR0EntryFast at fffff80004c988b0 and VMMR0EntryInt at fffff80004c988a0
00:00:00.273362 SUP: windbg> .reload /f C:\Program Files\Oracle\VirtualBox\VMMR0.r0=0xfffff80004c79000
00:00:00.275981 Guest OS type: 'Linux26'
00:00:00.281442 VMSetError: D:\tinderbox\win-4.3\src\VBox\VMM\VMMR3\VM.cpp(365) int __cdecl VMR3Create(unsigned int,const struct VMM2USERMETHODS *,void (__cdecl *)(struct UVM *,void *,int,const char *,unsigned int,const char *,const char *,char *),void *,int (__cdecl *)(struct UVM *,struct VM *,void *),void *,struct VM **,struct UVM **); rc=VERR_CFGM_INVALID_NODE_PATH
00:00:00.281445 VMSetError: The path of a new node contained slashs or was empty.
00:00:00.281732 ERROR [COM]: aRC=E_FAIL (0x80004005) aIID={8ab7c520-2442-4b66-8d74-4ff1e195d2b6} aComponent={Console} aText={The path of a new node contained slashs or was empty. (VERR_CFGM_INVALID_NODE_PATH)}, preserve=false
00:00:00.312315 Power up failed (vrc=VERR_CFGM_INVALID_NODE_PATH, rc=E_FAIL (0X80004005))
00:00:00.787014 VRDP: TCP server closed.

[Frank Mehnert]

Are there any extradata set for this VM?

[Jacob Klein]

I do not know what you mean - What do you mean? Also, I believe I've given as much information as I was able to capture; this BOINC task is now gone.

[Frank Mehnert]

One information you missed in your original report is that you use BOINC. Extradata are tweaks of the configuration information of a VBox VM (VBoxManage setextradata ...). I don't know if BOINC does such tweaks.

[Frank Mehnert]

Also please attach the complete VBox.log file, not only an excerpt of it. Use the 'Attach file' button.

[Jacob Klein]

I will monitor to see if this happens again. But right now, I have no additional information to provide. The BOINC task resulted in Computation Error, the slot has been cleared, and the VM and the log data have been removed. Sorry; I'm still not quite an expert on VM BOINC tasks, and debugging them, yet.

[Jacob Klein]

This issue has happened again. This time, I tried to save a local copy of the BOINC task and Oracle logs. I'll attach some of the logs that I hope are relevant.

Can you please investigate this problem further? Thanks.

[Jacob Klein]

Logs are attached. Please let me know what you find.

[Frank Mehnert]

Now I see the problem. Your settings file contains a bandwidth limitation for the network but there is no bandwidth group name defined. The missing check is probably a bug in VBox which needs to be fixed. But for now do

VBoxManage bandwidthctl boinc_f09401e13317bc7f list

To fix your VM I think you have to manually edit the settings file. Search for BandwidthGroup name and change the name from "" to some name, for example "foo".

Do you have any idea how this bandwidth is set up for your VM or is this "BOINC magic"?

[Jacob Klein]

I saved a copy of the VM data, but the BOINC task failed, and I don't really have any intention/need of modifying the existing VM to get it working again.

Regarding the bandwidth setup, BOINC's vboxwrapper changed a couple months ago. It had been using --nicspeed0 I believe, but Rom changed it to use the more modern (and documented) bandwidth groups. He sets up a single group for each task, giving the group a name that should be unique to the BOINC task name. The tasks referenced in this ticket have only ever seen the "bandwidth group" code.

I'll relay your comment/findings to him (since it sounds like BOINC may have done something wrong here), but it'd also be great if VirtualBox could better-handle this situation; do you agree?

Thanks, Jacob

[Frank Mehnert]

Yes, there is now a fix which shows an error message which makes more sense. But VirtualBox will still refuse to start this VM. So from our perspective, the bug is fixed (in trunk and in the 4.3 branch).

[Jacob Klein]

Just to confirm... Your change is to still error, but show an error that makes more sense, correct? And the "real cause" is likely some sort of problem setting up the bandwidth group, to where the group gets named "" (empty string). Do I have that right?

[Frank Mehnert]

Yes, that's correct.

[Jacob Klein]

Thanks Frank. I'll work with Rom to see if we can figure out if we're doing anything wrong.

[Rom Walton]

I managed to repro this issue and was able to snag the trace log to go with it.

The zip file contains vbox_trace.txt which contains all the commands used to create the VM and any output the command returned.

The last three commands were:
VBoxManage -q showvminfo "boinc_bf33e2819654dcff" --machinereadable
VBoxManage -q snapshot "boinc_bf33e2819654dcff" restorecurrent
VBoxManage -q startvm "boinc_bf33e2819654dcff" --type headless

The output to showvminfo shows the bandwidth group:
...
BandwidthGroup0=boinc_bf33e2819654dcff_net,Network,524288
...

After we restore the current snapshot it is missing and startvm returns with:
Waiting for VM "boinc_bf33e2819654dcff" to power on...
VBoxManage.exe: error: The path of a new node contained slashs or was empty. (VERR_CFGM_INVALID_NODE_PATH)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component Console, interface IConsole

[Rom Walton]

I have seen this with our new wrapper an additional time.

I think the scenario that sets the stage as it were is:

1. Create the VM
   
2. Poweron VM
   
3. Run for awhile
   
4. Create snapshot
   
5. Poweroff VM
   

6. Restore current snapshot
   
7. Poweron VM
   
8. Poweroff VM
   

9. Restore current snapshot
   
10. Poweron VM (VERR_CFGM_INVALID_NODE_PATH error)
    

[Frank Mehnert]

Thanks guys! We are now able to reproduce the problem and will fix it soon.

[Frank Mehnert]

We fixed the problem. Here is a 4.3.7 test build. The fix will be part of the next 4.3.x maintenance release.

[Jacob Klein]

Thanks, frank. Do you happen to know which changeset fixed the problem? (I'm curious)

[Frank Mehnert]

It's r50436. See src-server/BandwidthControlImpl.cpp line 163.

[Jacob Klein]

Thank you!

[Rom Walton]

Looks like we have confirmation on the fix with the 4.3.7 build: ​http://lhcathome2.cern.ch/test4theory/result.php?resultid=4215236

[Frank Mehnert]

Fix is part of VBox 4.3.8.