[vbox-dev] Too Many Snapshots: XML_PARSE_HUGE and NS_ERROR_CALL_FAILED

[Klaus Espenlaub]

not sure if this is something worth fixing, as a huge number of 
snapshots causes trouble elsewhere (stack overflows). That's the reason 
why the snapshot nesting depth has been limited to 250 with VirtualBox 
4.3.0.

On 30.06.2014 18:28, a. wrote:
> So we are trying to get a problem with VirtualBox working. We are
> heavily using the snapshot feature and hit a wall, as we probably have
> exceeded the maximum possible amount of active snapshots. As each
> Snapshot is written into the VBOX-file as a descendent of previous
> Snapshot, at one point, the nesting exceeds 256 entries - which is an
> arbitrary limit set by the libxml2. When trying to register a VM which
> has too many Snapshots, vboxmanage registervm fails with an error.

I've checked the code of libxml 2.6.31, and the only occurrence of 
XML_PARSE_HUGE is in some documentation. This means your patch will 
break our builds by referring to an undefined symbol.

Actually I never saw trouble parsing big XML files with the libxml2 
version we're using in all our builds, 2.6.31. It sets the variable 
xmlParserMaxDepth to 1024, which is plenty.

> We tried to work around this issue simply by adding the corresponding
> XML_PARSE_HUGE flag in the function that calls to the xml-library. We
> also created a patch:
>
> --- virtualbox-4.1.12-dfsg.orig/src/VBox/Runtime/r3/xml.cpp
> +++ virtualbox-4.1.12-dfsg/src/VBox/Runtime/r3/xml.cpp
> @@ -1500,7 +1500,7 @@ void XmlMemParser::read(const void* pvBu
>                                                      (int)cbSize,
>                                                      pcszFilename,
>                                                      NULL,       //
> encoding = auto
> - XML_PARSE_NOBLANKS | XML_PARSE_NONET)))
> + XML_PARSE_NOBLANKS | XML_PARSE_NONET | XML_PARSE_HUGE)))

XML_PARSE_HUGE was apparently introduced with version 2.7.3, and thus 
this needs proper version checking to make it an acceptable patch.

>            throw XmlError(xmlCtxtGetLastError(m_ctxt));
>
>        doc.refreshInternals();
> @@ -1630,7 +1630,7 @@ void XmlFileParser::read(const RTCString
>                                                  &context,
>                                                  pcszFilename,
>                                                  NULL,       // encoding
> = auto
> -                                              XML_PARSE_NOBLANKS |
> XML_PARSE_NONET)))
> +                                              XML_PARSE_NOBLANKS |
> XML_PARSE_NONET | XML_PARSE_HUGE)))
>            throw XmlError(xmlCtxtGetLastError(m_ctxt));
>
>        doc.refreshInternals();
>
> We successfully rebuild the source with that flag usign
> dpkg-buildpackage.

Would only compile with a new enough libxml2...

> That successfully resolved the previous error,
> however we now have the following problem by VBoxManage when trying to
> register the VM in question:
>
> vboxmanage registervm /mnt/storage/vm-vbox
> VBoxManage: error: Code NS_ERROR_CALL_FAILED (0x800706BE) - Call to
> remote object failed (extended info not available)
> Context: "OpenMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at
> line 90 of file VBoxManageMisc.cpp
>
> Unfortunately, we have no idea how to work with that error.

That's the mentioned stack overflow elsewhere in the API. When XPCOM is 
used (i.e. on all platforms besides Windows), the stack of each thread 
is limited to 128K, to maximize the scalability. That's not a lot...

> This is the VBoxSVC.log:
> VirtualBox (XP)COM Server 4.1.12_Ubuntu r77245 linux.amd64 (Jun 30 2014
> 16:40:09) release log
[...]
> 00:00:00.200 nspr-2   Loading settings file "/mnt/storage/vm.vbox" with
> version "1.12-linux"

Rather clear hint that the processing of this settings file fails after 
the XML parsing step.

> Any help would be immensely appreciated.

Do you truly have a use case which can't exist without such a deep 
snapshot nesting (the total snapshot count is still unlimited)? It costs 
some efficiency to have data spread across so many disk images.

Klaus