Hardening rejects DLL because of expired certificate nvldumdx.dll

[Peter B]

Im using a Nvidia GTX960 video card and it seems like the driver loader dll gets rejected because of an expired certificate.

The thing is that nvldumdx.dll is signed 2 times:

1. by Nvidia SHA1 (expired cert, was valid from 2018-2019)
2. by Microsoft SHA256 (valid cert)

Since Nvidia releases the file like that, I assume it's ok that only one of the signatures is valid, which is the case. But VirtualBox complains about the expired cert in the hardening log and rejects the file.

(VBox and video drivers are both the most recent releases)

2bc0.d1c: supHardenedWinVerifyImageByHandle: -> -23033 (\Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll) WinVerifyTrust
2bc0.d1c: Error (rc=0):
2bc0.d1c: supR3HardenedScreenImage/LdrLoadDll: rc=Unknown Status -23033 (0xffffa607) fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll: Certificate is not valid (ValidTime=2020-07-05T18:39:56.000000000Z Validity=[2018-07-18T17:42:53.000000000Z...2019-07-18T17:42:53.000000000Z]): \Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll
2bc0.d1c: supR3HardenedWinVerifyCacheInsert: \Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll
2bc0.d1c: Error (rc=0):
2bc0.d1c: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll): rcNt=0xc0000190
2bc0.d1c: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0xc0000190 'C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll'
2bc0.d1c: supR3HardenedScreenImage/LdrLoadDll: cache hit (Unknown Status -23033 (0xffffa607)) on \Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll
2bc0.d1c: Error (rc=0):
2bc0.d1c: supR3HardenedScreenImage/LdrLoadDll: cached rc=Unknown Status -23033 (0xffffa607) fImage=1 fProtect=0x0 fAccess=0x0 cHits=1 \Device\HarddiskVolume4\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_edab19158bdd0d0a\nvldumdx.dll
2bc0.d1c: Error (rc=0):

[Frank Batschulat (Oracle)]

see also:

Ticket #16531 VirtualBox has issue With my Windows 10 Build 15046

and:
​https://forums.virtualbox.org/viewtopic.php?f=6&t=99059
​https://forums.virtualbox.org/viewtopic.php?f=3&t=98777

[jacobd]

Just confirming this issue persists with:

VirtualBox 6.1.12 - Windows 10 Host, Debian Linux Guest. Nvidia Drivers - 451.85

Here is a thread on the NVIDIA forums as well: ​https://www.nvidia.com/en-us/geforce/forums/game-ready-drivers/13/385903/nvidia-driver-45148-appears-to-break-virtualbox-3d/

[jacobd]

Suggestion as to how the logic could work from a development perspective (from Open BSD): ​https://github.com/openbsd/src/commit/68c77ca0787122c2d29bbd258a3b565af1f323bf

[Klaus Espenlaub]

The issue here is rather different. It's not directly a problem with expired certificates in the chain. The problem is that the SHA1 signature is not verifiable (and Windows 10 actually says "The certificate in the signature cannot be verified."

Nvidia uses a internal CA to issue the SHA1 certificate (which is issued by 'NVIDIA Subordinate CA 2018-Prod-Sha1' for 'NVIDIA Corporation-PE-Prod-Sha1'), and since no one knows about this CA, the signature cannot be ever verified by anyone outside Nvidia.

So we're in the grey zone of signature validation: there is one signature which is valid but the certificate chain does not ever go to any trusted root CA certificates (and no immediately obvious reason why Nvidia even bothers having such a signature), and the other one (a SHA256 one from Microsoft) is valid.

Pretty quirky setup, in my entirely personal opinion, to have not a single valid signature which is from the entity responsible for the code. Sure, Microsoft only adds their signature there if they get the whole batch of driver files in an CAB file which is EV signed by the submitter, but it's still a little strange that the end user can't see any valid signature from Nvidia.

[jacobd]

This issue is still present in VirtualBox 6.1.14 and Nvidia Game Ready Drivers: 456.38 (September 2020).

[Mears148]

I too am having this issue. I am surprised this isn't brought up more by other users, considering the amount of people using NVIDIA cards. Here are my logs running on a GeForce GTX 980 w/ 456.38 WHQL driver from NVIDIA and VirtualBox 6.1.14:

00:00:03.277664 VMEmt: Halt method global1 (5)
00:00:03.277742 VMEmt: HaltedGlobal1 config: cNsSpinBlockThresholdCfg=50000
00:00:03.277758 Changing the VM state from 'CREATING' to 'CREATED'
00:00:03.278699 Changing the VM state from 'CREATED' to 'POWERING_ON'
00:00:03.347162 supR3HardenedErrorV: supR3HardenedScreenImage/LdrLoadDll: rc=VERR_CR_X509_CPV_NOT_VALID_AT_TIME fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume6\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll: Certificate is not valid (ValidTime=2020-09-14T19:10:45.000000000Z Validity=[2018-07-18T17:42:53.000000000Z...2019-07-18T17:42:53.000000000Z]): \Device\HarddiskVolume6\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll
00:00:03.347242 supR3HardenedErrorV: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll): rcNt=0xc0000190
00:00:03.347527 supR3HardenedErrorV: supR3HardenedScreenImage/LdrLoadDll: cached rc=VERR_CR_X509_CPV_NOT_VALID_AT_TIME fImage=1 fProtect=0x0 fAccess=0x0 cHits=1 \Device\HarddiskVolume6\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll
00:00:03.347559 supR3HardenedErrorV: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll): rcNt=0xc0000190
00:00:03.347848 supR3HardenedErrorV: supR3HardenedScreenImage/NtCreateSection: cached rc=VERR_CR_X509_CPV_NOT_VALID_AT_TIME fImage=1 fProtect=0x2 fAccess=0x5 cHits=2 \Device\HarddiskVolume6\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll
00:00:03.348097 supR3HardenedErrorV: supR3HardenedScreenImage/NtCreateSection: cached rc=VERR_CR_X509_CPV_NOT_VALID_AT_TIME fImage=1 fProtect=0x2 fAccess=0x5 cHits=3 \Device\HarddiskVolume6\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_324f172f3e662ec5\nvldumdx.dll

[theRman]

Replying to klaus:

The issue here is rather different. It's not directly a problem with expired certificates in the chain. The problem is that the SHA1 signature is not verifiable (and Windows 10 actually says "The certificate in the signature cannot be verified."

Nvidia uses a internal CA to issue the SHA1 certificate (which is issued by 'NVIDIA Subordinate CA 2018-Prod-Sha1' for 'NVIDIA Corporation-PE-Prod-Sha1'), and since no one knows about this CA, the signature cannot be ever verified by anyone outside Nvidia.

So we're in the grey zone of signature validation: there is one signature which is valid but the certificate chain does not ever go to any trusted root CA certificates (and no immediately obvious reason why Nvidia even bothers having such a signature), and the other one (a SHA256 one from Microsoft) is valid.

Pretty quirky setup, in my entirely personal opinion, to have not a single valid signature which is from the entity responsible for the code. Sure, Microsoft only adds their signature there if they get the whole batch of driver files in an CAB file which is EV signed by the submitter, but it's still a little strange that the end user can't see any valid signature from Nvidia.

It maybe interesting to note, that in certain older NVidia driver releases (e.g. 445.87), the offending file was signed with a cert, where the entire chain was valid at the time of singing - in 2020. But this particular "NVIDIA Corporation-PE-Prod-Sha1" expired in June 2020. For Nvidia now to use an even older one, instead of newer one, seems like some human error when updating the signing certificate.

But all this is besides the point. What we have is one cert chain (Nvidia one) that is untrusted for various reasons (also because SHA1 signing is deprecated after all). And we have another chain that is fully valid and has known & valid intermediate and root CA (Microsoft). The driver dll must be accepted as valid if any chain can be fully validated. I do not see how this is any kind of "gray zone". Also it should be noted that Windows own driver loading mechanism sees this driver as valid when signed this way. Also if it would be a Microsoft created driver (as they do for AHCI for example), it would have no Nvidia signature at all. Thus IMHO VirtualBox must handle it in the same manor as Windows and check all cert cains.

[Klaus Espenlaub]

Yes, we always planned to fix this ourselves (hoping for Nvidia to be faster with fixing the crazy SHA1 cert stuff), but the world isn't as simple as "The driver dll must be accepted as valid if any chain can be fully validated."

If it were true in this absolute sense then someone who wants to tweak the binary would "only" need to spend the money to come up with a SHA1 collision. Which is much cheaper than faking a valid SHA256 signature/cert. Guess why all the browsers stopped accepting SHA1 certs a while ago.

Today there needs to be some kind of SHA256 preference or we'll end up accepting binaries which Windows shouldn't accept any more in the Windows 10 day and age. This will increase the complexity of the fix significantly, and in this area complexity is the synonym "it will need longer, because there needs to be more testing to be sure everything works".

[theRman]

Well, you are right. It of course also depends on what one accepts as "valid chain" at all.

But until Windows blocks drivers signed with SHA1, the issue of SHA1 hash collisions is a separate one. One can also take the older Nvidia driver, correctly signed with SHA1 by Nvidia and modify the binary there. Maybe even delete the MS-signature for good measure. Then the current implementation would accept it happily.

It also valid point to reduce testing by involving such other cases. But maybe they also complicated the issue. And as for waiting for Nvidia... this issue is also raised with them and seemingly they just do not care. Most likely because the MS signature fixes it as far as Windows is concerned.

Maybe one just needs to write some tool to strip the broken nvidia signature from the dll, if that is possible. It should be, as both signatures should be independent and sign only the actual dll content. Then at least there would be some workaround and people can use it, if they really want.

[Ethics Gradient]

Just commenting to say I am also affected by this issue and get update notifications.

[Roze]

Also affected by this. I was so happy to have finally gotten HW acceleration to work on my guest. Then a couple of weeks ago I had to reinstall system. Which involved updating graphics drivers. And now upon starting Virtual Box again and booting my OpenWRT Development machine, HW is broken again. This as a result of hardening certificate issue.

[theRman]

So... I have now created some tool to remove the SHA1 Nvidia Signature from the files VirtualBox was complaining about (nvldumdx.dll, nvd3dumx.dll).

Now I get no more Hardening errors in VBox.log. Also File Properties of the affected dlls now shows single chain from MS and states its valid.

But still 3d does not work. When looking at the VBoxHardening.log I can see the following (seemingly relevant) lines:

2298.3238: \Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll: Owner is not trusted installer (01 05 00 00 00 00 00 05 15 00 00 00 03 ef 9b de 79 a2 df 61 15 b2 80 94 ea 03 00 00)
2298.3238: \Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll: Relaxing the TrustedInstaller requirement for this DLL (it's in system32).
2298.3238: supHardenedWinVerifyImageByHandle: -> 0 (\Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll) WinVerifyTrust
2298.3238: supR3HardenedWinVerifyCacheInsert: \Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll
2298.3238: supR3HardenedDllNotificationCallback: Unload 00007ffa5ec40000 LB 0x00102000 C:\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll [flags=0x0]
2298.3238: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0x0 hMod=00007ffa76de0000 'C:\Windows\System32\gdi32.dll'
2298.3238: supR3HardenedScreenImage/NtCreateSection: cache hit (VINF_SUCCESS) on \Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll
2298.3238: supR3HardenedMonitor_NtCreateSection: NtMapViewOfSection failed on 000000000000116c (hFile=0000000000001164) with 0xc0000022 -> STATUS_TRUST_FAILURE
2298.3238: supR3HardenedScreenImage/NtCreateSection: cache hit (VINF_SUCCESS) on \Device\HarddiskVolume11\Windows\System32\DriverStore\FileRepository\nvcvi.inf_amd64_3bbc87e96de023f6\nvldumdx.dll
2298.3238: supR3HardenedMonitor_NtCreateSection: NtMapViewOfSection failed on 0000000000001164 (hFile=000000000000116c) with 0xc0000022 -> STATUS_TRUST_FAILURE

Do I interpret this correctly, as Windows now rejecting the (modified) dll? Is there any way to find out why?

[Danial]

Replying to klaus:

The issue here is rather different. It's not directly a problem with expired certificates in the chain. The problem is that the SHA1 signature is not verifiable (and Windows 10 actually says "The certificate in the signature cannot be verified."

Nvidia uses a internal CA to issue the SHA1 certificate (which is issued by 'NVIDIA Subordinate CA 2018-Prod-Sha1' for 'NVIDIA Corporation-PE-Prod-Sha1'), and since no one knows about this CA, the signature cannot be ever verified by anyone outside Nvidia.

So we're in the grey zone of signature validation: there is one signature which is valid but the certificate chain does not ever go to any trusted root CA certificates (and no immediately obvious reason why Nvidia even bothers having such a signature), and the other one (a SHA256 one from Microsoft) is valid.

Pretty quirky setup, in my entirely personal opinion, to have not a single valid signature which is from the entity responsible for the code. Sure, Microsoft only adds their signature there if they get the whole batch of driver files in an CAB file which is EV signed by the submitter, but it's still a little strange that the end user can't see any valid signature from Nvidia.

That's not whats going on.

The issue is 2 fold in that <1 Oracle is performing an incomplete validation chain check and failing the moment it see's a signature older than the current system date/time, even though the certificate is countersigned and in the validation period.

and <2 that its not checking for valid certificates first. In the case of a dual signed binary (Nvidia + WHQL) the WHQL certificates existence should take precedent over any other signature present.

[Klaus Espenlaub]

What? The SHA1 signature is not verifiable. No one but Nvidia knows about their internal CA, leave alone would trust it. Either way, that's not what upsets VirtualBox. This quirky SHA1 signature isn't new, it was there for a long time. The only thing which is new is that since July 2020 Nvidia is doing SHA1 signing with a certificate which is well over a year beyond its expiry. This triggers the bug in VirtualBox, making it bail out instead of checking the 2nd signature (the WHQL one) which is indeed valid.

[Danial]

There are two signatures (with a timestamp each, that is also a signature). One is invalid with valid timestamp [Authenticode style] and the other is valid with valid timestamp. IF ONLY ONE signature (not timestamps even) is right, it is verified.

This is oracle developers playing cowboy with their code, always has been.

[bird]

The problem should be fixed with 6.1.15-140873 and later. Fresh build on the Testbuilds page. Test feedback is extra appreciated this time as the changes aren't entirely trivial (we didn't used to verify more than the first signature, now we check them all).

[Cruncher]

Post deleted. It works with Oracle_VM_VirtualBox_Extension_Pack-6.1.15-140899.vbox-extpack. I used VBoxGuestAdditions_6.1.15-140877.iso when the bugs occured.

[theRman]

With 6.1.15-140899 it also works for me. Strangely the hardening log does not mention the nvidia dlls anymore at all (not SUCCESS, not failed, just nothing at all)