VirtualBox on Windows 11 AMD 5950x host, w/AMD W6800 Pro GFX, Guest Additions hang/freeze or blinking

[vboxusr7]

Setup:

• VirtualBox version 7.0.8 r156879 (Qt5.15.2)
• Host: Windows 11 Pro (22H2, OS Build 22621.1555)
• Guest: Windows 11 Pro (22H2, OS Build 22621.1555)
• Host processor: AMD 5950x
• Host graphics: AMD W6800 Pro Graphics

Observations:

• I original had issues with what seems to be the well-known VirtualBox/Windows 11 "Guest Additions" post-boot freeze/hang ("GABootHang").
• I was originally able to workaround the GABootHang issue by enabling 3D Acceleration, but after doing so, I noticed the guest graphics exhibiting odd behavior...
• With 3D Acceleration enabled, I would see a transparent Windows Start Menu which itself would flicker/blink (example attached), and other areas of the UI were odd, like blank buttons (or buttons with text matching closely to the background such that they cannot be seen).
• I tried test build VirtualBox-7.0.9-156993-Win.exe but did not see any difference.
• I was able to use the VM with 3D Acceleration enabled to some extent by just dealing with the graphics anomalies...
• Oddly, though, after trying test builds, other steps, I recently noticed that simply dragging the above mentioned 7.0.8 r156879 build from one monitor to another, the VM itself would crash. I also noticed the same sort of crash when maximizing/minimizing the VM window. What is strange about this is I had not seen this before... maybe I had not dragged the VM nor maximized/minimized much, but was wondering if there was some "testing pollution" given installing/removing the test builds. I did a fresh install of 7.0.8 r156879 and removed it from Safe Mode, and fresh-installed again, just to try to clean things up, but I observed the same behavior described above.
• Overall, with my AMD-CPU/GFX setup, it seems VirtualBox is challenged to a degree that makes it so I cannot use it for Windows 11 on this host.
• (Note, I have used VirtualBox extensively with Linux on this same hardware/host OS, and it has been excellent, wonderful... no issues... so something about Windows 11 was Guest OS, and Guest Additions, and perhaps this hardware.)
• After searching around, I discovered forum discussions above the same/similar issues... the so-called the post-boot freeze (GABootHang), and the blinking Start Menu, all seem to be known experienced issues... so maybe it's not a compat issue with my host hardware but maybe something more generic. I cannot say more without digging further.
• The perfect storm for me here is this: I experienced GABootHang, used 3D Accel to workaround it, but that leads to different stability issues. To reiterate above, Linux as guest on same host has been wonderful... we are talking hours and hours, days of work with Linux and Guest Additions, no issues.

Following are forum posts which may or may not be related to above described experiences (they seem to be the same/overlapping)...

​https://forums.virtualbox.org/viewtopic.php?f=2&t=108880 My 64-bit W11 22H2 Home guest VM keeps locking up randomly.

​https://forums.virtualbox.org/viewtopic.php?f=2&t=108399 Host Win11-VM with Win11 works under VB 6.1.x under VB 7.0.x VM freezes after a few minutes

[vboxusr7]

VBox-log-Win11-guest-additions-3D-accel-freeze-hang.log

[vboxusr7]

VBoxHardening-log-Win11-guest-additions-3D-accel-freeze-hang.log

[fth0]

To subscribe to this ticket ;) and to give you some key information at the same time:

00:02:19.963010 GIM: HyperV: Guest indicates a fatal condition! P0=0x10e P1=0xb P2=0xffffd909991c4390 

I've seen this BSOD (Bug Check 0x10E: VIDEO_MEMORY_MANAGEMENT_INTERNAL) in other users' VBox.log files on the forums a few times in the previous days. This observation could be limited to VirtualBox 7.0.8+, but I'm not sure about that.

---

00:00:02.577627 NEM: NEMR3Init: Snail execution mode is active!

VirtualBox is running under Hyper-V instead of using VT-x directly. This will probably have a negative performance impact. In the VM window's status bar this is indicated by a green turtle icon.