Freeze on Oracle Linux 9.1 w. UEK Kernel

[Egon Rath]

Issue: VirtualBox 7.0.4 freezes immediatly when started on Oracle Linux 9.1 with the default UEK kernel running. It freezes so hard that not even a single line is written to the kernel log, so i have not even an idea what is causing this. This does not occur when the system is booted with the RedHat-Compatible Kernel (RHCK).

How to reproduce:

1. Install Oracle Linux 9.1 and apply all updates
2. Install VirtualBox 7.04 official RPM build
3. Create any VM and start it

The system i have used to reproduced the behavior is a AMD Epyc with AMD-V enabled if that makes any difference.

[galitsyn]

Hi egrath,

What exactly freezes? A VM process/GUI or entire host/Desktop Manager? What is the UEK kernel version?

[Egon Rath]

The entire system freezes completely as soon as a virtual machine is started. With VirtualBox 7.0.6 the behavior is similar (complete freeze), but the system reboots a few seconds after.

The last message written to the system log seems to come from VirtualBox:

SUPR0GipMap: fGetGipCpu=0x2b

There are no other messages regarding virtualbox or a indication on what exactly happened.

Running UEK 5.15.0-6.80.3.1.el9uek.x86_64

[galitsyn]

Hi egrath,

Could you please provide kernel crash report as generated by kdump tools (or at least dmesg out of it)?

[Egon Rath]

Here you are:

​https://egon.bot.nu/nextcloud/s/aotdPFoYZwamK2y

(I had to put it on my Nextcloud as i can't attach files that big to the ticket)

[galitsyn]

Hi egrath,

Thank you for crash dump. Cannot immediately tell why it is happening. Need to take a closer look. I will leave a comment once I find something.

[galitsyn]

Hi egrath,

I see in dmesg that you have AMD Memory Encryption enabled. Don't have AMD host at hand, but what if you try to disable it in BIOS/EFI settings? Does it change anything?

[Egon Rath]

This seems to be a false positive, as the CPU in this particular device does not supports memory encryption. To be absolutely sure i checked with AMD's TSME-Test driver (​https://github.com/AMDESE/mem-encryption-tests) which verifies that it's not enabled:

TSME Test: TSME is not active

[galitsyn]

Hi egrath,

Correct, TSME (Transparent Secure Memory Encryption) is not reported by CPU (AMD Ryzen 7 PRO 2700X in this case), however SME does:

[    0.101168] AMD Memory Encryption Features active: SME

From my understanding, SME implies corresponding option to be enabled in both BIOS/EFI settings and kernel configuration to take effect. From what I can see in my /boot/config-5.15.0-6.80.3.1.el9uek.x86_64:

CONFIG_AMD_MEM_ENCRYPT=y
CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y

So, it looks like it is enabled by default in kernel. I would still recommend to try to disable setting in BIOS/EFI and/or boot host kernel with "mem_encrypt=off" parameter.

[Egon Rath]

Thanks for the clarification on the memory encryption, didn't knew that those are different things.

When booting with mem_encrypt=off the issue no longer persists. To permanently enable the workaround i've issued the following command in case someone else needs that too:

sudo grubby --update-kernel=/boot/vmlinuz-$(uname -r) --args="mem_encrypt=off"

Unfortunately, the BIOS on my HP workstation does not exhibit this as a user-changeable setting ... as the above workaround needs to be applied every time the kernel gets updated, it would be nice if the virtualbox kernel module plays nice with SME.