VirtualBox freezes on btrfs / add check for COW

[Fabio Coatti]

On a linux machine with btrfs filesystem, installation of a guest goes just fine but at first restart the guest freezes with popup showing "20%" during launch phase. Installing the same machine on XFS, no problems at all. I've seen similar reports, but with slightly different behaviours and declared fixed. host machine: linux 3.9.5 x86_64 (gentoo) guests: W8/W7 64bit attached: vbox.log of failed session

[Fabio Coatti]

Vbox.log

[Michael Thayer]

Was this a one-time occurrence or is it reproducible?

[Fabio Coatti]

Replying to michael:

Was this a one-time occurrence or is it reproducible?

Happens every time, always reproducible.

I can add that I tested both packages available on gentoo (one built from sources, the other using binary from virtualbox site) and the behaviour is the same.

[Ufven]

I'm experiencing the same issue running Fedora 19 and btrfs. However, if I enable "Use host I/O cache" on the VM's SATA controller, the problem goes away.

[Frank Mehnert]

In that case I assume that there is some problem when opening files with O_DIRECT on btrfs. We saw such problems with EXT4 and XFS before with older Linux kernels.

[Fabio Coatti]

After some day, I'm no longer able to reproduce the issue. I don't know if this is relevant, as I've got no time to check kernels changelog, but I changed kernel and now I'm running on 3.10 and I'm not able to reproduce the same behaviour.

[Frank Mehnert]

Thanks, let's close this ticket then.

[zerolagtime]

I experienced this same problem on BTRFS with Ubuntu Trusty (14.04) on the 3.13.0 kernel and btrfs-tools 3.12. During any virtual machine operation, the system would thrash the disk very badly. Eventually, it would get so bad that the whole system would freeze.

I did some digging on the btrfs wiki and discovered that Copy-On-Write (COW) causes this pain for any virtualization solution (​https://wiki.archlinux.org/index.php/Btrfs#Copy-On-Write_.28CoW.29). All I did was go to the location of every VDI and type chattr +C file.vdi and on the whole directory. The VM was powered off while I did this.

During the next boot, it was quite fast and the thrashing was gone.

Now, I also took the advice from above and also marked it to use the hosts I/O cache, but that didn't fix it.

[zerolagtime]

Please add a check when opening a virtual machine image under Linux. If any images are stored on a BTRFS filesystem, and the COW flag is not activated, put up a warning flag that a file system flag may severely degrade performance. Offer to suspend the system, change the attribute on all applicable files, and then resume the system.

This applied even under VirtualBox 4.3.18.

[Frank Mehnert]

I think this check makes sense. Actually a feature request...

[krichter]

little update on the symptoms on a Linux 3.19 host with 4.3.20 and 4.3.22: after moving the VM folder from one btrfs filesystem to another (btrfs) the hang moves to a point in time after boot, i.e. the starting of the VM succeeds, but then the guest boot hangs.

[Jan Hnatek]

+1 for the warning dialog (seeing the issue on 4.3.32, 5.0.8, 5.0.10).

[rsd]

Please this is more important than an enhancement.

When creating a image (.vdi file) please add the no Copy on Write attribute while it still empty.

chattr +C only works it the file is empty, so it is a major pain to work around full images.

Right now the solution has being the one pointed here: [​http://superuser.com/questions/623989/virtual-box-stuck-at-starting-virtual-machine-0 ]

Using this script to correct the image:

F=vmimage
T=tmpfile
touch $T
chattr +C $T
dd if=$F of=$T bs=1M
rm $F
mv $T $F

[rfkrocktk-oracle]

I am also affected by this in bug #15728.

If there's a bug in BTRFS, I'd be happy to bisect/debug the kernel and submit a patch for it. Is there any consensus on the exact nature of the problem, so I can have a good starting point for my search?

Alternatively, is there a way for us to have VirtualBox do the chattr +C fix if it detects that the VirtualBox VMs folder is on a BTRFS filesystem? Is there a mount flag we can use to disable CoW on BTRFS?

[Mo_]

Any news here? I have all my VBox images on a separated subvolume without snapshots, defragmented and enabled nodatacow:

# chattr -VR +C /mnt/btrfs-top-lvl/data/VirtualMachines/
# lsattr /mnt/btrfs-top-lvl/data/VirtualMachines/
----------------C-- /mnt/btrfs-top-lvl/data/VirtualMachines/OpenSuse LEAP
----------------C-- /mnt/btrfs-top-lvl/data/VirtualMachines/vmgentoo_2
----------------C-- /mnt/btrfs-top-lvl/data/VirtualMachines/OpenSuse Tumbleweed new
----------------C-- /mnt/btrfs-top-lvl/data/VirtualMachines/Windows 7 Professional

The Windows 7 VM gets bluescreens on btrfs, while running fine on an ext4. The Linux VMs are running fine on both. I have disabled the swapfile in Windows and tried disabling write cache on the VBOX hdd, but that's not supported.

[Michael Thayer]

When I look at the chattr(1) man mage it says that +C on folders is automatically applied to new files created in the folders. Does it also apply recursively to new sub-folders?

Questions which come to mind are: is this a fundamental problem of copy on write, or just the btrfs implementation of it? And can anyone think of reasons why people might still want to use copy on write with virtual machine images?

I looked at the code for chattr, and it calls ioctl(fd, EXT2_IOC_GETFLAGS, &f) (fd an open file descriptor for the file to change, f an integer) to get the current flags and ioctl(fd, EXT2_IOC_SETFLAGS, &f) (where f was or-ed with FS_NOCOW_FL) to add the NOCOW flag.

[Michael Thayer]

Looking at reports of people running VirtualBox on ZFS suggests that this may indeed be a btrfs problem. Perhaps someone can talk to the btrfs people?

[bam]

Has something changes since then?

[Michael Thayer]

@bam Are you experiencing this? I don't know about anyone seeing this who has talked to the btrfs developers, so perhaps you might try.