Opened 4 years ago
Closed 3 years ago
#19695 closed defect (fixed)
Corrupted downloads over network with hyperv backend
| Reported by: | bendem | Owned by: | |
|---|---|---|---|
| Component: | network | Version: | VirtualBox 6.1.10 |
| Keywords: | corruption | Cc: | |
| Guest type: | Linux | Host type: | Windows |
Description
I've been trying to use VBox with the hyperv backend to run ansible molecule tests. Since I'm running this inside WSL, I can't use the virtualbox backend.
I'm getting weird corruptions when downloading stuff from the internet.
Running from my host system:
➜ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
Running inside the guest VM:
[vagrant@instance01 ~]$ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
07db5c367b7e1395a0ed725f184ef093669c2680ab1a88e3b60909d7e0543a72 -
aeaf6054b4cd865031ccb4a505cee358bffe1c8e9cceca24ec2696308a4815cd -
36ba4ce9fd41db373febe4fda7678b3d1769090e7a6f68ba67c04badc2b02369 -
90c6e105ecc3d5b77aea257b3e287b0cae2f74e2974c01657a64cc1b4d7df36a -
57d1fd185bc5bfa100338f1c200b91cd8cf8649775250fa32876056d882cf64f -
cab806c4f8ae9bc5d59127c417bc1532faed58026103d005b56080c830345b3b -
f46d2f63a84497678f5dae7f158c296d2e73eed5ebe0a1795da0a96d90664b57 -
51a2a7ce5f67607162cca7e05739d7b57460712ee1593d9e2b17b30fea23711c -
044cd79f32c3b51546265ab6e70991533e70ad2071814169832ce20a3dd99811 -
7c9aca42a718c275650c28681a2fd80de01f50f5632da0a08a60facba5b5d1c8 -
307248c92a464c35d45686f3ca438683b5f8de41eb6f01f5bf27a778f6d5579d -
I can reproduce that in centos 7, 8 and debian buster. It renders any network interaction dangerous as anything could be corrupted (god bless package managers checking for file integrity).
More context: I'm running molecule using a vagrant driver that invokes vagrant inside WSL. Vagrant knows it runs in WSL and escapes it to run VBoxManager.exe from the host system, basically meaning that me running inside WSL should not change anything.
Attachments (2)
Change History (35)
by , 4 years ago
comment:1 by , 4 years ago
This shows the corruption is generally very minimal (generally a few bytes) but never at the same offset:
vagrant@buster:~$ diff <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 |hexdump -C) <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | hexdump -C) 131619,131620c131619,131620 < 00202280 48 c9 d4 f3 aa 77 41 ac 27 ec 2d e9 8a 4a d2 37 |H....wA.'.-..J.7| < 00202290 fd ce 98 df 8f bd 27 07 cb 70 41 f7 32 ff b7 41 |......'..pA.2..A| --- > 00202280 48 e6 75 14 1f 73 6f 4f 66 e5 5e 21 df 02 90 03 |H.u..soOf.^!....| > 00202290 31 ce 98 df 8f bd 27 07 cb 70 41 f7 32 ff b7 41 |1.....'..pA.2..A| vagrant@buster:~$ diff <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 |hexdump -C) <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | hexdump -C) vagrant@buster:~$ diff <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 |hexdump -C) <(curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | hexdump -C) 54478,54479c54478,54479 < 000d4d30 3e 62 41 42 03 b4 4c de 3b 26 a5 2b fb 5c 7f 73 |>bAB..L.;&.+.\.s| < 000d4d40 d3 b4 3c 35 03 33 61 46 12 08 24 40 e4 95 0c c9 |..<5.3aF..$@....| --- > 000d4d30 3e 33 21 10 03 9e 49 15 35 28 c8 a8 41 26 1a 62 |>3!...I.5(..A&.b| > 000d4d40 10 b4 3c 35 03 33 61 46 12 08 24 40 e4 95 0c c9 |..<5.3aF..$@....|
comment:2 by , 4 years ago
I guess i have the same problem. My laptop is Windows 10 64 bits (Host). And can't use any guest VM like Ubuntu 18.04 or MX Linux or Debian, because I get errors like write errors. And when I download files from internet on the host VM , using wget, I get corrupted files (md5 hashfile doesn't equals).
For example, I can't install Anaconda package.
Do I have to go back to a previous version?
Thanks.
Any info I may give, please let me know
Sergio
comment:3 by , 4 years ago
wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
This is the error: Do you accept the license terms? [yes|no] [no] >>> yes
Anaconda3 will now be installed into this location: /home/sergio/anaconda3
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below
home/sergio/anaconda3 >>> PREFIX=/home/sergio/anaconda3 WARNING: md5sum mismatch of tar archive expected: 826b70d8ad34b7d9230d4da350d1b598
got: 1f4fb0b8b191ca3213a99d6e0da5d866 -
Unpacking payload ... [1083] Error -3 from inflate: invalid block type [1083] Error decompressing Crypto/PublicKey/_ec_ws.cpython-37m-x86_64-linux-gnu.so Failed to write all bytes for Crypto/PublicKey/_ec_ws.cpython-37m-x86_64-linux-gnu.so fwrite: Bad address [1087] Error -3 from inflate: invalid block type [1087] Error decompressing Crypto/PublicKey/_ec_ws.cpython-37m-x86_64-linux-gnu.so Failed to write all bytes for Crypto/PublicKey/_ec_ws.cpython-37m-x86_64-linux-gnu.so fwrite: Bad address
And this is the MD5 Sum that doesn't match: md5sum Anaconda3-2020.02-Linux-x86_64.sh de821f42cded310a101184ed305196d0 Anaconda3-2020.02-Linux-x86_64.sh
Web: 17600d1f12b2b047b62763221f29f2bc
follow-up: 5 comment:4 by , 4 years ago
I had the same issue with my Linux guest VMs (Xubuntu and Arch) on a Windows 10 host. Anything downloaded from the internet had a hash sum mismatch; the byte size of files was correct but the content was corrupted.
It started after having installed Docker Desktop for Windows.
The issue disappeared after uninstalling Docker and disabling Hyper-V and Containers features (Apps and Features > Programs and Features > Turn Windows features on or off).
At first I only disabled Hyper-V but despite a reboot and the check box being clear it was still active (can be checked by running systeminfo.exe in command prompt). The additional disabling of Containers features and another reboot finally did the trick in my case. My Linux VMs started working correctly after that. I hope this helps.
comment:5 by , 4 years ago
Replying to zelky:
I had the same issue with my Linux guest VMs (Xubuntu and Arch) on a Windows 10 host. Anything downloaded from the internet had a hash sum mismatch; the byte size of files was correct but the content was corrupted.
It started after having installed Docker Desktop for Windows.
The issue disappeared after uninstalling Docker and disabling Hyper-V and Containers features (Apps and Features > Programs and Features > Turn Windows features on or off).
At first I only disabled Hyper-V but despite a reboot and the check box being clear it was still active (can be checked by running systeminfo.exe in command prompt). The additional disabling of Containers features and another reboot finally did the trick in my case. My Linux VMs started working correctly after that. I hope this helps.
Using Windows Containers and Services for Linux features automatically enables HyperV.
comment:6 by , 4 years ago
I can confirm this issue of VB 6.1.12-r139181 in Windows10 (=host) with Ubuntu 20.04 and CentOS 7.8 (=guest);
two more notes to add:
(1) this issue occurs only for "some" source URLs, for example download of CentOS ISO image from CERN succeeded (e.g. sha256 is correct), stuff from Github often fails as described above
(2) the binary differences are in blocks, 16B located randomly several times anywhere in the file
comment:7 by , 4 years ago
I have ran across this exact problem. My research has narrowed it down to having WSL2 enabled. Disabling it, by disabling the "Virtual Machine Platform" in "windows features" and restarting will fix the problem. The download is corrupt over HTTPS even if downloading from a machine on my local network. I've tried different VM Guests (redhat and opensuse linux) differnet vNics, and swapped between NAT and Bridged, wired and wireless.
To me, it seems 100% to be the way virtualbox runs when "Virtual machine Platform" is enabled. (which is required for WSL2)
It is happening in 6.1.12 also
comment:8 by , 4 years ago
I can also confirm this issue with VB 6.1.12-r139181 on Windows 10 host machine with WSL2 enabled and running Debian Buster guests.
In WSL2:
$ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; doneb1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
In Debian Buster guest:
$ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
79c38fb1510cf295be19553f7fe24b145c96cb3d87eb25108fadf0f9021d0fea -
02a0f9877ca83df516b1a36ffb5bef8bbc73dcda0b213df9a006e55ea2be5eca -
e87af43aa9ca42ae64effc13d2d9e3543b24e38ead81ee2bc7a3ed79fd545522 -
17a37b6deb6f26897b78e5d11a4db13b279811f33d7d5131ba608be7ed9e6ad8 -
8ab8a590378dd06ada199ad6ed7cfde518f9ff372f7c9f42c936ac97bcd10305 -
65e40ccb30f3d2cf580e75320c3df118f2c76e3c9525048e47e43d40b1d5aa62 -
aee827047fb122bcb55328a44c8044d1d91f160fc310d725c913288648008d7c -
1c1d78f8d4fec196c86afdec24b41618bbf93cbfba99f2fd7cd37766813e35d3 -
a63a7381a8beccc425b8a92d78f78e5cfd30fa3f4a1e237489f96d075e0dec77 -
40e70f1d38bd822ceb100cd2a4b9effe48d3f4c8187fb81a6762f02398786728 -
comment:9 by , 4 years ago
Thank you, Simon_c, for that insight. I had the same problem but only tested with an Ubuntu 18.04 guest on VB 6.1.12r139181 on Windows 10 Pro version 2004, latest (August 2020) Windows updates installed. I, too, saw the problem with both NAT and Bridged options for the guest NIC. The problem disappeared after disabling the Virtual Machine Platform feature for Windows (and downgrading my WSL "apps" to use WSL 1) and rebooting Windows.
comment:10 by , 4 years ago
Hi there,
just as a quick update. We are able to reproduce something very similar (disk reads return corrupted data) and are investigating it further, can't say when a fix will be available though.
follow-up: 12 comment:11 by , 4 years ago
Would be great if anyone could try out the latest 6.1 testbuild from here and report back if the issue is gone. The testbuild contains a possible workaround for an observed memory corruption. The cause is still unknown but the workaround got rid of the locally observed issues and I would like to know whether it works elsewhere to confirm that the reported issues have the same root cause.
comment:12 by , 4 years ago
Unfortunately not yet solved
- VB 6.1.13 r140124
- Windows 10 Home Version 2004, Latest Updates August 2020
- WSL 2 active
Debian Buster guest:
$ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
2e074cd45d5b533619273357b432f3b632809e8cb2bd1fc95505315ac48e0606 -
3e936ffd76ec7d0cc4f578b46f84379fd925f465edbd21a33927cf810ff74a94 -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
3217e2465bcd3dd7b2127e3f219938ee7eddbdd0c8fd79760e576aacdbfa87ad -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
1d59964c5639902b3665cc2bf0cf137d63398377c8e90869af5c92f163377535 -
1b994d2bb27e65eb61e119f92b42d64ace0ed167bcbbb7b8563b446dd3d31a35 -
d9b1a4065935257100306618db83397f22fcfeb03e8e87cdd207a9fe2de5f8bc -
Ubuntu Xenial guest:
$ for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/download/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
f6b9301e561b4e77ecb8c3dd31f878d2f8934f676d1e7a923c3868cdc0da7e7e -
cd70d1ee2d9af7322621b4ec616aa44df839de3c35080051e718cef813f53fdb -
bbe325cd25bb6d8f20fa8397a54d8fc76bd90d271828f4f4fe6163ab5780f82e -
a983100048284f16202eb32655c48560e11902a2cd815844b091a555bf24dedb -
869bdb246ea3de6f78bfa833b2da2069d92148bdbcde924fa6543c00e04119b5 -
e4d5453b4b5beb8ac3b6f71ddbc74651f203f0e007f2b8618a4d90d3c673c350 -
baa3f1349d21f87b3c19e599ae4f9a9c99f719c14468f040c493c158da733c16 -
2fe89148ef2a29376b914653be07c3783750bc23c02d9d61e1859ceb72c34b37 -
a7152674c5636db150350dbe221b3d292163eaf896635afbf44cddc816b7d612 -
cdf5d058db6e357e43c8955edf65ac1f9f568fb364ea717c1462dbf00a7e0c8c -
4f4ba95b02c1713a72a55950683dea8736490225cee9a12320a04c4afc01897e -
comment:13 by , 4 years ago
Having the Windows 10 Feature "Windows Sandbox" installed will trigger this as well. My understanding is that it uses hyperv.
You don't need to be running it at all, just installed.
Paravirtualization is set to none.
Removing Windows Sandbox and rebooting fixed it.
comment:14 by , 4 years ago
I think this is a massive bug, that prevents users to use Windows as host with virtualbox. Linux-guests are not updateable ! There is no problem with linux as host.
comment:15 by , 4 years ago
I'm also encountering this bug for a few months with all 6.1.x releases. Package installations and downloads in the guest OS are regularly corrupted, sometimes I'm even getting a Guru Meditation. Wanted to use VirtualBox next to Hyper-V, but this bug makes it impossible. Just tested the latest 6.1.13 r140124 build and it still isn't fixed.
Instead of uninstalling Hyper-V and friends, I'm simply disabling the hypervisor through:
bcdedit /set hypervisorlaunchtype off
This makes VirtualBox work again, at the expense of Hyper-V...
comment:16 by , 4 years ago
comment:17 by , 4 years ago
I have the same problem with Linux and FreeBSD guests when installing packages with dnf/pkg. Hyper-V, Containers, Windows Sandbox are disabled, but I do use WSL2.
follow-ups: 19 20 comment:18 by , 4 years ago
We identified the issue and fixed it, unfortunately not in time for 6.1.14. There is a new testbuild available on the Testbuilds page, make sure you use a build >= r140270 . Any feedback is greatly appreciated!
comment:19 by , 4 years ago
Replying to aeichner:
We identified the issue and fixed it, unfortunately not in time for 6.1.14. There is a new testbuild available on the Testbuilds page, make sure you use a build >= r140270 . Any feedback is greatly appreciated!
Tried that build with Ubuntu 20.04... got plenty of CRC errors running an apt update.
I'm afraid it doesn't work.
follow-up: 22 comment:20 by , 4 years ago
Replying to aeichner:
We identified the issue and fixed it, unfortunately not in time for 6.1.14. There is a new testbuild available on the Testbuilds page, make sure you use a build >= r140270 . Any feedback is greatly appreciated!
For me, the problem (curl downloading slightly corrupted files over HTTPS) persists even with new test build r140270. The host is Windows 10.0.19041 (20H1) with Hyper-V and WSL2 enabled and the guest is Ubuntu Bionic.
I noticed that the problem does not occur when I use the environment variable OPENSSL_ia32cap="~0x20000000000" (disabling SSSE3); however, this fix only affects OpenSSL.
comment:21 by , 4 years ago
I have right now tried as well from my side and unfortunately CRC errors are coming up on my side with build r140270 as well.
follow-up: 23 comment:22 by , 4 years ago
Replying to ttalvitie:
[...]
I noticed that the problem does not occur when I use the environment variable
OPENSSL_ia32cap="~0x20000000000"(disabling SSSE3); however, this fix only affects OpenSSL.
Thanks so much for for this hint, this made investigating the possible location of the other issue so much easier. What we fixed in the first iteration was something different which could show the same symptoms but wasn't the cause of the issue people were seeing in this ticket, a different testcase was used so we didn't notice this one. Should be finally fixed now (the testcase of the original reporter of this ticket succeeds now), new testbuild is up on Testbuilds as usual, >= r140284. Thanks again!
comment:23 by , 4 years ago
Replying to aeichner:
[...]
Thanks so much for for this hint, this made investigating the possible location of the other issue so much easier. What we fixed in the first iteration was something different which could show the same symptoms but wasn't the cause of the issue people were seeing in this ticket, a different testcase was used so we didn't notice this one. Should be finally fixed now (the testcase of the original reporter of this ticket succeeds now), new testbuild is up on Testbuilds as usual, >= r140284. Thanks again!
I can confirm that the testbuild r140284 fixes the issue in my case as well. I'm glad I could help. Thank you very much for the fix!
comment:25 by , 4 years ago
I can confirm it's working for Ubuntu 20.04 too. But 3D does not work with VMSVGA with Windows as Host. It works, if Linux as Host. Should I start a separate Thread for this ?
comment:26 by , 4 years ago
Does it work on Windows and without Hyper-V as well? Anyway, should be handled in a new ticket but please look for already existing tickets which describe the same issue before opening a new one.
comment:27 by , 4 years ago
I can confirm now from my side it's finally working. Thank you, this is huge.
for i in {0..10}; do curl -sL https://github.com/docker/compose/releases/dow
nload/1.26.1/docker-compose-Linux-x86_64 | sha256sum -; done
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
b1c7e9b835dd6a4e6436772fe984bf52c93c13149a6b6d06616c7eda1f66123b -
comment:29 by , 4 years ago
Ubuntu 20.04 now installs with test build r140284, but if 3D acceleration is enabled, GUI does not start. I logged a separate ticket for that: https://www.virtualbox.org/ticket/19884
comment:30 by , 4 years ago
Is the fix in r140284 included in 6.1.14? If not, is it included in the following nightly builds? Since that's kind of old.
comment:31 by , 4 years ago
The fix is present in all recent 6.1 testbuilds as well as development snapshots from Testbuilds. It is not available in 6.1.14 but will be in the next maintenance release.
comment:32 by , 4 years ago
I installed the testbuild r140899. However it deleted my HostOnly interfaces and now I cannot create a new one. The error:
Assertion failed: [!aInterfaceName.isEmpty()] at 'F:\tinderbox\win-6.1\src\VBox\Main\src-server\HostNetworkInterfaceImpl.cpp' (76) in long cdecl HostNetworkInterface::init(class com::Utf8Str,class com::Utf8Str,class com::Guid,enum MIDL_MIDL_itf_VirtualBox_0000_0000_0046). Please contact the product vendor!.
Result Code: E_FAIL (0x80004005) Component: HostNetworkInterfaceWrap Interface: IHostNetworkInterface {455f8c45-44a0-a470-ba20-27890b96dba9}
Also none of my VMs will start.
the debian box logs