Connection reset by peer while copying files from Ubuntu VM

[mmokrzycki]

I have some problems with copying files through scp using port forwarding from Ubuntu VM to Windows host. ssh connection to VM, using port forwarding in VirtualBox (localhost:3333 -> VM:22) works stable, but when i want to copy any file, it stops after ~300kB with error "Connection reset by peer". There is nothing besides that error in log files and when i'm using Ubuntu as a host, everything works fine and stable. Network is made under natnetwork and when i change it to hostonly network (without port forwarding) everything works fine. Right now i'm clueless about the core of this problem. I've been using different programs for connections (from winscp to filezilla), with compression enabled/disabled, different SSH version protocols and other different options for connections, as well as different drivers for VM NIC. Here is a script that i'm using to import .ova file and configure machines (i'm using newest vbox on Win7 and Ubuntu 14.04 in VM):

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm "vm-1" poweroff
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm "vm-2" poweroff
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm "vm-3" poweroff
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm "vm-4" poweroff
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" unregistervm "vm-1" --delete
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" unregistervm "vm-2" --delete
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" unregistervm "vm-3" --delete
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" unregistervm "vm-4" --delete
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p delete vm-2
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p delete vm-3
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p delete vm-1
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p delete vm-4
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork remove -t inet-nat-vm-net

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork add -t inet-nat-vm-net -n "192.168.15.0/24" -e -h off

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" import net-cluster.ova --options keepallmacs

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-2 --nictype1 virtio
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-4 --nictype1 virtio
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nictype1 virtio
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-3 --nictype1 virtio

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-2 --nic1 natnetwork
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-4 --nic1 natnetwork
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nic1 natnetwork
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-3 --nic1 natnetwork
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-2 --nat-network1 inet-nat-vm-net
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-4 --nat-network1 inet-nat-vm-net
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-3 --nat-network1 inet-nat-vm-net
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nat-network1 inet-nat-vm-net
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-2:tcp:[]:3331:[192.168.15.4]:22"
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-3:tcp:[]:3332:[192.168.15.5]:22"
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-31:tcp:[]:3335:[192.168.15.5]:8888"
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-1:tcp:[]:3333:[192.168.15.6]:22"
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-4:tcp:[]:3334:[192.168.15.7]:22"

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" startvm "vm-1" --type headless
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" startvm "vm-2" --type headless
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" startvm "vm-3" --type headless
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" startvm "vm-4" --type headless

Environment can be simply reproduced, by installing latest Ubuntu server (x64) as VM named "vm-1" with static ip 192.168.15.6/24 and exporting it to vm1.ova. Then it can be imported and reconfigured by this .bat script:

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm "vm-1" poweroff
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" unregistervm "vm-1" --delete
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p delete vm-1
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork remove -t inet-nat-vm-net

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork add -t inet-nat-vm-net -n "192.168.15.0/24" -e -h off

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" import vm1.ova --options keepallmacs

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nictype1 virtio

"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nic1 natnetwork
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm vm-1 --nat-network1 inet-nat-vm-net
"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" natnetwork modify -t inet-nat-vm-net -p "vm-1:tcp:[]:3333:[192.168.15.6]:22"

Environment:

1. Host OS: Windows 7 Enterprise SP1
2. Guest OS: Ubuntu 14.04 server x64 (clean official install with static ip)

Tools:

1. ssh client (PuTTY)
2. scp client (WinSCP)
3. VirtualBox version 4.3.12 r93733

Connection from host to guest: host: localhost port: 3333

Test steps:

1. Login to guest with ssh
2. List home directory

SSH is working.

3. Login to guest using WinSCP
4. Try to copy >10MB file from guest home directory to host disk

File transfer fails.

Here are some ssh logs, that were collected during steps 3. & 4.

Jul  3 12:57:08 ubuntu sshd[2505]: Accepted password for vm1 from 192.168.15.2 port 62666 ssh2
Jul  3 12:57:08 ubuntu sshd[2505]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:09 ubuntu sshd[2553]: fatal: Write failed: Connection reset by peer
Jul  3 12:57:09 ubuntu sshd[2505]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:10 ubuntu sshd[2555]: Accepted password for vm1 from 192.168.15.2 port 62668 ssh2
Jul  3 12:57:10 ubuntu sshd[2555]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:11 ubuntu sshd[2555]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:12 ubuntu sshd[2605]: Accepted password for vm1 from 192.168.15.2 port 62670 ssh2
Jul  3 12:57:12 ubuntu sshd[2605]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:13 ubuntu sshd[2605]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:14 ubuntu sshd[2655]: Accepted password for vm1 from 192.168.15.2 port 62672 ssh2
Jul  3 12:57:14 ubuntu sshd[2655]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:15 ubuntu sshd[2655]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:17 ubuntu sshd[2705]: Accepted password for vm1 from 192.168.15.2 port 62674 ssh2
Jul  3 12:57:17 ubuntu sshd[2705]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:17 ubuntu sshd[2705]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:19 ubuntu sshd[2755]: Accepted password for vm1 from 192.168.15.2 port 62676 ssh2
Jul  3 12:57:19 ubuntu sshd[2755]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:19 ubuntu sshd[2755]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:21 ubuntu sshd[2805]: Accepted password for vm1 from 192.168.15.2 port 62678 ssh2
Jul  3 12:57:21 ubuntu sshd[2805]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:57:21 ubuntu sshd[2805]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:57:43 ubuntu sshd[2855]: Did not receive identification string from 192.168.15.2
Jul  3 12:57:57 ubuntu sshd[2856]: Accepted password for vm1 from 192.168.15.2 port 62683 ssh2
Jul  3 12:57:57 ubuntu sshd[2856]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:58:05 ubuntu sshd[2906]: Accepted password for vm1 from 192.168.15.2 port 62685 ssh2
Jul  3 12:58:05 ubuntu sshd[2906]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:58:05 ubuntu sshd[2906]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:58:07 ubuntu sshd[2956]: Accepted password for vm1 from 192.168.15.2 port 62687 ssh2
Jul  3 12:58:07 ubuntu sshd[2956]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 12:58:34 ubuntu sshd[2956]: pam_unix(sshd:session): session closed for user vm1
Jul  3 12:58:34 ubuntu sshd[2856]: pam_unix(sshd:session): session closed for user vm1
Jul  3 16:25:42 ubuntu sshd[860]: Server listening on 0.0.0.0 port 22.
Jul  3 16:25:42 ubuntu sshd[860]: Server listening on :: port 22.
Jul  3 14:26:18 ubuntu sshd[1067]: Accepted password for vm1 from 192.168.15.2 port 62905 ssh2
Jul  3 14:26:18 ubuntu sshd[1067]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:26:44 ubuntu sshd[1131]: Accepted password for vm1 from 192.168.15.2 port 62908 ssh2
Jul  3 14:26:44 ubuntu sshd[1131]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:26:53 ubuntu sshd[1179]: fatal: Write failed: Connection reset by peer
Jul  3 14:26:53 ubuntu sshd[1131]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:26:55 ubuntu sshd[1181]: Accepted password for vm1 from 192.168.15.2 port 62911 ssh2
Jul  3 14:26:55 ubuntu sshd[1181]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:26:55 ubuntu sshd[1181]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:26:57 ubuntu sshd[1231]: Accepted password for vm1 from 192.168.15.2 port 62913 ssh2
Jul  3 14:26:57 ubuntu sshd[1231]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:26:57 ubuntu sshd[1231]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:26:59 ubuntu sshd[1281]: Accepted password for vm1 from 192.168.15.2 port 62915 ssh2
Jul  3 14:26:59 ubuntu sshd[1281]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:26:59 ubuntu sshd[1281]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:27:01 ubuntu sshd[1331]: Accepted password for vm1 from 192.168.15.2 port 62917 ssh2
Jul  3 14:27:01 ubuntu sshd[1331]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:27:01 ubuntu sshd[1331]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:27:03 ubuntu sshd[1381]: Accepted password for vm1 from 192.168.15.2 port 62919 ssh2
Jul  3 14:27:03 ubuntu sshd[1381]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:27:03 ubuntu sshd[1381]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:27:05 ubuntu sshd[1431]: Accepted password for vm1 from 192.168.15.2 port 62921 ssh2
Jul  3 14:27:05 ubuntu sshd[1431]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:27:05 ubuntu sshd[1431]: pam_unix(sshd:session): session closed for user vm1
Jul  3 14:27:07 ubuntu sshd[1481]: Accepted password for vm1 from 192.168.15.2 port 62923 ssh2
Jul  3 14:27:07 ubuntu sshd[1481]: pam_unix(sshd:session): session opened for user vm1 by (uid=0)
Jul  3 14:27:07 ubuntu sshd[1481]: pam_unix(sshd:session): session closed for user vm1

Here is a topic at vbox forum: ​https://forums.virtualbox.org/viewtopic.php?f=6&t=62391&p=291809#

mmokrzycki

[Frank Mehnert]

When you report such bugs then it's quite useless to post scripts for setting up your complete test environment. Instead you should try to find a small test case. You write that this happens if you use NAT together with port forwarding. Are you really copying the big file from the guest to the host and if so, how comes port forwarding into play?

[mmokrzycki]

Thank You for quick reply and tips.

The whole cluster consists of 4 VMs that needs to have fixed IP, so i'm using natnetwork to keep communication between them inside NAT. OVA file is distributed to some servers and laptops along with import script to make an automated testing cluster and i need port forwarding to get to those VMs to collect big log files after performing tests inside VMs.

When i want to copy some small files (less than 300kB) than there is no problem in communication. The problem occurs while copying bigger files from VM to host and when i keep resuming transfer after each failure, than the whole file can be copied to host with those small chunks of transfer.

[mmokrzycki]

I have found some defects, that are probably correlated to this problem: https://www.virtualbox.org/ticket/13156 https://www.virtualbox.org/ticket/12136 And as i can see, i have to wait for release 4.3.14.

Thank You for Your time, mmokrzycki

[Valery Ushakov]

Duplicate of 13156.