VirtualBox

Opened 16 years ago

Closed 5 years ago

Last modified 5 years ago

#819 closed defect (fixed)

vboxvfs seems to have problems with mmapped access to files => retry with 6.0.6

Reported by: Jan Owned by:
Component: shared folders Version: VirtualBox 3.0.10
Keywords: vboxsf vboxvfs shared folder Cc:
Guest type: Linux Host type: other

Description (last modified by Frank Mehnert)

I've created a VM with a clone of my webserver's configuration to test my projects under "real" conditions. The Apache webroot is a shared folder mountpoint which is connected to the folder in the host system where I store my sources for the web development.

In the guest VM, I have no problems to access the documents with cat, but the Apache doesn't deliver the files if they're larger than 255 bytes. If I reconfigure Apache to use a handler (e.g. mod_php) to process the requested file before delivering, everything works as desired.

Up to my knowledge, Apache uses mmap() for fast access to static content. Probably this mechanism needs a special support of the underlying file system which vboxvfs doesn't provide.

Host:

  • Gentoo Linux (Sabayon) amd64
  • Kernel 2.6.23-sabayon
  • CPU: Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz

Guest:

  • uses VT
  • Gentoo Linux x86
  • Kernel 2.6.22-gentoo-r8
  • Apache 2.2

Attachments (1)

VBox-2009-08-07-09-49-02.log (40.5 KB ) - added by Martin Burger 15 years ago.

Download all attachments as: .zip

Change History (84)

comment:1 by Frank Mehnert, 16 years ago

Owner: set to Frank Mehnert
Status: newassigned
Summary: vboxvfs seems to have problems with mmapped access to filesvboxvfs seems to have problems with mmapped access to files => Fixed in SVN.

Actually this problem resulted from a missing implementation of the Linux sendfile system call in our VFS module. We fixed this in SVN and will backport this fix to 1.5.4 if no regressions are observed. To fix your guest, reinstall the guest additions containing this fix (SVN for now).

comment:2 by Frank Mehnert, 16 years ago

Resolution: fixed
Status: assignedclosed

Fixed in 1.5.4. Make sure to reinstall the guest additions.

comment:3 by Martin Thielecke, 15 years ago

Resolution: fixed
Status: closedreopened

This bug still exists in 2.1.0 (Host: Ubuntu Intrepid) with a fresh installed Debian 4.0 as guest.

The kernel on the guest is 2.6.18-6-686 (Debian 4.0 standard) and Apache is 2.2.3-4+etch6. I'm using a workaround for apache now, but I think, this bug should be fixed :)

Set following option as workaround for Apache:

EnableMMAP Off

comment:4 by Martin Thielecke, 15 years ago

Forgot the second option for Apache, so please use following workaround: EnableSendfile Off EnableMMAP Off

comment:5 by Frank Mehnert, 15 years ago

Cannot reproduce this problem here with an Etch guest but a home-brewn 2.6.18 kernel. I can try to reproduce this later using the 2.6.18-Etch kernel but are you sure that you actually installed the latest additions?

comment:6 by Frank Mehnert, 15 years ago

Component: othershared folders
Guest type: otherLinux

comment:7 by Martin Thielecke, 15 years ago

The additions installer says 2.1.0.

I've installed a Debian Etch (not etchnhalf!) with a netboot installer (6 MB image) without any additional packages (no GUI, no standard system, only the pure Debian system to keep it very small).

I changed the /etc/apt/sources.list to:

===

deb http://ftp.de.debian.org/debian/ etch main

deb http://security.debian.org/ etch/updates main contrib

deb http://www.backports.org/debian etch-backports main contrib non-free

===

And I executed:

aptitude -y update

aptitude -y dist-upgrade

aptitude install -y make linux-kernel-headers gcc linux-headers-uname -r

After that I've installed the Additions 2.1.0 and have the problems.

comment:8 by Frank Mehnert, 15 years ago

I assume apache2 as well?

comment:9 by Martin Thielecke, 15 years ago

Apache is 2.2.3-4+etch6

comment:10 by Frank Mehnert, 15 years ago

Installed such an Etch system and I'm still not able to reproduce the problem (although using a Debian/Sid host). Is your host x86 or AMD64?

comment:11 by Martin Thielecke, 15 years ago

My host and guest both are x86... Did you try a Netboot-Image?

comment:12 by Yann, 15 years ago

please see also #2921

comment:13 by Martin Burger, 15 years ago

I have the same issue on my virtual machine with Etch installed:

Linux etch08 2.6.24-etchnhalf.1-686 #1 SMP Fri Dec 26 04:10:16 UTC 2008 i686 GNU/Linux

Apache:

Apache/2.2.3 (Debian) PHP/4.4.4-8+etch6 configured

For me, EnableSendfile Off is sufficient as a workaround, I don't have to set EnableMMAP to Off.

comment:14 by MrX1980, 15 years ago

Please try it with v3.0.0 again.

comment:15 by Martin Burger, 15 years ago

If I comment EnableSendfile Off out, the failure occurs.

VirtualBox: 3.0.2 r49928

Can I provide more information that could be helpful?

comment:16 by Frank Mehnert, 15 years ago

mburger, please could you attach a VBox.log file of such a VM session?

by Martin Burger, 15 years ago

comment:17 by Martin Burger, 15 years ago

I could reproduce the failure in VBox 3.0.4 r50677 and attached the requested log file.

comment:18 by Frank Mehnert, 14 years ago

Resolution: fixed
Status: reopenedclosed

Please check VBox 3.0.10. Don't forget to update the guest additions. Reopen this ticket if necessary.

comment:19 by Martin Burger, 14 years ago

Resolution: fixed
Status: closedreopened

I still have the same issue. Although I updated the guest additions. How can I double-check the version?

comment:20 by Frank Mehnert, 14 years ago

In the guest, please execute

dmesg|grep vbox

and post the output here.

comment:21 by Martin Burger, 14 years ago

# dmesg | grep vbox
vboxadd: Successfully loaded version 3.0.10 (interface 0x00010004)
vboxvfs: Successfully loaded version 3.0.10 (interface 0x00010004)
# uname -a
Linux xxx 2.6.24-etchnhalf.1-686 #1 SMP Sat Aug 15 16:51:49 UTC 2009 i686 GNU/Linux

comment:22 by Frank Mehnert, 14 years ago

Version: VirtualBox 1.5.0VirtualBox 3.0.10

So that's still a problem. What is your test case, apache?

in reply to:  22 comment:23 by Martin Burger, 14 years ago

Replying to frank:

So that's still a problem. What is your test case, apache?

Yes, Apache serves a static CSS file from a vboxsf.

comment:24 by Supermighty, 14 years ago

I believe that I am suffering from this same issues.

I setup a Debian (Lenny 5.0) guest OS in virtualbox 3.0.12 on a Windows 7 pro host with ntfs (4k blocks). I setup a shared folder on the Win7 host that is mounted in the Debian guest. I have the apache web server setup on the guest and it points it's root folder to the mounted file share.

This is a great setup. I use host-only networking and I develop my php web app on the Win7 host and test using a web browser pointed at the Debian guest.

All was going well until I noticed that my css files were not loading the most up to date version. What I discovered was that apache served up corrupted css files when the css file was edited in the Win7 host OS.

When I edited the css file in the Win7 host OS apache served up files that had the black diamonds with question makes in them at the end of the file. I edited the css files with Eclipse, GVIM and notepad. The other thing was that the file served up did not have the most recent changes. If I used the host OS to edit the file and add a /* comment */ to the beginning or end of the file it did not show up. It doesn't matter where in the file the text is edited, when editing with host editors the black diamonds show up at the bottom of the file.

The problem corrected itself once I edited the css file in the Debian guest OS using VIM. I did a couple more test to see exactly what was going on. I copied the contents of a css file to 'test.html' so it was a css file but with an .html file extension and there was no issues. I could edit the test file in the host OS and have it served up by apache in the guest OS without any corruption. However when I changed the extension back to .css the problem resumed. I tested the other way too, creating an html file with .css as an extension and the corruption occurred, but not when the html file extension was reverted to .html.

I tried this with a couple different css files that I had and it seems that the problem only happens when the file size is more than ~2k. I could not replicate the problem on files that were ~120b but as soon as I added a bunch of css to the file to make it grow to more than ~4k the problem cropped up.

I have booted the Win7 host OS in restore mode and ran chkdsk while c:\ wasn't mounted and there were no returned errors. I have tried changing the css encoding from utf-8 to latin1.

To me it seems that css files are being corrupted when edited in the Win7 host OS if the css file is larger than ~2k on disk.

comment:25 by Supermighty, 14 years ago

Update:

The guest OS is loaded and apache is running. I edit one of the css files in the host OS and save it. When I reload the css file in firefox in the host OS there are black diamonds with question marks at the bottom of the file.

I stop apache, unmount the vboxsf share, remount the share, and start apache. When I reload (f5) the file in Firefox the file is correct and as it should be. This leads me to believe that there is a problem in the way that vboxsf looks at how some files are changed or updated.

What can I do for the devs to help them track down this problem?

comment:26 by Supermighty, 14 years ago

The solution that worked for me was suggested by a forum user (MarkCranness) was to turn off EnableSendfile in Apache. This changes the way that Apache reads a file before it serves it up when requested.

comment:27 by Frank Mehnert, 14 years ago

Summary: vboxvfs seems to have problems with mmapped access to files => Fixed in SVN.vboxvfs seems to have problems with mmapped access to files

Yes, this workaround is mentioned here in this ticket as well. We will try to fix this problem but not for 3.1.0 yet.

comment:28 by Jesse Glick, 14 years ago

I seem to be having a similar problem with OpenSolaris 2009.06 guest in Ubuntu 9.10 host on VBox 3.0.12 using a freshly installed guest additions package. I mount a shared folder contain an Apache Ant 1.7.1 installation. Trying to run Ant fails:

$ /space/ant171/bin/ant -version mmap failed for CEN and END part of zip file Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/Launcher

even though I can access its files otherwise:

$ jar tvf /space/ant171/lib/ant.jar | head -5

0 Fri Jun 27 05:03:52 EDT 2008 META-INF/

449 Fri Jun 27 05:03:50 EDT 2008 META-INF/MANIFEST.MF

0 Fri Jun 27 05:03:44 EDT 2008 org/ 0 Fri Jun 27 05:03:44 EDT 2008 org/apache/ 0 Fri Jun 27 05:03:50 EDT 2008 org/apache/tools/

Generally, running any other Java application based on JARs fails if the JARs are on a shared folder. If I 'cp -av' the app to a local disk it runs fine.

comment:29 by Frank Mehnert, 14 years ago

Summary: vboxvfs seems to have problems with mmapped access to filesvboxvfs seems to have problems with mmapped access to files => retry with 3.1.4

Retry with 3.1.4 when released. Make sure to update the guest additions to 3.1.4 of course.

comment:30 by Frank Mehnert, 14 years ago

Resolution: fixed
Status: reopenedclosed

Reopen if necessary.

comment:31 by Jesse Glick, 14 years ago

Resolution: fixed
Status: closedreopened

Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic running OpenSolaris 9.06 with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26 (rebooted after updating new guest additions pkg).

jglick@jglick-solaris:~$ mount -v | fgrep /space space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010 jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar mmap failed for CEN and END part of zip file Exception in thread "main" java.util.zip.ZipException: error in opening zip file

at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:114) at java.util.jar.JarFile.<init>(JarFile.java:133) at java.util.jar.JarFile.<init>(JarFile.java:70)

jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar . /space/tmp/sayhello/dist/sayhello.jar' -> ./sayhello.jar' jglick@jglick-solaris:~$ java -jar sayhello.jar hello world jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar

0 Tue Feb 23 09:48:08 EST 2010 META-INF/

207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF

0 Tue Feb 23 09:48:08 EST 2010 sayhello/

533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class

jglick@jglick-solaris:~$

comment:32 by Jesse Glick, 14 years ago

This may be easier to read:

Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic running OpenSolaris 9.06 with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26 (rebooted after updating new guest additions pkg).

jglick@jglick-solaris:~$ mount -v | fgrep /space
space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010
jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar
mmap failed for CEN and END part of zip file
Exception in thread "main" java.util.zip.ZipException: error in opening zip file
	at java.util.zip.ZipFile.open(Native Method)
	at java.util.zip.ZipFile.<init>(ZipFile.java:114)
	at java.util.jar.JarFile.<init>(JarFile.java:133)
	at java.util.jar.JarFile.<init>(JarFile.java:70)
jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar .
`/space/tmp/sayhello/dist/sayhello.jar' -> `./sayhello.jar'
jglick@jglick-solaris:~$ java -jar sayhello.jar
hello world
jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar
     0 Tue Feb 23 09:48:08 EST 2010 META-INF/
   207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF
     0 Tue Feb 23 09:48:08 EST 2010 sayhello/
   533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class
jglick@jglick-solaris:~$ 

comment:33 by Frank Mehnert, 14 years ago

Moment, you are talking about a Solaris guest. Any positive feedback for Linux guests?

comment:34 by Eric Herrera, 14 years ago

I have a similar problem, in that apache is delivering what appears to be a previous version of the file however truncated at the length of the current file.

Adding this to apache fixes the problem: EnableMMAP Off EnableSendfile Off

Here's the setup:

Host is: Ubuntu 9.10 Linux 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 02:39:34 UTC 2010 x86_64 GNU/Linux VirtualBox 3.1.4 r57640

Guest is: Debian GNU/Linux 4.0 Linux 2.6.18-6-amd64 #1 SMP Thu Dec 25 21:08:54 UTC 2008 x86_64 GNU/Linux VirtualBox 3.1.4 Guest Additions for Linux MD5="2490b313eb8c0168a7d7648e12f5803d"

DocumentRoot is mounted vboxsf of host

comment:35 by josephusmullally@gmail.com, 14 years ago

I recently came across what seems to be the same issue when trying to use mmap'd files on a vboxsf mounted share, on the (as of now) latest vbox version.

When files are opened read-only and used with mmap() it works fine, but it fails for files opened read-write and later used with mmap for writing.

Here is some code to reproduce:

// mmap_test.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <error.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>

int main(int argc, char *argv[])
{
    char *ro_mem, *rw_mem;      // mmapped IO
    int ro_fd, rw_fd;
    off_t len_ro, len_rw;

    // read test - works OK on vboxsf
    if( (ro_fd = open("ro_file", O_RDONLY)) == -1)
        error(-1, errno, "Error opening 'ro_file'");
    len_ro = lseek(ro_fd, 0L, SEEK_END);
    if((ro_mem = (char *) mmap(NULL, len_ro, PROT_READ, MAP_SHARED, ro_fd, 0)) == MAP_FAILED)
        error(-1, errno, "Error memory-mapping ro_file");
    printf("ro_mem contents:   %s", ro_mem);
    munmap(ro_mem, len_ro); close(ro_fd);

    // write test - fails on vboxsf
    if( (rw_fd = open("rw_file", O_RDWR)) == -1)
        error(-1, errno, "Error opening 'rw_file'");
    len_rw = lseek(rw_fd, 0L, SEEK_END);
    if((rw_mem = (char *) mmap(NULL, len_ro, PROT_READ|PROT_WRITE, MAP_SHARED, rw_fd, 0)) == MAP_FAILED)
        error(-1, errno, "Error memory-mapping rw_file");
    rw_mem[5] = 'X';
    munmap(rw_mem, len_rw); close(rw_fd);

    // Changing the rw case to MAP_PRIVATE will succeed, but changes 
    // aren't propogated to 'rw_file', as expected.
    // Removing PROT_WRITE still fails.
    // --> dies on open(O_RDWR),mmap(MAP_SHARED) pair

    return 0;
}
#!/bin/sh
# test.sh
gcc -o mmap_test mmap_test.c
echo "Sample text for readonly file" > ro_file
echo "Write test text" > rw_file
./mmap_test
cat rw_file

# Output on a regular ext3fs dir:
#    ro_mem contents:   Sample text for readonly file
#    WriteXtest text

# Output on a vboxsf dir:
#    ro_mem contents:   Sample text for readonly file
#    ./mmap_test: Error memory-mapping rw_file: Invalid argument
#    Write test text
versions:
Host:   Windows 7, Virtualbox 3.1.6 r59338

Guest:  Ubuntu 9.10 Karmic - 2.6.31-20-generic-pae
        (Using vbox guest additions from host supplied ISO)
    
        vboxguest: major 0, IRQ 9, I/O port d040, MMIO at 00000000f0400000 (size 0x400000)
        vboxguest: Successfully loaded version 3.1.6 (interface 0x00010004)
        vboxvfs: Successfully loaded version 3.1.6 (interface 0x00010004)

Hope that helps

comment:36 by mesnakh, 14 years ago

Host: Windows 7 64-bit running Virtual Box 3.1.8, r61349 Guest: Up to date Arch Linux: Linux perelman 2.6.33-ARCH #1 SMP PREEMPT Sun May 2 10:40:03 CEST 2010 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux Guest additions for 3.1.8 have been installed and are running.

To reproduce the problem:

  1. Run rtorrent 0.8.6 with a vboxsf mount as its download directory
  2. Try to download any torrent
  3. Observe every download failing with "Storage error: [File chunk write error: Invalid argument.]"

rtorrent uses mmap for this.

comment:37 by Bryan Drewery, 14 years ago

I'm having this exact problem in 3.1.8 (OpenSolaris 134 Host) in a Linux guest. I'm unable to try newer versions at this time. Reading a file with mmap() works fine while writing it does not.

comment:38 by zimbatm, 13 years ago

Same problem with VirtualBox 3.2.12, Host: OSX 10.6.5, Guest: ubuntu-lucid-32

comment:39 by Alexey Fyodorov, 13 years ago

Same problem with VirtualBox 4.0.4, Host: Windows 7 x64, Guest: Oracle Solaris 11 Express 2010.11

comment:40 by Jesse Glick, 13 years ago

No longer having any problems using Ubuntu host and XP guest (w/ 4.0.4 guest additions); ZIP file access working as expected.

comment:41 by carstep, 13 years ago

Same issue with 4.0.8 on W7 ultimate and debian(lenny) guest with apache2 with JavaScript files EnableSendfile Off didn't help only sync; echo 3 > /proc/sys/vm/drop_caches, but running this every time I develop further is inconvenient

  1. Sandor

comment:42 by carstep, 13 years ago

update:

using EnableMMAP Off helped

comment:43 by Frank Mehnert, 13 years ago

carstep, do you read files from the guest or do you write files to the guest?

comment:44 by carstep, 13 years ago

I write files to the quest

comment:45 by David-Sarah Hopwood, 13 years ago

The mmap problem may be the cause of http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.

in reply to:  45 comment:46 by David-Sarah Hopwood, 13 years ago

Replying to davidsarah2:

The mmap problem may be the cause of http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.

Note that this problem occurs still in VirtualBox 3.2.8_OSE r64453. In my case, the host is Ubuntu 10.10 and the shared folder is accessed from a Windows XP SP3 guest, although the original reporter of the darcs bug also had the problem when accessing a shared folder from Linux.

comment:47 by SimonSimCity, 13 years ago

I can confirm this issue - not using Apache but using nginx. I don't know an option of nginx like EnableMMAP what you've written concerning Apache.

I've moved back to mount my shared-folder by samba to work.

To work arround this problem you can clear the pagecache of linux every second - like this:

watch -n 1 `sync; echo 1 > /proc/sys/vm/drop_caches`

Source: http://www.linuxinsight.com/proc_sys_vm_drop_caches.html

comment:48 by SimonSimCity, 13 years ago

Sorry for this double-post .. but I'm not able to edit my last information. Here's another link where they explain a simular problem using more text :) http://serverfault.com/questions/30240/disable-all-disk-caching-for-apache2-on-linux

comment:49 by Jesse Glick, 12 years ago

Using virtualbox-4.1.2-dfsg-1ubuntu1 on Ubuntu Oneiric with an XP guest (4.0.4 guest additions) I continue to have this problem sometimes, but not always; certain JAR files cannot be accessed from the guest, giving a ZIPException. Dropping the page caches from the host seems to make no difference. Using guest additions 4.1.2 it seems to be working better. Perhaps related to ticket #9276, mentioned in the changelog for this release?

comment:50 by laoshanlong, 12 years ago

Also getting this issue with Virtualbox 4.1.6. Using a Windows 7 x64 host and an Ubuntu Server guest. Fresh install from today. Attempting to use rtorrent to download to a vboxsf mount results in Storage error: [File chunk write error: Invalid argument.], an error that occurs due to an issue with mmap.

comment:51 by rhcoe, 12 years ago

I'm experiencing a similar issue on VirtualBox 4.1.6, first experienced on 4.1.4. java apparently uses mmap to read zip'd jar files, and this fails for files larger than 2129615 (0x207ecf) bytes.

The next largest jar is 2213055 (0x21c4bf) and cannot be read.

I'm running a win xp guest on a linux host.

e: is a mounted shared directory from the linux host.

jar tvf e:\some\file.jar
java.util.zip.ZipException: error in opening zip file
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:127)
    at java.util.zip.ZipFile.<init>(ZipFile.java:88)
    at sun.tools.jar.Main.list(Main.java:979)
    at sun.tools.jar.Main.run(Main.java:224)
    at sun.tools.jar.Main.main(Main.java:1149)

comment:52 by LemADEC, 11 years ago

Issue reproduced under Apache when updating .css files.
Disabling caches and MMAP didn't help.
Disabling also Sendfile fixed the issue.

Version used are:
Host = Windows 7 SP1 32 US
Guest = Entreprise Linux 2.6.32-279.11.1.el6.x86_64 + httpd-2.2.15-15.el6
using VirtualBox 4.2.0 r80737 + vboxguest/vboxsf 4.2.0

comment:53 by Bartzy, 11 years ago

Any updates on this issue?

Thanks!

comment:54 by eKIK, 10 years ago

Issue is still occurring with VirtualBox 4.2.16, Ubuntu 13.10 host and Ubuntu 12.04 guest.

A workaround for nginx is to set "sendfile off;" (http://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile).

comment:55 by eugeneware, 10 years ago

This is still an issue in the current VirtualBox version 4.3.6 with Ubuntu 12 and 13.

When using mmap operations on a vboxfs mapped folder it fails.

I was trying to use leveldb and lmdb operations on vboxfs mount, and couldn't open the files. Using an nfs share works however.

Here's some code that replicates the problem:

https://gist.github.com/9045290

If you compile this mmap-test, then running:

$ mmap-test /file/on/vboxfs/partition

fails with an invalid argument error

However, running it with a file on an nfs or ext4 partition works fine.

This is messing up local development workflows for anything that uses mmap (ie. a lot of software).

Please fix!

comment:56 by eugeneware, 10 years ago

Just checking to see if anyone is currently working on this? Frank? Any luck replicating?

comment:57 by eugeneware, 10 years ago

Hi Frank. I've just tested the latest version of VirtualBox (4.3.8) and the problem still remains?

Any update?

comment:58 by Frank Mehnert, 10 years ago

Description: modified (diff)

Sorry, no update. Currently we have other priorities. If mmap-access to shared folders does not work, try to prevent it. For instance, Apache can be forced to not access files this way. If you are capable of Linux programming you might also want to check the source code. Patches help to fix such bugs faster.

comment:60 by azend, 10 years ago

Still a problem in VirtualBox 4.3.12 r93733 for Mac OSX (Tested with Debian Wheezy guest).

comment:61 by Brendan, 9 years ago

I've replicated this in 4.3.10, host and guest both Ubuntu 14.04 x86_64.

mmap()ing a file with PROT_READ|PROT_WRITE fails with EINVAL.

comment:62 by eugeneware, 9 years ago

Hi Frank,

Iskra Vitaly reported a potential diff that may fix the issue above:

https://build.merproject.org/package/view_file?file=vbox-mmap.diff&package=virtualbox&project=nemo%3Adevel%3Ahw%3Ax86%3Ax86-common&rev=54d38848b69eea5527f8e1bbe9aae3b2

Do you think you could spend a few minutes to see if it works?

Here's the code to replicate the issue: https://gist.github.com/9045290

We love your work, and there's a lot of us in the database community who can't get our apps to run because of this bug.

Pretty please? :-)

comment:63 by Frank Mehnert, 9 years ago

Thank you! I will have a look and consider to add the fix to the tree.

comment:64 by cedric, 9 years ago

The issue is still present in 4.3.20 (OSX Yosemite) Observed with mkimage doing an mmap() of the kernel uImage file to create it.

comment:65 by g4b0, 9 years ago

I can confirm that this issue is already present in VirtualBox 4.3.18 [linux]. It was some hours that I was looking around for solve an issue of apache filling my javascript files with null bytes when modified on the host system (I'm developing using Vagrant).

EnableMMAP off in apache configuration solved my issue

g4b0

comment:66 by mikeatlas, 8 years ago

This is still an issue.

VirtualBox version: 5.0.10 r104061
OS X version: 10.11.2 (15C50)

In my personal case, I'm attempting to map local OS X volumes for data storage of Cassandra within a Docker container (for data persistence) and, end up hitting https://issues.apache.org/jira/browse/CASSANDRA-9635 through a shared volume within docker-machine (previously known as boot2docker in that ticket.)

@frank, are you still interested in that patch? It's an 8 year old bug now?

comment:67 by mikeatlas, 8 years ago

I should add that other Dockerized projects like Mongo, in addition to Cassandra, die when attempting to mount (OS X) volumes via vboxvfs due to the lack of mmap support.

https://github.com/docker-library/mongo/issues/30#issuecomment-108619916 https://github.com/docker-library/docs/tree/master/mongo#where-to-store-data

Last edited 8 years ago by mikeatlas (previous) (diff)

in reply to:  63 comment:68 by Matthew L Daniel, 8 years ago

Replying to frank:

Thank you! I will have a look and consider to add the fix to the tree.

That comment was made 15 months ago. Was that patch considered, but rejected, or never considered further?

comment:69 by oleksiitokar, 8 years ago

bump :)

comment:70 by bronek, 8 years ago

Exactly the same problem as described by Supermighty. I've VM with JS file and I can see the recent version via cat within VM, but not when requesting via Apache.

So (in VM):

cat foo.js -> current version
curl "http://localhost/foo.js?nocache" -> old version

I've -HUP killed my apache, fully restarting, and there is no way I could see the current file.

I could see in access.log file being accessed, but it's the old version:

127.0.0.1 - - [09/May/2016:14:39:04 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
::1 - - [09/May/2016:14:39:18 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
::1 - - [09/May/2016:14:39:20 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"

I've checked using strace, open syscall is called on the file, so it looks like even Apache doesn't know the file is cached.

Here is the strace log:

$ sudo strace -fp 4911 -s 500
Process 4911 attached
semop(524292, {{0, -1, SEM_UNDO}}, 1)   = 0
epoll_wait(14, {{EPOLLIN, {u32=4157165232, u64=139831112453808}}}, 2, 10000) = 1
accept4(4, {sa_family=AF_INET6, sin6_port=htons(65349), inet_pton(AF_INET6, "::ffff:192.168.88.1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28], SOCK_CLOEXEC) = 15
semop(524292, {{0, 1, SEM_UNDO}}, 1)    = 0
getsockname(15, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "::ffff:192.168.88.89", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
fcntl(15, F_GETFL)                      = 0x2 (flags O_RDWR)
fcntl(15, F_SETFL, O_RDWR|O_NONBLOCK)   = 0
read(15, "GET /confirmation-page.js?asd HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Lan"..., 8000) = 2254
gettimeofday({1462805490, 328896}, NULL) = 0
stat("/confirmation-page.js", {st_mode=S_IFREG|0666, st_size=382, ...}) = 0
...
open("/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/confirmation-page.js/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOTDIR (Not a directory)
stat("/confirmation-page.js", {st_mode=S_IFREG|0666, st_size=382, ...}) = 0
stat("/confirmation-page.js.gz", 0x7ffcc65ebd90) = -1 ENOENT (No such file or directory)
open("/confirmation-page.js", O_RDONLY|O_CLOEXEC) = 16
fcntl(16, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
fcntl(16, F_SETFD, FD_CLOEXEC)          = 0
brk(0x7f2cf7ce8000)                     = 0x7f2cf7ce8000
setsockopt(15, SOL_TCP, TCP_CORK, [1], 4) = 0
writev(15, [{"HTTP/1.1 200 OK\r\nDate: Mon, 09 May 2016 14:51:30 GMT\r\nServer: Apache/2.2.15 (CentOS)\r\nX-Content-Type-Options: nosniff\r\nLast-Modified: Mon, 09 May 2016 14:25:11 GMT\r\nETag: \"3724-17e-53269937dbfc0\"\r\nAccept-Ranges: bytes\r\nContent-Length: 382\r\nCache-Control: max-age=1209600\r\nExpires: Mon, 23 May 2016 14:51:30 GMT\r\nConnection: close\r\nContent-Type: text/javascript\r\n\r\n", 364}], 1) = 364
sendfile(15, 16, [0], 382)              = 382
setsockopt(15, SOL_TCP, TCP_CORK, [0], 4) = 0
write(10, "192.168.88.1 - - [09/May/2016:14:51:30 +0000] \"GET /confirmation-page.js?asd HTTP/1.1\" 200 382 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36\"\n", 275) = 275
shutdown(15, SHUT_WR)                   = 0

I also see black diamonds at the end of the file like:

������������������������������������������������������������

In curl as well:

$ curl "http://localhost/foo.js | cat -v
(function($) {
// Some content.
})(jQuery);
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@[vagrant@html]$

No way it's some Apache level caching.

$ VBoxManage --version
5.0.14r105127
$ httpd -V
Server version: Apache/2.2.15 (Unix)
Architecture:   64-bit
$ uname -a
Linux localhost 2.6.32-573.18.1.el6.x86_64 #1 SMP Tue Feb 9 22:46:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

My docroot directory is synced in Vagrantfile as:

# Allow override of the default synced folder type.
  config.vm.synced_folder ".", "/vagrant",
    owner: vconfig.include?('vagrant_user') ? vconfig['vagrant_user'] : 'vagrant',
    group: "vagrant",
    type: vconfig.include?('vagrant_synced_folder_default_type') ? vconfig['vagrant_default_synced_folder_type'] : 'nfs',
    mount_options: vconfig.include?('vagrant_synced_folder_mount_options') ? vconfig['vagrant_synced_folder_mount_options'] : ["dmode=777,fmode=666"]

As for workaround, disabling EnableSendfile in Apache config worked great (as suggested by Supermighty)!

EnableSendfile Off

E.g.

<Directory "/path-to-nfs-files">
EnableSendfile Off
</Directory>

This disables using the kernel sendfile support to deliver files to the client.

Related tickets: #9069, #12597, #14920

Last edited 8 years ago by bronek (previous) (diff)

comment:71 by roylee17, 8 years ago

This is still an issue in 5.1.4. Is there any timeline to include fixes in the future release?

comment:72 by jml0, 7 years ago

This also affects cargo (https://github.com/rust-lang/cargo/issues/2808), and anything that uses libgit2 (https://github.com/libgit2/libgit2/issues/3845). I tried to look at the patch mentioned by eugeneware, but all I see is a page with author & commit metadata but no code diff.

comment:73 by jml0, 7 years ago

Ah hah, apparently I needed to resize the text box. Repasted at https://gist.github.com/jml/bfa756a071cfbf8c23033a8748aa4a11 in case that helps people.

comment:74 by vanam, 7 years ago

Any update, please? This is really bugging me... Thank you

My use case is described at https://github.com/nette/caching/issues/37 (Issue with PRAGMA journal_mode = WAL; with SQLite)

  • Virtualbox: 5.1.26r117224
  • Host OS: Ubuntu 16.04
  • Guest OS: Ubuntu 16.04
Last edited 7 years ago by vanam (previous) (diff)

comment:75 by Michael Thayer, 7 years ago

Owner: Frank Mehnert removed
Status: reopenedassigned

comment:76 by Heath, 7 years ago

Info at ticket #17053 regarding the readv system call may apply here.

comment:77 by jstangroome, 6 years ago

As per the #14920 (this issue as seen in a Docker container) I am experiencing this too.

  • Host OS: Windows 10 x64 version 1709 (OS Build 16299.192)
  • VirtualBox 5.2.6 r120293
  • Vagrant 2.0.1
  • Guest OS: Ubuntu 14.04, kernel 3.13.0-93-generic, vboxguest 4.3.36-dfsg-1+deb8u1ubuntu1.14.04.1, docker 17.05.0-ce

Guest OS sees latest edits of file mounted under /vagrant/ via vboxsf. Docker container with read-only volume mount sees (with cat, grep, and python) the older version of the file. Removing the Docker container and creating a new container still sees the older version. Executing mount in the container shows the volume is using vboxsf.

Restarting the VM with vagrant reload allows a new container to be started which sees the latest version of the file but then subsequent edits of the file are ignored again.

comment:78 by hansg, 6 years ago

Hi All,

Someone pointed me to this bug by email, because I've been working on a version of the vboxsf driver for the upstream/mainline Linux kernel.

I've written a couple of patches on top of my driver, which I believe fix this.

My version of the vboxsf driver needs the upstream/mainline Linux kernel version of the vboxguest driver. So to test this you will need a 4.16-rc1 kernel with the vboxguest driver enabled in yuor guest, e.g.: https://koji.fedoraproject.org/koji/buildinfo?buildID=1043729

Then clone this git repo: https://github.com/jwrdegoede/vboxsf/ and do "cd vboxsf", "make" ,"sudo make modules_install" and then reboot.

With my vboxsf driver the problems reported in this bug will hopefully belong to the past.

There really were 3 problems:

1) MAP_SHARED not being supported 2) Changes done through regular read/write on the guest side not being seen by guest apps using mmap() access 3) Any changes done on the host side not being seen by guest apps using mmap() access

The last 3 commits here contain fixes for each of them for those of you who are interested: https://github.com/jwrdegoede/vboxsf/commits/master

Regards,

Hans

comment:79 by bird, 5 years ago

Resolution: fixed
Status: assignedclosed
Summary: vboxvfs seems to have problems with mmapped access to files => retry with 3.1.4vboxvfs seems to have problems with mmapped access to files => retry with 6.0.6

6.0.6 will ship some updates to shared folders, including read-write mmap for all supported linux guest kernels (including 2.4.x). We also consistently avoid any guest side caching for all 2.6.x and later kernels (for 2.4.x guests sendfile will use the cache), with the exception of read-write mmapped files where we by default go thru the cache to catch guest side modifications (dirty pages). The latter behavior can be modified by the new cache=none mount option.

We'll put out test builds later this week that includes the modifications, I hope.

comment:80 by xnoreq, 5 years ago

The latter behavior can be modified by the new cache=none mount option.

Mounting with that option in 6.0.6 fails with:

kernel: vboxsf: Unrecognized mount option "cache=none" or missing value

comment:81 by bird, 5 years ago

re comment:80: You haven't installed the guest additions correctly. There is no message like that in our sources. The mount options gets repackaged in userland by a tool we install to /sbin/mount.vboxsf.

comment:82 by xnoreq, 5 years ago

That was through fstab. Calling mount.vboxsf directly doesn't help:

$ sudo mount.vboxsf --options cache=none test /mnt/test
mount: /mnt/test: wrong fs type, bad option, bad superblock on test, missing codepage or helper program, or other error.
$ mount|grep test
$ sudo mount.vboxsf test /mnt/test
$ mount|grep test
test on /mnt/test type vboxsf (rw,relatime)

comment:83 by oracleaccount34, 5 years ago

VirtualBox 6.0.6 actually broke certain parts of shared folders. I stumbled upon it due to this bug: https://github.com/Ocramius/PackageVersions/issues/107, but after some digging it turns out that downgrading to 6.0.4 fixed the issue (I tried all newer versions up to and including 6.0.10).

Note: See TracTickets for help on using tickets.

© 2023 Oracle
ContactPrivacy policyTerms of Use