#819 closed defect (fixed)
vboxvfs seems to have problems with mmapped access to files => retry with 6.0.6
Reported by: | Jan | Owned by: | |
---|---|---|---|
Component: | shared folders | Version: | VirtualBox 3.0.10 |
Keywords: | vboxsf vboxvfs shared folder | Cc: | |
Guest type: | Linux | Host type: | other |
Description (last modified by )
I've created a VM with a clone of my webserver's configuration to test my projects under "real" conditions. The Apache webroot is a shared folder mountpoint which is connected to the folder in the host system where I store my sources for the web development.
In the guest VM, I have no problems to access the documents with cat, but the Apache doesn't deliver the files if they're larger than 255 bytes. If I reconfigure Apache to use a handler (e.g. mod_php) to process the requested file before delivering, everything works as desired.
Up to my knowledge, Apache uses mmap() for fast access to static content. Probably this mechanism needs a special support of the underlying file system which vboxvfs doesn't provide.
Host:
- Gentoo Linux (Sabayon) amd64
- Kernel 2.6.23-sabayon
- CPU: Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz
Guest:
- uses VT
- Gentoo Linux x86
- Kernel 2.6.22-gentoo-r8
- Apache 2.2
Attachments (1)
Change History (84)
comment:1 by , 17 years ago
Owner: | set to |
---|---|
Status: | new → assigned |
Summary: | vboxvfs seems to have problems with mmapped access to files → vboxvfs seems to have problems with mmapped access to files => Fixed in SVN. |
comment:2 by , 17 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
Fixed in 1.5.4. Make sure to reinstall the guest additions.
comment:3 by , 16 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
This bug still exists in 2.1.0 (Host: Ubuntu Intrepid) with a fresh installed Debian 4.0 as guest.
The kernel on the guest is 2.6.18-6-686 (Debian 4.0 standard) and Apache is 2.2.3-4+etch6. I'm using a workaround for apache now, but I think, this bug should be fixed :)
Set following option as workaround for Apache:
EnableMMAP Off
comment:4 by , 16 years ago
Forgot the second option for Apache, so please use following workaround: EnableSendfile Off EnableMMAP Off
comment:5 by , 16 years ago
Cannot reproduce this problem here with an Etch guest but a home-brewn 2.6.18 kernel. I can try to reproduce this later using the 2.6.18-Etch kernel but are you sure that you actually installed the latest additions?
comment:6 by , 16 years ago
Component: | other → shared folders |
---|---|
Guest type: | other → Linux |
comment:7 by , 16 years ago
The additions installer says 2.1.0.
I've installed a Debian Etch (not etchnhalf!) with a netboot installer (6 MB image) without any additional packages (no GUI, no standard system, only the pure Debian system to keep it very small).
I changed the /etc/apt/sources.list to:
===
deb http://ftp.de.debian.org/debian/ etch main
deb http://security.debian.org/ etch/updates main contrib
deb http://www.backports.org/debian etch-backports main contrib non-free
===
And I executed:
aptitude -y update
aptitude -y dist-upgrade
aptitude install -y make linux-kernel-headers gcc linux-headers-uname -r
After that I've installed the Additions 2.1.0 and have the problems.
comment:10 by , 16 years ago
Installed such an Etch system and I'm still not able to reproduce the problem (although using a Debian/Sid host). Is your host x86 or AMD64?
comment:13 by , 15 years ago
I have the same issue on my virtual machine with Etch installed:
Linux etch08 2.6.24-etchnhalf.1-686 #1 SMP Fri Dec 26 04:10:16 UTC 2008 i686 GNU/Linux
Apache:
Apache/2.2.3 (Debian) PHP/4.4.4-8+etch6 configured
For me, EnableSendfile Off
is sufficient as a workaround, I don't have to set EnableMMAP
to Off
.
comment:15 by , 15 years ago
If I comment EnableSendfile Off
out, the failure occurs.
VirtualBox: 3.0.2 r49928
Can I provide more information that could be helpful?
by , 15 years ago
Attachment: | VBox-2009-08-07-09-49-02.log added |
---|
comment:17 by , 15 years ago
I could reproduce the failure in VBox 3.0.4 r50677 and attached the requested log file.
comment:18 by , 15 years ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
Please check VBox 3.0.10. Don't forget to update the guest additions. Reopen this ticket if necessary.
comment:19 by , 15 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
I still have the same issue. Although I updated the guest additions. How can I double-check the version?
comment:21 by , 15 years ago
# dmesg | grep vbox vboxadd: Successfully loaded version 3.0.10 (interface 0x00010004) vboxvfs: Successfully loaded version 3.0.10 (interface 0x00010004)
# uname -a Linux xxx 2.6.24-etchnhalf.1-686 #1 SMP Sat Aug 15 16:51:49 UTC 2009 i686 GNU/Linux
follow-up: 23 comment:22 by , 15 years ago
Version: | VirtualBox 1.5.0 → VirtualBox 3.0.10 |
---|
So that's still a problem. What is your test case, apache?
comment:23 by , 15 years ago
Replying to frank:
So that's still a problem. What is your test case, apache?
Yes, Apache serves a static CSS file from a vboxsf.
comment:24 by , 15 years ago
I believe that I am suffering from this same issues.
I setup a Debian (Lenny 5.0) guest OS in virtualbox 3.0.12 on a Windows 7 pro host with ntfs (4k blocks). I setup a shared folder on the Win7 host that is mounted in the Debian guest. I have the apache web server setup on the guest and it points it's root folder to the mounted file share.
This is a great setup. I use host-only networking and I develop my php web app on the Win7 host and test using a web browser pointed at the Debian guest.
All was going well until I noticed that my css files were not loading the most up to date version. What I discovered was that apache served up corrupted css files when the css file was edited in the Win7 host OS.
When I edited the css file in the Win7 host OS apache served up files that had the black diamonds with question makes in them at the end of the file. I edited the css files with Eclipse, GVIM and notepad. The other thing was that the file served up did not have the most recent changes. If I used the host OS to edit the file and add a /* comment */ to the beginning or end of the file it did not show up. It doesn't matter where in the file the text is edited, when editing with host editors the black diamonds show up at the bottom of the file.
The problem corrected itself once I edited the css file in the Debian guest OS using VIM. I did a couple more test to see exactly what was going on. I copied the contents of a css file to 'test.html' so it was a css file but with an .html file extension and there was no issues. I could edit the test file in the host OS and have it served up by apache in the guest OS without any corruption. However when I changed the extension back to .css the problem resumed. I tested the other way too, creating an html file with .css as an extension and the corruption occurred, but not when the html file extension was reverted to .html.
I tried this with a couple different css files that I had and it seems that the problem only happens when the file size is more than ~2k. I could not replicate the problem on files that were ~120b but as soon as I added a bunch of css to the file to make it grow to more than ~4k the problem cropped up.
I have booted the Win7 host OS in restore mode and ran chkdsk while c:\ wasn't mounted and there were no returned errors. I have tried changing the css encoding from utf-8 to latin1.
To me it seems that css files are being corrupted when edited in the Win7 host OS if the css file is larger than ~2k on disk.
comment:25 by , 15 years ago
Update:
The guest OS is loaded and apache is running. I edit one of the css files in the host OS and save it. When I reload the css file in firefox in the host OS there are black diamonds with question marks at the bottom of the file.
I stop apache, unmount the vboxsf share, remount the share, and start apache. When I reload (f5) the file in Firefox the file is correct and as it should be. This leads me to believe that there is a problem in the way that vboxsf looks at how some files are changed or updated.
What can I do for the devs to help them track down this problem?
comment:26 by , 15 years ago
The solution that worked for me was suggested by a forum user (MarkCranness) was to turn off EnableSendfile in Apache. This changes the way that Apache reads a file before it serves it up when requested.
comment:27 by , 15 years ago
Summary: | vboxvfs seems to have problems with mmapped access to files => Fixed in SVN. → vboxvfs seems to have problems with mmapped access to files |
---|
Yes, this workaround is mentioned here in this ticket as well. We will try to fix this problem but not for 3.1.0 yet.
comment:28 by , 15 years ago
I seem to be having a similar problem with OpenSolaris 2009.06 guest in Ubuntu 9.10 host on VBox 3.0.12 using a freshly installed guest additions package. I mount a shared folder contain an Apache Ant 1.7.1 installation. Trying to run Ant fails:
$ /space/ant171/bin/ant -version mmap failed for CEN and END part of zip file Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/Launcher
even though I can access its files otherwise:
$ jar tvf /space/ant171/lib/ant.jar | head -5
0 Fri Jun 27 05:03:52 EDT 2008 META-INF/
449 Fri Jun 27 05:03:50 EDT 2008 META-INF/MANIFEST.MF
0 Fri Jun 27 05:03:44 EDT 2008 org/ 0 Fri Jun 27 05:03:44 EDT 2008 org/apache/ 0 Fri Jun 27 05:03:50 EDT 2008 org/apache/tools/
Generally, running any other Java application based on JARs fails if the JARs are on a shared folder. If I 'cp -av' the app to a local disk it runs fine.
comment:29 by , 15 years ago
Summary: | vboxvfs seems to have problems with mmapped access to files → vboxvfs seems to have problems with mmapped access to files => retry with 3.1.4 |
---|
Retry with 3.1.4 when released. Make sure to update the guest additions to 3.1.4 of course.
comment:31 by , 15 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic running OpenSolaris 9.06 with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26 (rebooted after updating new guest additions pkg).
jglick@jglick-solaris:~$ mount -v | fgrep /space space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010 jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar mmap failed for CEN and END part of zip file Exception in thread "main" java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:114) at java.util.jar.JarFile.<init>(JarFile.java:133) at java.util.jar.JarFile.<init>(JarFile.java:70)
jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar .
/space/tmp/sayhello/dist/sayhello.jar' ->
./sayhello.jar'
jglick@jglick-solaris:~$ java -jar sayhello.jar
hello world
jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar
0 Tue Feb 23 09:48:08 EST 2010 META-INF/
207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF
0 Tue Feb 23 09:48:08 EST 2010 sayhello/
533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class
jglick@jglick-solaris:~$
comment:32 by , 15 years ago
This may be easier to read:
Still easily reproducible in virtualbox-3.1 3.1.4-57640_Ubuntu_karmic
running OpenSolaris 9.06
with SUNWvboxguest 3.1.4,REV=r57640.2010.02.12.20.26
(rebooted after updating new guest additions pkg).
jglick@jglick-solaris:~$ mount -v | fgrep /space space on /space type vboxfs read/write/setuid/devices/dev=4e00001 on Tue Feb 23 09:45:26 2010 jglick@jglick-solaris:~$ java -jar /space/tmp/sayhello/dist/sayhello.jar mmap failed for CEN and END part of zip file Exception in thread "main" java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:114) at java.util.jar.JarFile.<init>(JarFile.java:133) at java.util.jar.JarFile.<init>(JarFile.java:70) jglick@jglick-solaris:~$ cp -v /space/tmp/sayhello/dist/sayhello.jar . `/space/tmp/sayhello/dist/sayhello.jar' -> `./sayhello.jar' jglick@jglick-solaris:~$ java -jar sayhello.jar hello world jglick@jglick-solaris:~$ jar tvf /space/tmp/sayhello/dist/sayhello.jar 0 Tue Feb 23 09:48:08 EST 2010 META-INF/ 207 Tue Feb 23 09:48:06 EST 2010 META-INF/MANIFEST.MF 0 Tue Feb 23 09:48:08 EST 2010 sayhello/ 533 Tue Feb 23 09:48:08 EST 2010 sayhello/Main.class jglick@jglick-solaris:~$
comment:33 by , 15 years ago
Moment, you are talking about a Solaris guest. Any positive feedback for Linux guests?
comment:34 by , 15 years ago
I have a similar problem, in that apache is delivering what appears to be a previous version of the file however truncated at the length of the current file.
Adding this to apache fixes the problem: EnableMMAP Off EnableSendfile Off
Here's the setup:
Host is: Ubuntu 9.10 Linux 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 02:39:34 UTC 2010 x86_64 GNU/Linux VirtualBox 3.1.4 r57640
Guest is: Debian GNU/Linux 4.0 Linux 2.6.18-6-amd64 #1 SMP Thu Dec 25 21:08:54 UTC 2008 x86_64 GNU/Linux VirtualBox 3.1.4 Guest Additions for Linux MD5="2490b313eb8c0168a7d7648e12f5803d"
DocumentRoot is mounted vboxsf of host
comment:35 by , 15 years ago
I recently came across what seems to be the same issue when trying to use mmap'd files on a vboxsf mounted share, on the (as of now) latest vbox version.
When files are opened read-only and used with mmap() it works fine, but it fails for files opened read-write and later used with mmap for writing.
Here is some code to reproduce:
// mmap_test.c #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <error.h> #include <errno.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/mman.h> int main(int argc, char *argv[]) { char *ro_mem, *rw_mem; // mmapped IO int ro_fd, rw_fd; off_t len_ro, len_rw; // read test - works OK on vboxsf if( (ro_fd = open("ro_file", O_RDONLY)) == -1) error(-1, errno, "Error opening 'ro_file'"); len_ro = lseek(ro_fd, 0L, SEEK_END); if((ro_mem = (char *) mmap(NULL, len_ro, PROT_READ, MAP_SHARED, ro_fd, 0)) == MAP_FAILED) error(-1, errno, "Error memory-mapping ro_file"); printf("ro_mem contents: %s", ro_mem); munmap(ro_mem, len_ro); close(ro_fd); // write test - fails on vboxsf if( (rw_fd = open("rw_file", O_RDWR)) == -1) error(-1, errno, "Error opening 'rw_file'"); len_rw = lseek(rw_fd, 0L, SEEK_END); if((rw_mem = (char *) mmap(NULL, len_ro, PROT_READ|PROT_WRITE, MAP_SHARED, rw_fd, 0)) == MAP_FAILED) error(-1, errno, "Error memory-mapping rw_file"); rw_mem[5] = 'X'; munmap(rw_mem, len_rw); close(rw_fd); // Changing the rw case to MAP_PRIVATE will succeed, but changes // aren't propogated to 'rw_file', as expected. // Removing PROT_WRITE still fails. // --> dies on open(O_RDWR),mmap(MAP_SHARED) pair return 0; }
#!/bin/sh # test.sh gcc -o mmap_test mmap_test.c echo "Sample text for readonly file" > ro_file echo "Write test text" > rw_file ./mmap_test cat rw_file # Output on a regular ext3fs dir: # ro_mem contents: Sample text for readonly file # WriteXtest text # Output on a vboxsf dir: # ro_mem contents: Sample text for readonly file # ./mmap_test: Error memory-mapping rw_file: Invalid argument # Write test text
versions: Host: Windows 7, Virtualbox 3.1.6 r59338 Guest: Ubuntu 9.10 Karmic - 2.6.31-20-generic-pae (Using vbox guest additions from host supplied ISO) vboxguest: major 0, IRQ 9, I/O port d040, MMIO at 00000000f0400000 (size 0x400000) vboxguest: Successfully loaded version 3.1.6 (interface 0x00010004) vboxvfs: Successfully loaded version 3.1.6 (interface 0x00010004)
Hope that helps
comment:36 by , 14 years ago
Host: Windows 7 64-bit running Virtual Box 3.1.8, r61349 Guest: Up to date Arch Linux: Linux perelman 2.6.33-ARCH #1 SMP PREEMPT Sun May 2 10:40:03 CEST 2010 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux Guest additions for 3.1.8 have been installed and are running.
To reproduce the problem:
- Run rtorrent 0.8.6 with a vboxsf mount as its download directory
- Try to download any torrent
- Observe every download failing with "Storage error: [File chunk write error: Invalid argument.]"
rtorrent uses mmap for this.
comment:37 by , 14 years ago
I'm having this exact problem in 3.1.8 (OpenSolaris 134 Host) in a Linux guest. I'm unable to try newer versions at this time. Reading a file with mmap() works fine while writing it does not.
comment:38 by , 14 years ago
Same problem with VirtualBox 3.2.12, Host: OSX 10.6.5, Guest: ubuntu-lucid-32
comment:39 by , 14 years ago
Same problem with VirtualBox 4.0.4, Host: Windows 7 x64, Guest: Oracle Solaris 11 Express 2010.11
comment:40 by , 14 years ago
No longer having any problems using Ubuntu host and XP guest (w/ 4.0.4 guest additions); ZIP file access working as expected.
comment:41 by , 13 years ago
Same issue with 4.0.8 on W7 ultimate and debian(lenny) guest with apache2 with JavaScript files EnableSendfile Off didn't help only sync; echo 3 > /proc/sys/vm/drop_caches, but running this every time I develop further is inconvenient
- Sandor
comment:43 by , 13 years ago
carstep, do you read files from the guest or do you write files to the guest?
follow-up: 46 comment:45 by , 13 years ago
The mmap problem may be the cause of http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.
comment:46 by , 13 years ago
Replying to davidsarah2:
The mmap problem may be the cause of http://bugs.darcs.net/issue2091 , where darcs fails to write to a repository stored in a vbox shared folder.
Note that this problem occurs still in VirtualBox 3.2.8_OSE r64453. In my case, the host is Ubuntu 10.10 and the shared folder is accessed from a Windows XP SP3 guest, although the original reporter of the darcs bug also had the problem when accessing a shared folder from Linux.
comment:47 by , 13 years ago
I can confirm this issue - not using Apache but using nginx. I don't know an option of nginx like EnableMMAP what you've written concerning Apache.
I've moved back to mount my shared-folder by samba to work.
To work arround this problem you can clear the pagecache of linux every second - like this:
watch -n 1 `sync; echo 1 > /proc/sys/vm/drop_caches`
Source: http://www.linuxinsight.com/proc_sys_vm_drop_caches.html
comment:48 by , 13 years ago
Sorry for this double-post .. but I'm not able to edit my last information. Here's another link where they explain a simular problem using more text :) http://serverfault.com/questions/30240/disable-all-disk-caching-for-apache2-on-linux
comment:49 by , 13 years ago
Using virtualbox-4.1.2-dfsg-1ubuntu1 on Ubuntu Oneiric with an XP guest (4.0.4 guest additions) I continue to have this problem sometimes, but not always; certain JAR files cannot be accessed from the guest, giving a ZIPException. Dropping the page caches from the host seems to make no difference. Using guest additions 4.1.2 it seems to be working better. Perhaps related to ticket #9276, mentioned in the changelog for this release?
comment:50 by , 13 years ago
Also getting this issue with Virtualbox 4.1.6. Using a Windows 7 x64 host and an Ubuntu Server guest. Fresh install from today. Attempting to use rtorrent to download to a vboxsf mount results in Storage error: [File chunk write error: Invalid argument.], an error that occurs due to an issue with mmap.
comment:51 by , 13 years ago
I'm experiencing a similar issue on VirtualBox 4.1.6, first experienced on 4.1.4. java apparently uses mmap to read zip'd jar files, and this fails for files larger than 2129615 (0x207ecf) bytes.
The next largest jar is 2213055 (0x21c4bf) and cannot be read.
I'm running a win xp guest on a linux host.
e: is a mounted shared directory from the linux host.
jar tvf e:\some\file.jar java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:127) at java.util.zip.ZipFile.<init>(ZipFile.java:88) at sun.tools.jar.Main.list(Main.java:979) at sun.tools.jar.Main.run(Main.java:224) at sun.tools.jar.Main.main(Main.java:1149)
comment:52 by , 12 years ago
Issue reproduced under Apache when updating .css files.
Disabling caches and MMAP didn't help.
Disabling also Sendfile fixed the issue.
Version used are:
Host = Windows 7 SP1 32 US
Guest = Entreprise Linux 2.6.32-279.11.1.el6.x86_64 + httpd-2.2.15-15.el6
using VirtualBox 4.2.0 r80737 + vboxguest/vboxsf 4.2.0
comment:54 by , 11 years ago
Issue is still occurring with VirtualBox 4.2.16, Ubuntu 13.10 host and Ubuntu 12.04 guest.
A workaround for nginx is to set "sendfile off;" (http://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile).
comment:55 by , 11 years ago
This is still an issue in the current VirtualBox version 4.3.6 with Ubuntu 12 and 13.
When using mmap operations on a vboxfs mapped folder it fails.
I was trying to use leveldb and lmdb operations on vboxfs mount, and couldn't open the files. Using an nfs share works however.
Here's some code that replicates the problem:
https://gist.github.com/9045290
If you compile this mmap-test, then running:
$ mmap-test /file/on/vboxfs/partition
fails with an invalid argument error
However, running it with a file on an nfs or ext4 partition works fine.
This is messing up local development workflows for anything that uses mmap (ie. a lot of software).
Please fix!
comment:56 by , 11 years ago
Just checking to see if anyone is currently working on this? Frank? Any luck replicating?
comment:57 by , 11 years ago
Hi Frank. I've just tested the latest version of VirtualBox (4.3.8) and the problem still remains?
Any update?
comment:58 by , 11 years ago
Description: | modified (diff) |
---|
Sorry, no update. Currently we have other priorities. If mmap-access to shared folders does not work, try to prevent it. For instance, Apache can be forced to not access files this way. If you are capable of Linux programming you might also want to check the source code. Patches help to fix such bugs faster.
comment:59 by , 10 years ago
I've came across this https://build.merproject.org/package/view_file?file=vbox-mmap.diff&package=virtualbox&project=nemo%3Adevel%3Ahw%3Ax86%3Ax86-common&rev=54d38848b69eea5527f8e1bbe9aae3b2
modification by mer project which supposedly adds mmap support to vboxsf module.
comment:60 by , 10 years ago
Still a problem in VirtualBox 4.3.12 r93733 for Mac OSX (Tested with Debian Wheezy guest).
comment:61 by , 10 years ago
I've replicated this in 4.3.10, host and guest both Ubuntu 14.04 x86_64.
mmap()ing a file with PROT_READ|PROT_WRITE fails with EINVAL.
comment:62 by , 10 years ago
Hi Frank,
Iskra Vitaly reported a potential diff that may fix the issue above:
Do you think you could spend a few minutes to see if it works?
Here's the code to replicate the issue: https://gist.github.com/9045290
We love your work, and there's a lot of us in the database community who can't get our apps to run because of this bug.
Pretty please? :-)
follow-up: 68 comment:63 by , 10 years ago
Thank you! I will have a look and consider to add the fix to the tree.
comment:64 by , 10 years ago
The issue is still present in 4.3.20 (OSX Yosemite) Observed with mkimage doing an mmap() of the kernel uImage file to create it.
comment:65 by , 9 years ago
I can confirm that this issue is already present in VirtualBox 4.3.18 [linux]. It was some hours that I was looking around for solve an issue of apache filling my javascript files with null bytes when modified on the host system (I'm developing using Vagrant).
EnableMMAP off in apache configuration solved my issue
g4b0
comment:66 by , 9 years ago
This is still an issue.
VirtualBox version: 5.0.10 r104061
OS X version: 10.11.2 (15C50)
In my personal case, I'm attempting to map local OS X volumes for data storage of Cassandra within a Docker container (for data persistence) and, end up hitting https://issues.apache.org/jira/browse/CASSANDRA-9635 through a shared volume within docker-machine (previously known as boot2docker in that ticket.)
@frank, are you still interested in that patch? It's an 8 year old bug now?
comment:67 by , 9 years ago
I should add that other Dockerized projects like Mongo, in addition to Cassandra, die when attempting to mount (OS X) volumes via vboxvfs due to the lack of mmap support.
https://github.com/docker-library/mongo/issues/30#issuecomment-108619916 https://github.com/docker-library/docs/tree/master/mongo#where-to-store-data
comment:68 by , 9 years ago
Replying to frank:
Thank you! I will have a look and consider to add the fix to the tree.
That comment was made 15 months ago. Was that patch considered, but rejected, or never considered further?
comment:70 by , 8 years ago
Exactly the same problem as described by Supermighty. I've VM with JS file and I can see the recent version via cat within VM, but not when requesting via Apache.
So (in VM):
cat foo.js -> current version curl "http://localhost/foo.js?nocache" -> old version
I've -HUP killed my apache, fully restarting, and there is no way I could see the current file.
I could see in access.log file being accessed, but it's the old version:
127.0.0.1 - - [09/May/2016:14:39:04 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2" ::1 - - [09/May/2016:14:39:18 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2" ::1 - - [09/May/2016:14:39:20 +0000] "GET /foo.js?foo HTTP/1.1" 200 382 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
I've checked using strace, open syscall is called on the file, so it looks like even Apache doesn't know the file is cached.
Here is the strace log:
$ sudo strace -fp 4911 -s 500 Process 4911 attached semop(524292, {{0, -1, SEM_UNDO}}, 1) = 0 epoll_wait(14, {{EPOLLIN, {u32=4157165232, u64=139831112453808}}}, 2, 10000) = 1 accept4(4, {sa_family=AF_INET6, sin6_port=htons(65349), inet_pton(AF_INET6, "::ffff:192.168.88.1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28], SOCK_CLOEXEC) = 15 semop(524292, {{0, 1, SEM_UNDO}}, 1) = 0 getsockname(15, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "::ffff:192.168.88.89", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0 fcntl(15, F_GETFL) = 0x2 (flags O_RDWR) fcntl(15, F_SETFL, O_RDWR|O_NONBLOCK) = 0 read(15, "GET /confirmation-page.js?asd HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Lan"..., 8000) = 2254 gettimeofday({1462805490, 328896}, NULL) = 0 stat("/confirmation-page.js", {st_mode=S_IFREG|0666, st_size=382, ...}) = 0 ... open("/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/confirmation-page.js/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOTDIR (Not a directory) stat("/confirmation-page.js", {st_mode=S_IFREG|0666, st_size=382, ...}) = 0 stat("/confirmation-page.js.gz", 0x7ffcc65ebd90) = -1 ENOENT (No such file or directory) open("/confirmation-page.js", O_RDONLY|O_CLOEXEC) = 16 fcntl(16, F_GETFD) = 0x1 (flags FD_CLOEXEC) fcntl(16, F_SETFD, FD_CLOEXEC) = 0 brk(0x7f2cf7ce8000) = 0x7f2cf7ce8000 setsockopt(15, SOL_TCP, TCP_CORK, [1], 4) = 0 writev(15, [{"HTTP/1.1 200 OK\r\nDate: Mon, 09 May 2016 14:51:30 GMT\r\nServer: Apache/2.2.15 (CentOS)\r\nX-Content-Type-Options: nosniff\r\nLast-Modified: Mon, 09 May 2016 14:25:11 GMT\r\nETag: \"3724-17e-53269937dbfc0\"\r\nAccept-Ranges: bytes\r\nContent-Length: 382\r\nCache-Control: max-age=1209600\r\nExpires: Mon, 23 May 2016 14:51:30 GMT\r\nConnection: close\r\nContent-Type: text/javascript\r\n\r\n", 364}], 1) = 364 sendfile(15, 16, [0], 382) = 382 setsockopt(15, SOL_TCP, TCP_CORK, [0], 4) = 0 write(10, "192.168.88.1 - - [09/May/2016:14:51:30 +0000] \"GET /confirmation-page.js?asd HTTP/1.1\" 200 382 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36\"\n", 275) = 275 shutdown(15, SHUT_WR) = 0
I also see black diamonds at the end of the file like:
������������������������������������������������������������
In curl as well:
$ curl "http://localhost/foo.js | cat -v (function($) { // Some content. })(jQuery); ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@[vagrant@html]$
No way it's some Apache level caching.
$ VBoxManage --version 5.0.14r105127 $ httpd -V Server version: Apache/2.2.15 (Unix) Architecture: 64-bit $ uname -a Linux localhost 2.6.32-573.18.1.el6.x86_64 #1 SMP Tue Feb 9 22:46:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
My docroot directory is synced in Vagrantfile as:
# Allow override of the default synced folder type. config.vm.synced_folder ".", "/vagrant", owner: vconfig.include?('vagrant_user') ? vconfig['vagrant_user'] : 'vagrant', group: "vagrant", type: vconfig.include?('vagrant_synced_folder_default_type') ? vconfig['vagrant_default_synced_folder_type'] : 'nfs', mount_options: vconfig.include?('vagrant_synced_folder_mount_options') ? vconfig['vagrant_synced_folder_mount_options'] : ["dmode=777,fmode=666"]
As for workaround, disabling EnableSendfile in Apache config worked great (as suggested by Supermighty)!
EnableSendfile Off
E.g.
<Directory "/path-to-nfs-files"> EnableSendfile Off </Directory>
This disables using the kernel sendfile support to deliver files to the client.
comment:71 by , 8 years ago
This is still an issue in 5.1.4. Is there any timeline to include fixes in the future release?
comment:72 by , 8 years ago
This also affects cargo (https://github.com/rust-lang/cargo/issues/2808), and anything that uses libgit2 (https://github.com/libgit2/libgit2/issues/3845). I tried to look at the patch mentioned by eugeneware, but all I see is a page with author & commit metadata but no code diff.
comment:73 by , 8 years ago
Ah hah, apparently I needed to resize the text box. Repasted at https://gist.github.com/jml/bfa756a071cfbf8c23033a8748aa4a11 in case that helps people.
comment:74 by , 7 years ago
Any update, please? This is really bugging me... Thank you
My use case is described at https://github.com/nette/caching/issues/37 (Issue with PRAGMA journal_mode = WAL;
with SQLite)
- Virtualbox: 5.1.26r117224
- Host OS: Ubuntu 16.04
- Guest OS: Ubuntu 16.04
comment:75 by , 7 years ago
Owner: | removed |
---|---|
Status: | reopened → assigned |
comment:77 by , 7 years ago
As per the #14920 (this issue as seen in a Docker container) I am experiencing this too.
- Host OS: Windows 10 x64 version 1709 (OS Build 16299.192)
- VirtualBox 5.2.6 r120293
- Vagrant 2.0.1
- Guest OS: Ubuntu 14.04, kernel 3.13.0-93-generic, vboxguest 4.3.36-dfsg-1+deb8u1ubuntu1.14.04.1, docker 17.05.0-ce
Guest OS sees latest edits of file mounted under /vagrant/ via vboxsf. Docker container with read-only volume mount sees (with cat, grep, and python) the older version of the file. Removing the Docker container and creating a new container still sees the older version. Executing mount
in the container shows the volume is using vboxsf.
Restarting the VM with vagrant reload
allows a new container to be started which sees the latest version of the file but then subsequent edits of the file are ignored again.
comment:78 by , 7 years ago
Hi All,
Someone pointed me to this bug by email, because I've been working on a version of the vboxsf driver for the upstream/mainline Linux kernel.
I've written a couple of patches on top of my driver, which I believe fix this.
My version of the vboxsf driver needs the upstream/mainline Linux kernel version of the vboxguest driver. So to test this you will need a 4.16-rc1 kernel with the vboxguest driver enabled in yuor guest, e.g.: https://koji.fedoraproject.org/koji/buildinfo?buildID=1043729
Then clone this git repo: https://github.com/jwrdegoede/vboxsf/ and do "cd vboxsf", "make" ,"sudo make modules_install" and then reboot.
With my vboxsf driver the problems reported in this bug will hopefully belong to the past.
There really were 3 problems:
1) MAP_SHARED not being supported 2) Changes done through regular read/write on the guest side not being seen by guest apps using mmap() access 3) Any changes done on the host side not being seen by guest apps using mmap() access
The last 3 commits here contain fixes for each of them for those of you who are interested: https://github.com/jwrdegoede/vboxsf/commits/master
Regards,
Hans
comment:79 by , 6 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
Summary: | vboxvfs seems to have problems with mmapped access to files => retry with 3.1.4 → vboxvfs seems to have problems with mmapped access to files => retry with 6.0.6 |
6.0.6 will ship some updates to shared folders, including read-write mmap for all supported linux guest kernels (including 2.4.x). We also consistently avoid any guest side caching for all 2.6.x and later kernels (for 2.4.x guests sendfile will use the cache), with the exception of read-write mmapped files where we by default go thru the cache to catch guest side modifications (dirty pages). The latter behavior can be modified by the new cache=none mount option.
We'll put out test builds later this week that includes the modifications, I hope.
comment:80 by , 5 years ago
The latter behavior can be modified by the new cache=none mount option.
Mounting with that option in 6.0.6 fails with:
kernel: vboxsf: Unrecognized mount option "cache=none" or missing value
comment:81 by , 5 years ago
re comment:80: You haven't installed the guest additions correctly. There is no message like that in our sources. The mount options gets repackaged in userland by a tool we install to /sbin/mount.vboxsf.
comment:82 by , 5 years ago
That was through fstab. Calling mount.vboxsf directly doesn't help:
$ sudo mount.vboxsf --options cache=none test /mnt/test mount: /mnt/test: wrong fs type, bad option, bad superblock on test, missing codepage or helper program, or other error. $ mount|grep test $ sudo mount.vboxsf test /mnt/test $ mount|grep test test on /mnt/test type vboxsf (rw,relatime)
comment:83 by , 5 years ago
VirtualBox 6.0.6 actually broke certain parts of shared folders. I stumbled upon it due to this bug: https://github.com/Ocramius/PackageVersions/issues/107, but after some digging it turns out that downgrading to 6.0.4 fixed the issue (I tried all newer versions up to and including 6.0.10).
Actually this problem resulted from a missing implementation of the Linux
sendfile
system call in our VFS module. We fixed this in SVN and will backport this fix to 1.5.4 if no regressions are observed. To fix your guest, reinstall the guest additions containing this fix (SVN for now).