[vbox-dev] [Patch] Fix shared folder rewinddir() failure with guest Linux kernel >= 2.6.37

[Paul Mitchell]

Calls to rewinddir() on shared folder directory fds silently fail with kernel 2.6.37 and later.
This patch to the vboxsf driver restores the pre-2.6.37 behaviour.

For background details see "change default_llseek action" at http://lwn.net/Articles/405097/
The change was released with kernel 2.6.37, commit 776c163b1b93c8dfa5edba885bc2bfbc2d228a5f
PHP bug #55364 is symptomatic, see https://bugs.php.net/bug.php?id=55364

This test program checks that a call to readdir returns the same entry after rewinddir()
as one does after opendir(). Userspace is Ubuntu 10.04.3 LTS.

#include <sys/types.h>
#include <dirent.h>

void test_rewinddir( const char *path ) {
    struct dirent *e1, *e2;
    DIR *d = opendir( path );
    e1 = readdir( d );
    rewinddir( d );
    e2 = readdir( d );
    closedir( d );

    if( e1 == 0 ) return;
    puts( e2 && 0 == strcmp( e1->d_name, e2->d_name )
          ? "OK" : "FAIL" );
}

void main( int argc, const char **argv ) {
    while( --argc ) {
	puts( argv[argc] );
	test_rewinddir( argv[argc] );
    }
}

With the unpatched vboxsf driver on Linux kernel >= 2.6.37, the test will fail for a shared folder,
indicating that rewinddir failed. Running under strace, a lseek syscall for the directory fd can be
seen to fail with ESPIPE - the new default llseek behaviour.

With the patch to vboxsf on kernel >= 2.6.37, behaviour reverts to normal.

MIT licensed.

Index: src/VBox/Additions/linux/sharedfolders/dirops.c
===================================================================
--- src/VBox/Additions/linux/sharedfolders/dirops.c	(revision 38435)
+++ src/VBox/Additions/linux/sharedfolders/dirops.c	(working copy)
@@ -292,6 +292,9 @@
     .readdir = sf_dir_read,
     .release = sf_dir_release,
     .read    = generic_read_dir
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37)
+  , .llseek  = generic_file_llseek
+#endif
 };