vboxwebsrv dies when session is left open too long (fingerprint PAM module)

[Chris Brown]

When a session is established to vboxwebsrv and is open for too long vboxwebsrv goes into the following state (or dies):

00:41:32.956733 SQW01 authenticate(): result of AuthEntry(): 0
00:41:32.956775 SQPmp Request 1328 on socket 15 queued for processing (1 items on Q)
00:41:32.956818 SQPmp #### SOAP FAULT: Too many open files [SOAP-ENV:Server]
00:41:32.956836 SQPmp #### SOAP FAULT: Too many open files [SOAP-ENV:Server]
00:41:32.956847 SQW05 Processing connection from IP= socket=15 (2 out of 5 threads idle)

[Chris Brown]

vboxwebsrv log

[Chris Brown]

I note that this can also be caused by sending a large amount of API commands either over time or all at once to vboxwebsrv.

The vboxwebsrv crash also sometimes leaves the following in /var/log/messages:

kernel: SQW04[29960]: segfault at 0 ip 000000000040779f sp 00007fabbdfbea20 error 4 in vboxwebsrv[400000+1007000]
abrt[30620]: Saved core dump of pid 29929 (/usr/lib/virtualbox/vboxwebsrv) to /var/spool/abrt/ccpp-2014-03-12-14:35:32-29929 (15269888 bytes)

[Frank Mehnert]

crash0veride, is there any chance to provide a core dump?

[Chris Brown]

php test case

[Chris Brown]

coredump

[Chris Brown]

coredump and testcase attached.

[Frank Mehnert]

Thanks for the core dump. I can already say that the crash happens during authentication in pam_authenticate().

[Klaus Espenlaub]

Correct, it seems that the way you configured PAM causes a file descriptor leak. I'm quite certain that VBoxPAM.so doesn't contain simple leaks, so the question is how exactly you configured the PAM authentication. Can you provide the PAM config for the service which is used by VBoxPAM (defaults to "login")?

Another way of getting hints what's leaking (which in turn might give a hint what's going wrong) if you let your sample code run for a while, and then use the "lsof" utility to check what unexpected file descriptors are open.

Doesn't look like I can reproduce this. Most likely my system's PAM config is different, and the result is that for me the leak isn't happening.

[Chris Brown]

There is nothing special about the pam configuration everything is left at the OEL/EL 6.x defaults. I just verified it also occurs on Fedora 20 in exactly the same manner.

Brief OS load steps

• Load OEL or any EL 6.5, basic server load
  
• disable selinux
• Load VBox
  
• Create a user "vbox" via system-config-users UID:999 GID: 999
  
• Add vbox to group vboxusers (this is user is used for webservice authentication)
  
• vboxwebsrv runs as root (initscript default)
  

-> EG: /etc/vbox/vbox.cfg contents to have the start the service successfully:

VBOXWEB_USER=root <-- Required
VBOXWEB_HOST=0.0.0.0 <-- Optional

• testcase authenticates via user vbox
  
• actions of the webservice are performed by root (this behavior is actually desired)
  

[Frank Mehnert]

Tried to reproduce your problem on OL 6.4 and Debian 7.4. To run your script I commented out the utils.php line and defined foobar as false. Then the script does an endless run trying to authenticate forever.

On both systems the script will not run more than 30 seconds due to max_execution_time for PHP scripts. I don't see any crash of vboxwebsrv on both systems. However, on OL 6.4 I see an increasing amount of open pipes of the vboxwebsrv process while I can't see such a leak on Debian 7.4.

To me this looks like a problem of a system pam module or a problem of a system library.

[Frank Mehnert]

Actually the PAM config on Debian 7 and OL 6 is very different. Just found out that I cannot reproduce any pipe handle leak on OL6 if I uncomment the pam_fprintd.so line from /etc/pam.d/system-auth. Could you try the same and check if you still see crashes of vboxwebsrv on your systems?

[Frank Mehnert]

Actually

authconfig --disablefingerprint --update

is the correct command to disable that module system-wide.

[Chris Brown]

Verified. Disabling that module on 3 seperate OL 6.5 VBox hosts does clear up the issue.

[Chris Brown]

Looks like this is a lingering issue in EL 6.x
​RH BZ1010844

[aeichner]

Please reopen if still relevant with a recent VirtualBox release.