In summary (for the benefit of anyone else who comes across these issues), if you get the VERR_VM_DRIVER_VERSION_MISMATCH error yet your kernel driver and VirtualBox userland version are the same, but you're running a 64-bit kernel with 32-bit userspace, create a 64-bit chroot environment:

sudo debootstrap --arch amd64 sid /var/64 ftp://ftp.debian.org/debian

Configure it:

sudo ln /etc/hostname /var/64/etc/hostname
sudo ln /etc/environment /var/64/etc/environment
sudo ln /etc/passwd /var/64/etc/passwd
sudo ln /etc/group /var/64/etc/group
sudo ln /etc/shadow /var/64/etc/shadow
sudo ln /etc/sudoers /var/64/etc/sudoers
sudo ln /etc/localtime /var/64/etc/localtime
sudo ln /etc/timezone /var/64/etc/timezone
sudo ln /etc/resolv.conf /var/64/etc/resolv.conf
sudo ln /etc/apt/sources.list /var/64/etc/apt/sources.list

Ensure various stuff is bound to the chroot environment; for this, add lines to your /etc/fstab:

/sys            /var/64/sys         none    bind                            0      0
/proc           /var/64/proc        none    bind                            0      0
/dev            /var/64/dev         none    bind                            0      0
/home           /var/64/home        none    bind                            0      0
/tmp            /var/64/tmp         none    bind                            0      0
/var/tmp        /var/64/var/tmp     none    bind                            0      0
/usr/src        /var/64/usr/src     none    bind                            0      0
/lib/modules    /var/64/lib/modules none    bind                            0      0

Then mount them:

for i in /var/64/{sys,proc,dev,home,tmp,var/tmp,usr/src,lib/modules}; do sudo mount $i; done

chroot into the environment:

sudo chroot /var/64 su - ${LOGNAME}

Set up the environment so you can launch X applications:

export XAUTHORITY=${HOME}/.Xauthority DISPLAY=:0

Install VirtualBox (I'm presuming you've already built modules outside of the chroot environment using m-a auto-install virtualbox-ose, which will pull virtualbox-ose-source out of apt and build the modules for you):

sudo touch /etc/init.d/udev
sudo apt-get install virtualbox-ose

Add yourself to the vboxusers group and load the driver:

sudo adduser ${LOGNAME} vboxusers
sudo modprobe vboxdrv

Create a symbolic link from ${HOME}/.VirtualBox to ~root/.VirtualBox:

sudo ln -s ${HOME}/.VirtualBox ~root/.VirtualBox

For bridged networking, create two scripts that will be used to bring up and down the TAP interfaces and install brctl:

sudo apt-get install uml-utilities bridge-utils
cat > vbox-ifup.sh <<EOF
#!/bin/bash
/usr/bin/sudo /sbin/ifconfig $2 up
/usr/bin/sudo /usr/sbin/brctl addif br0 $2
EOF
cat > vbox-ifdown.sh <<EOF
#!/bin/bash
/usr/bin/sudo /usr/sbin/brctl delif br0 $2
/usr/bin/sudo /sbin/ifconfig $2 down
EOF
chmod a+x vbox-ifup.sh vbox-ifdown.sh

Then start up the VirtualBox GUI using something like

sudo virtualbox &

and configure your VM to connect to a host interface named (e.g.) tap1. Point the startup and shutdown scripts at vbox-ifup.sh and vbox-ifdown.sh respectively.

Outside of the chroot, it would help to edit /etc/network/interfaces and make a bridged interface your primary network interface, and add the ethernet adapter as its slave:

auto br0
iface br0 inet dhcp
      bridge_ports eth0
      bridge_fd 9
      bridge_hello 2
      bridge_maxage 12
      bridge_stp off

Bring down the ethernet interface and bring up the bridge:

sudo ifdown eth0
sudo ifconfig eth0 up
sudo ifup br0

At this juncture VirtualBox should work when launched from the 64-bit chroot environment and provide bridged access to the VM.

Oh yeah, you'll also want to add root into the vboxusers group, since that's what VirtualBox will actually run as:

sudo adduser root vboxusers