I'm having the same problem with a Windows host and Windows guest. It happens not only over FTP but also large HTTP POSTs and probably any other upload. I examined a Wireshark capture and I believe the problem is as follows: after a fair amount of data has been transmitted by the guest, I guess there's a buffer in VirtualBox that starts to fill up and the ACKs that it sends back to the guest have smaller window sizes. Occasionally one of those window sizes will be less than the MSS, and because of Windows' (the guest) Silly Window Syndrome avoidance, it will stop transmitting packets to avoid sending one smaller than the MSS until a 5 second timeout has passed. So the transfer ends up going in burts, with 5 second delays interspersed.
Thank you for investigation and summary. Could you please clarify a bit your environment? For Virtualbox settings you may attach your log file. Do I understand right that you have got web server on the guest?
The workaround of increasing one of the NAT TCP/IP buffers helps, specifically the last number: