Shrink hard disk operation is not implemented for .vhd

I can confirm this bug on XP sp3, VirtualBox 2.1.0

After defragging and zeroing a VDI, I attempted to compact it and got the 'temporarily unavailable' message.

I rolled back to VirtualBox 2.0.6 and the same command successfully compacted the same VDI.

I got the same error on Linux x64 host. This has probably been disabled by the developers. The question is why and when it will come back?

Is there any problem if I use a previous version of the VBoxManage to shrink it?

As of version 2.1.2 on a Linux x64 host, the error message has been updated. Now it says:

"Error: Shrink hard disk operation is not implemented!"

I guess this is better than the previous message. Any idea on when the implementation will be done, or why it was taken out? Are new features on the way?

I get the same Error on a Linux x32 Host (openSuse 10.3.) and VirtualBox 2.1.2.. Will this function return in coming versions? My WinXP-Image is now too big to fit on a DVD for backup. It really needs a shrink ;-)

I get this in Kubuntu Intrepid too, I'm posting to CC myself. I would really appreciate this getting fixed and would help if I knew how.

VirtualBox 2.1.2, Ubuntu 8.10 32-bit host, XP Home SP2 guest. Worked in 2.0.6.

Here's what it returns:

[!] FAILED calling a->virtualBox->FindHardDisk?2(filepath, hardDisk.asOutParam()) at line 226! [!] Primary RC = VBOX_E_OBJECT_NOT_FOUND (0x80BB0001) - Object corresponding to the supplied arguments does not exist [!] Full error info present: true , basic error info present: true [!] Result Code = VBOX_E_OBJECT_NOT_FOUND (0x80BB0001) - Object corresponding to the supplied arguments does not exist [!] Text = Could not find a hard disk with location '(vdi name)' in the media registry ('(correct path to xml file)') [!] Component = VirtualBox, Interface: IVirtualBox, {339abca2-f47a-4302-87f5-7bc324e6bbde} [!] Callee = IVirtualBox, {339abca2-f47a-4302-87f5-7bc324e6bbde} Error: Shrink hard disk operation is not implemented!

I can confirm the Solaris Nevada host running the VBox 2.1.2 giving for compact: --- bash-3.2$ VBoxManage modifyhd ~/.VirtualBox/VDI/WinXP.vdi compact VirtualBox Command Line Management Interface Version 2.1.2 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented! --- What kind of the shrink operation is implemented instead? It was working in 2.0.6. Is it regression?

I can confirm this in Ubuntu Intrepid (8.10) i386 (32 bit), VirtualBox 2.1.0 and 2.1.2.

Output:

VirtualBox Command Line Management Interface Version 2.1.2 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

Yes, this is a regression and it will still take some time to fix it. The reason are changes of the internal API. As a workaround, copying the VDI with VBoxManage clonehd should create a shrunk copy of the original image as well.

But keep in mind that cloning a HD will create a new image with a different UUID than the original.

Replying to frank:

But keep in mind that cloning a HD will create a new image with a different UUID than the original.

That's no problem, if I can shrink it. I have an old image, 7 GB so far, but it use 2 GB really. I had shrink it in the past, with VirtualBox 2.0.x (and perhaps 1.8.x?)

Since 2.1, I can't do it anymore. I tried using the old VDI format and the new VKMD format, without success.

I need to uninstall VirtualBox 2.1.x and go back to 2.0.x?. I have a few images in 2.1.x format, and it will be some problem for me now :(

I will be tuned :)

Thanks in advance (Sorry for my English)

Be very careful about using the clone HD feature, I have seen posts in other forums where people have corrupted the original drive and the cloned one by trying that method. Be sure to make a backup copy of your image before trying it.

Also, I upgraded to version 2.1.4 today, and the shrink hard disk operation is still not implemented.

win32-host users (or *nix users with a C compiler) could be interested in this workaround: http://forums.virtualbox.org/viewtopic.php?p=59796#59796

I can confirm this in Windows Server 2008 32bit Simplified Chinese Edtion, VirtualBox 2.1.4

Output:

VirtualBox Command Line Management Interface Version 2.1.4 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

Compact doesn't seem to work any more. Using latest windows version 2.1.4

....VDI\WinXPsp3.vdi" compact

VirtualBox Command Line Management Interface Version 2.1.4 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

Guys, it is not necessary to post this bug again and again. The bug is known and will be fixed as soon as possible.

Could you tell that modifications are needed in VDI header and what checks should be done for the current (2.1.4) image version to compress the image considering that there is a lot of zeroed sectors at the end of vdi file? Perhaps we would be able to create a script as a temporary workaround to the issue without waiting for new release.

@frank:

Obviously it needs to be reported again and again because you are not getting the clue that this issue should have been fixed 3 months ago when the bug was originally filed and need to be made aware of the large # of users it is affecting. Maybe your boss has prioritized other work so this got shoved aside, so the amount of people posting 'me too' may help you to get highlight the need to fix this issue.

Anyway.

This also happens to me, VirtualBox 2.1.4 x64, Windows 7

esac: your comment is completely useless. You are in no position to tell us what to do or not. No amount of complaints from your side will change the priority of this issue.

We are aware of the problem and will fix it. That's all you need to know.

Any more off-topic comments will be removed. Thank you.

I don't mean to upset anyone. Just want to be kept informed:
If you add a comment to an existing ticket, you will receive e-mail notification every time that ticket is updated.
(http://www.virtualbox.org/wiki/Bugtracker)

@sandervl73 I don't think yout attitude is appropriate to this situation. If users (=customers) are reporting bugs, I don't think its the right way to insult them. Maybe we're not in the position to tell you what to do but we're free to use another virtualization solution avaliable on the market. Your posting isn't very helpful to increase the trust in your company. Remember: We're talking about a feature which worked well for many versions and which disappeared suddenly without any visible reason and without any explanation from your side. Please understand that many users are angry about this situation.

Problem still occurs with VirtualBox 2.1.4 on openSuse Linux 11.1

Before you ask: That feature is still disabled in the upcoming release 2.2.

Some additional remarks: esac wrote: ``Obviously it needs to be reported again and again because you are not getting the clue that this issue should have been fixed 3 months ago when the bug was originally filed...'' Actually I think this statement is insulting and not appropriate for a a bug report. Note:

1. We don't want to bother you not implementing this missing feature for the new VBox media API. There are priorities and fixing host crashes and other stuff might have a higher priority than a function for shrinking a virtual hard disk. We will fix this bug as soon as possible.
2. As I wrote, you can use VBoxManage clonehd to achieve the same result for the time being. If this does not work for any reason then this is a bug and should be reported properly.
3. If you think implementing this missing feature is trivial to implement, feel free to submit a patch. The relevant source code is freely available.

Hello all, I've also stumbled upon this 'suddenly' dropped feature. Here's my IMO on this (and some workarounds too).

For backup purposes you don't actually need that image compacting feature. Zeroing-out is really essential, however (sdelete under win32 or other tools might be used). I always 7zip the images before burning/backing them up. Zeros are compressed efficiently (if you avoid scattered zero segments by defragging - it would be even better), so size of resulting 7zipped image should not be significantly affected by skipping the compacting step.

As for live usage of compacted images - it's hard to recommend anything, as the general case boils down to manual disk repartition/copy strategies. One of reasonable approaches is to allocate 20G expanding hd image and use only first 4G until your OS needs to expand. Then you resize partition or add a new partition (or disk) and mount it at existing folder, moving the original data to the new partition. So, in general, you *have* to use some carefully thought-out expansion scheme beforehand.

This compacting feature is not critical for most usages (though it of course might be handy sometimes).

Excellent news. Thank you.

We have an unconfirmed report of VDI corruption when using compact. Please backup your files before attempting to use this until further notice.

Indeed there was a typo and the shrink functionality of the 2.2.2 release should be considered as broken. Sorry guys but you will have to wait for the next maintenance release.

In the meantime, can we compact VDI files generated with VirtualBox 2.2.2 with a 2.2.0 install?

Replying to Tanelorn:

In the meantime, can we compact VDI files generated with VirtualBox 2.2.2 with a 2.2.0 install?

Hum, sorry, I didn't read the above posts properly (any way to delete a comment?). I'll wait for the next release and use clonehd for now.

Adding to akraievoy's 2009-04-15 post.

I found using GParted tool on the SystemRescueCD (http://www.sysresccd.org) quite useful as a workaround for managing this issue.

I had tried the VBoxManage clonehd command with an Open Suse 10.2 image of about 8 gigs (Ext3 file system). Resulting image was the same size. (vdi was using about 4 gigs but file size had grown to 8 gigs on disk). No joy!

So, I just made a new VDI (about 10 gigs expanding), attached it to my Machine (3rd hard drive), booted machine with SystemRescueCD. Once booted I fired up Gparted, shrunk the original partition to about 4 gigs (leaving the balance unformatted), then copied the partition to the new (vdi) hard drive (leaving the balance of this VDI HD unformatted, too).

Resulting new VDI was 4 gigs (both real disk space and used space on vdi). (Old VDI was still an 8 gig file.)

Then I just unattached the old VDI HD and attached the new one in its place. System booted right up without error.

(Haven't tested with a Windows VDI, check forums, thought I saw an issue on this regarding the UUID and booting, just too lazy to find it again right now.)

Also, search forums for: VBoxManage internalcommands sethduuid <filename>

Above command allows you to just reset the UUID without using VBoxManage clonehd.

It's a bit faster to just copy a VDI file and reset the UUID than using the clonehd command when just making a second machine. (Doesn't help the "shrinking" issue.)

I know the above properly belongs with some ongoing forum topics rather than here. But, since the "Shrink" feature is still bugged and not working, thought I'd point out this additional work-around to fill the gap until it's fixed.

You most likely didn't zero the free space in your virtual image using something like 'dd if=/dev/zero of=/tmp/zeroallspace' followed by 'rm /tmp/zeroallspace'.

If you had done so, the compacting would most likely had worked using VBoxManage clonehd.

I'm replying to subscribe to the mail notification. Thanks for the fixing effort.

Replying to fuUser:

You most likely didn't zero the free space in your virtual image using something like 'dd if=/dev/zero of=/tmp/zeroallspace' followed by 'rm /tmp/zeroallspace'. If you had done so, the compacting would most likely had worked using VBoxManage clonehd.

Actually I did try this again after your message. Here are the results:

Host system: Win XP SP2

Guest: Open Suse 10.3

VBox vers.: 2.14

Ran the dd commands from within the booted guest OS. Then closed the guest, detached the VDI from machine. Ran vboxmanage clonehd and got a small amount of size reduction on VDI file.

Original VDI file size: 5,896,233; reduced to 5,558,313. Approx 4.5 gigs used by OS, (plus a 760 MB swap disk in the VDI)

Then I started with the same VDI file and followed the routine I described above with the system rescue cd and GParted. VDI reduced to 5,234,729. Which is about the total of actual file sizes on the disk (within the guest OS).

Results were 5.8% reduction using clonehd and 11.3% using my method. I'll test again using a VDI that had grown about 3 gigs larger than "used" space in the guest and report results.

Question: Does the "DD" routine work better on a non-active partition? (Boot the machine from a rescue cd, mount the partition and run the "DD" routine while the actual guest OS is not running.) Any thoughts or experience with this?

You have to do it as root (or with sudo), as Linux reserves 5% (from total size) space by default, which can only be filled (or zeroed) from the root user (or a system process). You also have to make sure wherever you write to (the of=*something*) really writes to disk (some Linux buffer /tmp/ in RAM)

However my point is, that 'VBoxManage clonehd' and a working 'compact' would show exactly the same result, as you didn't zero all free space. (copying the disk (image) to another disk (image) however does)

But I think this is inappropriate to discuss here, please continue

to post whatever you may want to reply about your special on the forums.

(sry, pressed the Enter a little bit to soon and there seems to be no edit function)

Can be assumed that this "bug" is fixed with Version 2.2.4? It is not explicitly stated in the changelog:

Changelog: ... VBoxManage modifyhd --compact: fixed bug which could lead to crashes and image corruption (bug #3864) ...

Yes, shrink should finally work again. Still todo: implement shrink for .vhd images.

Replying to frank:

Still todo: implement shrink for .vhd images.

...and .vmdk images? I get the following message when attempting to compact a .vmdk:
Error: Compact hard disk operation for this format is not implemented yet!

Thanks for your efforts!

I also use .vmdk format. It would be great to have a buit-in shrink feature for this