VirtualBox

Ticket #2833 (closed defect: fixed)

Opened 5 years ago

Last modified 11 months ago

Shrink hard disk operation is not implemented for .vhd

Reported by: mark_orion Owned by:
Priority: major Component: virtual disk
Version: VirtualBox 2.2.4 Keywords: VDI shrink VBoxManage
Cc: Guest type: other
Host type: other

Description (last modified by klaus) (diff)

  1. Shrinking a VDI fails with error - see below.
  2. Error message is confusing and not helpful. It is not clear if "temporary" relates to my system state, a missing component, the current version of Virtualbox or if it is just a polite way to admit a missing feature.

Others have reported the same problem on Windows host systems.

Here is the output of my attempt to shrink a VDI:

mark@aurora:~/.VirtualBox/VDI$ VBoxManage modifyhd /home/mark/.VirtualBox/VDI/WindowsXP.vdi compact VirtualBox Command Line Management Interface Version 2.1.0 (C) 2005-2008 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is temporarily unavailable!

Change History

comment:1 Changed 5 years ago by belzecue

I can confirm this bug on XP sp3, VirtualBox 2.1.0

After defragging and zeroing a VDI, I attempted to compact it and got the 'temporarily unavailable' message.

I rolled back to VirtualBox 2.0.6 and the same command successfully compacted the same VDI.

comment:2 Changed 5 years ago by nvivo

I got the same error on Linux x64 host. This has probably been disabled by the developers. The question is why and when it will come back?

Is there any problem if I use a previous version of the VBoxManage to shrink it?

comment:3 Changed 5 years ago by jbfoley

As of version 2.1.2 on a Linux x64 host, the error message has been updated. Now it says:

"Error: Shrink hard disk operation is not implemented!"

I guess this is better than the previous message. Any idea on when the implementation will be done, or why it was taken out? Are new features on the way?

comment:4 Changed 5 years ago by frank

  • Summary changed from Attempt to shrink VDI fails and throws "Error: Shrink hard disk operation is temporarily unavailable!" error to Shrink hard disk operation is not implemented

comment:5 Changed 5 years ago by margin-auto

I get the same Error on a Linux x32 Host (openSuse 10.3.) and VirtualBox 2.1.2.. Will this function return in coming versions? My WinXP-Image is now too big to fit on a DVD for backup. It really needs a shrink ;-)

comment:6 Changed 5 years ago by MichaelJE2

I get this in Kubuntu Intrepid too, I'm posting to CC myself. I would really appreciate this getting fixed and would help if I knew how.

comment:7 Changed 5 years ago by afo

VirtualBox 2.1.2, Ubuntu 8.10 32-bit host, XP Home SP2 guest. Worked in 2.0.6.

Here's what it returns:

[!] FAILED calling a->virtualBox->FindHardDisk2(filepath, hardDisk.asOutParam()) at line 226! [!] Primary RC = VBOX_E_OBJECT_NOT_FOUND (0x80BB0001) - Object corresponding to the supplied arguments does not exist [!] Full error info present: true , basic error info present: true [!] Result Code = VBOX_E_OBJECT_NOT_FOUND (0x80BB0001) - Object corresponding to the supplied arguments does not exist [!] Text = Could not find a hard disk with location '(vdi name)' in the media registry ('(correct path to xml file)') [!] Component = VirtualBox, Interface: IVirtualBox, {339abca2-f47a-4302-87f5-7bc324e6bbde} [!] Callee = IVirtualBox, {339abca2-f47a-4302-87f5-7bc324e6bbde} Error: Shrink hard disk operation is not implemented!

comment:8 Changed 5 years ago by jurasek

I can confirm the Solaris Nevada host running the VBox 2.1.2 giving for compact: --- bash-3.2$ VBoxManage modifyhd ~/.VirtualBox/VDI/WinXP.vdi compact VirtualBox Command Line Management Interface Version 2.1.2 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented! --- What kind of the shrink operation is implemented instead? It was working in 2.0.6. Is it regression?

comment:9 Changed 5 years ago by gorlok

I can confirm this in Ubuntu Intrepid (8.10) i386 (32 bit), VirtualBox 2.1.0 and 2.1.2.

Output:

VirtualBox Command Line Management Interface Version 2.1.2 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

comment:10 Changed 5 years ago by frank

Yes, this is a regression and it will still take some time to fix it. The reason are changes of the internal API. As a workaround, copying the VDI with VBoxManage clonehd should create a shrunk copy of the original image as well.

comment:11 follow-up: ↓ 12 Changed 5 years ago by frank

But keep in mind that cloning a HD will create a new image with a different UUID than the original.

comment:12 in reply to: ↑ 11 Changed 5 years ago by gorlok

Replying to frank:

But keep in mind that cloning a HD will create a new image with a different UUID than the original.

That's no problem, if I can shrink it. I have an old image, 7 GB so far, but it use 2 GB really. I had shrink it in the past, with VirtualBox 2.0.x (and perhaps 1.8.x?)

Since 2.1, I can't do it anymore. I tried using the old VDI format and the new VKMD format, without success.

I need to uninstall VirtualBox 2.1.x and go back to 2.0.x?. I have a few images in 2.1.x format, and it will be some problem for me now :(

I will be tuned :)

Thanks in advance (Sorry for my English)

comment:13 Changed 5 years ago by jbfoley

Be very careful about using the clone HD feature, I have seen posts in other forums where people have corrupted the original drive and the cloned one by trying that method. Be sure to make a backup copy of your image before trying it.

Also, I upgraded to version 2.1.4 today, and the shrink hard disk operation is still not implemented.

comment:14 Changed 5 years ago by jerome.hode

win32-host users (or *nix users with a C compiler) could be interested in this workaround:  http://forums.virtualbox.org/viewtopic.php?p=59796#59796

comment:15 Changed 5 years ago by dingdangy

I can confirm this in Windows Server 2008 32bit Simplified Chinese Edtion, VirtualBox 2.1.4

Output:

VirtualBox Command Line Management Interface Version 2.1.4 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

comment:16 Changed 5 years ago by iafrate

Compact doesn't seem to work any more. Using latest windows version 2.1.4

....VDI\WinXPsp3.vdi" compact

VirtualBox Command Line Management Interface Version 2.1.4 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.

Error: Shrink hard disk operation is not implemented!

comment:17 Changed 5 years ago by frank

Guys, it is not necessary to post this bug again and again. The bug is known and will be fixed as soon as possible.

comment:18 Changed 5 years ago by techtonik

Could you tell that modifications are needed in VDI header and what checks should be done for the current (2.1.4) image version to compress the image considering that there is a lot of zeroed sectors at the end of vdi file? Perhaps we would be able to create a script as a temporary workaround to the issue without waiting for new release.

comment:19 Changed 5 years ago by esac

@frank:

Obviously it needs to be reported again and again because you are not getting the clue that this issue should have been fixed 3 months ago when the bug was originally filed and need to be made aware of the large # of users it is affecting. Maybe your boss has prioritized other work so this got shoved aside, so the amount of people posting 'me too' may help you to get highlight the need to fix this issue.

Anyway.

This also happens to me, VirtualBox 2.1.4 x64, Windows 7

comment:20 Changed 5 years ago by sandervl73

esac: your comment is completely useless. You are in no position to tell us what to do or not. No amount of complaints from your side will change the priority of this issue.

We are aware of the problem and will fix it. That's all you need to know.

Any more off-topic comments will be removed. Thank you.

comment:21 Changed 5 years ago by olithered

I don't mean to upset anyone. Just want to be kept informed:
If you add a comment to an existing ticket, you will receive e-mail notification every time that ticket is updated.
( http://www.virtualbox.org/wiki/Bugtracker)

comment:22 Changed 5 years ago by margin-auto

@sandervl73 I don't think yout attitude is appropriate to this situation. If users (=customers) are reporting bugs, I don't think its the right way to insult them. Maybe we're not in the position to tell you what to do but we're free to use another virtualization solution avaliable on the market. Your posting isn't very helpful to increase the trust in your company. Remember: We're talking about a feature which worked well for many versions and which disappeared suddenly without any visible reason and without any explanation from your side. Please understand that many users are angry about this situation.

Problem still occurs with VirtualBox 2.1.4 on openSuse Linux 11.1

comment:23 Changed 5 years ago by frank

  • Version changed from VirtualBox 2.1.0 to VirtualBox 2.1.4

Before you ask: That feature is still disabled in the upcoming release 2.2.

comment:24 Changed 5 years ago by frank

Some additional remarks: esac wrote: ``Obviously it needs to be reported again and again because you are not getting the clue that this issue should have been fixed 3 months ago when the bug was originally filed...'' Actually I think this statement is insulting and not appropriate for a a bug report. Note:

  1. We don't want to bother you not implementing this missing feature for the new VBox media API. There are priorities and fixing host crashes and other stuff might have a higher priority than a function for shrinking a virtual hard disk. We will fix this bug as soon as possible.
  2. As I wrote, you can use VBoxManage clonehd to achieve the same result for the time being. If this does not work for any reason then this is a bug and should be reported properly.
  3. If you think implementing this missing feature is trivial to implement, feel free to submit a patch. The relevant source code is freely available.

comment:25 Changed 5 years ago by akraievoy

Hello all, I've also stumbled upon this 'suddenly' dropped feature. Here's my IMO on this (and some workarounds too).

For backup purposes you don't actually need that image compacting feature. Zeroing-out is really essential, however (sdelete under win32 or other tools might be used). I always 7zip the images before burning/backing them up. Zeros are compressed efficiently (if you avoid scattered zero segments by defragging - it would be even better), so size of resulting 7zipped image should not be significantly affected by skipping the compacting step.

As for live usage of compacted images - it's hard to recommend anything, as the general case boils down to manual disk repartition/copy strategies. One of reasonable approaches is to allocate 20G expanding hd image and use only first 4G until your OS needs to expand. Then you resize partition or add a new partition (or disk) and mount it at existing folder, moving the original data to the new partition. So, in general, you *have* to use some carefully thought-out expansion scheme beforehand.

This compacting feature is not critical for most usages (though it of course might be handy sometimes).

comment:26 Changed 5 years ago by klaus

  • Summary changed from Shrink hard disk operation is not implemented to Shrink hard disk operation is not implemented => fixed in SVN

comment:27 Changed 5 years ago by sandervl73

  • Status changed from new to closed
  • Resolution set to fixed

comment:28 Changed 5 years ago by gorlok

Excellent news. Thank you.

comment:29 Changed 5 years ago by sandervl73

We have an unconfirmed report of VDI corruption when using compact. Please backup your files before attempting to use this until further notice.

comment:30 Changed 5 years ago by frank

  • Status changed from closed to reopened
  • Resolution fixed deleted

comment:31 Changed 5 years ago by frank

Indeed there was a typo and the shrink functionality of the 2.2.2 release should be considered as broken. Sorry guys but you will have to wait for the next maintenance release.

comment:32 follow-up: ↓ 33 Changed 5 years ago by Tanelorn

In the meantime, can we compact VDI files generated with VirtualBox 2.2.2 with a 2.2.0 install?

comment:33 in reply to: ↑ 32 Changed 5 years ago by Tanelorn

Replying to Tanelorn:

In the meantime, can we compact VDI files generated with VirtualBox 2.2.2 with a 2.2.0 install?

Hum, sorry, I didn't read the above posts properly (any way to delete a comment?). I'll wait for the next release and use clonehd for now.

comment:34 Changed 5 years ago by timsart

Adding to akraievoy's 2009-04-15 post.

I found using GParted tool on the SystemRescueCD ( http://www.sysresccd.org) quite useful as a workaround for managing this issue.

I had tried the VBoxManage clonehd command with an Open Suse 10.2 image of about 8 gigs (Ext3 file system). Resulting image was the same size. (vdi was using about 4 gigs but file size had grown to 8 gigs on disk). No joy!

So, I just made a new VDI (about 10 gigs expanding), attached it to my Machine (3rd hard drive), booted machine with SystemRescueCD. Once booted I fired up Gparted, shrunk the original partition to about 4 gigs (leaving the balance unformatted), then copied the partition to the new (vdi) hard drive (leaving the balance of this VDI HD unformatted, too).

Resulting new VDI was 4 gigs (both real disk space and used space on vdi). (Old VDI was still an 8 gig file.)

Then I just unattached the old VDI HD and attached the new one in its place. System booted right up without error.

(Haven't tested with a Windows VDI, check forums, thought I saw an issue on this regarding the UUID and booting, just too lazy to find it again right now.)

Also, search forums for: VBoxManage internalcommands sethduuid <filename>

Above command allows you to just reset the UUID without using VBoxManage clonehd.

It's a bit faster to just copy a VDI file and reset the UUID than using the clonehd command when just making a second machine. (Doesn't help the "shrinking" issue.)

I know the above properly belongs with some ongoing forum topics rather than here. But, since the "Shrink" feature is still bugged and not working, thought I'd point out this additional work-around to fill the gap until it's fixed.

comment:35 follow-up: ↓ 37 Changed 5 years ago by fuUser

You most likely didn't zero the free space in your virtual image using something like 'dd if=/dev/zero of=/tmp/zeroallspace' followed by 'rm /tmp/zeroallspace'.

If you had done so, the compacting would most likely had worked using VBoxManage clonehd.

comment:36 Changed 5 years ago by dark

I'm replying to subscribe to the mail notification. Thanks for the fixing effort.

comment:37 in reply to: ↑ 35 Changed 5 years ago by timsart

Replying to fuUser:

You most likely didn't zero the free space in your virtual image using something like 'dd if=/dev/zero of=/tmp/zeroallspace' followed by 'rm /tmp/zeroallspace'.

If you had done so, the compacting would most likely had worked using VBoxManage clonehd.

Actually I did try this again after your message. Here are the results:

Host system: Win XP SP2

Guest: Open Suse 10.3

VBox vers.: 2.14

Ran the dd commands from within the booted guest OS. Then closed the guest, detached the VDI from machine. Ran vboxmanage clonehd and got a small amount of size reduction on VDI file.

Original VDI file size: 5,896,233; reduced to 5,558,313. Approx 4.5 gigs used by OS, (plus a 760 MB swap disk in the VDI)

Then I started with the same VDI file and followed the routine I described above with the system rescue cd and GParted. VDI reduced to 5,234,729. Which is about the total of actual file sizes on the disk (within the guest OS).

Results were 5.8% reduction using clonehd and 11.3% using my method. I'll test again using a VDI that had grown about 3 gigs larger than "used" space in the guest and report results.

Question: Does the "DD" routine work better on a non-active partition? (Boot the machine from a rescue cd, mount the partition and run the "DD" routine while the actual guest OS is not running.) Any thoughts or experience with this?

comment:38 Changed 5 years ago by fuUser

You have to do it as root (or with sudo), as Linux reserves 5% (from total size) space by default, which can only be filled (or zeroed) from the root user (or a system process). You also have to make sure wherever you write to (the of=*something*) really writes to disk (some Linux buffer /tmp/ in RAM)

However my point is, that 'VBoxManage clonehd' and a working 'compact' would show exactly the same result, as you didn't zero all free space. (copying the disk (image) to another disk (image) however does)

But I think this is inappropriate to discuss here, please continue

comment:39 Changed 5 years ago by fuUser

to post whatever you may want to reply about your special on the forums.

(sry, pressed the Enter a little bit to soon and there seems to be no edit function)

comment:40 Changed 5 years ago by fuUser

Can be assumed that this "bug" is fixed with Version 2.2.4? It is not explicitly stated in the changelog:

Changelog: ... VBoxManage modifyhd --compact: fixed bug which could lead to crashes and image corruption (bug #3864) ...

comment:41 Changed 5 years ago by frank

Yes, shrink should finally work again. Still todo: implement shrink for .vhd images.

comment:42 follow-up: ↓ 43 Changed 5 years ago by frank

  • Host type changed from Linux to other
  • Version changed from VirtualBox 2.1.4 to VirtualBox 2.2.4
  • Component changed from VMM to virtual disk
  • Guest type changed from Windows to other
  • Summary changed from Shrink hard disk operation is not implemented => fixed in SVN to Shrink hard disk operation is not implemented for .vhd

comment:43 in reply to: ↑ 42 Changed 5 years ago by avfan

Replying to frank:

Still todo: implement shrink for .vhd images.

...and .vmdk images? I get the following message when attempting to compact a .vmdk:
Error: Compact hard disk operation for this format is not implemented yet!

Thanks for your efforts!

comment:44 Changed 5 years ago by bliss

I also use .vmdk format. It would be great to have a buit-in shrink feature for this

comment:45 Changed 3 years ago by lambermo

The new --compact results are unexpected. Linux 64-bit guest running CentOS-5.5 x86_64, host runs virtualbox-ose 3.2.8-dfsg-2ubuntu1 I start with a VDI of : 13097865728 bytes. This VDI has a single fs in it, it uses 10 GB. Running VBoxManage modifyhd --compact on this without preparation results in : 13097865728 bytes. Which is the same, OK, i did not clean the unused space in the guest : I ran in the guest : dd if=/dev/zero of=/zeroes bs=1M ; rm -f /zeroes This made the VDI grow to : 16130347520 bytes. A bit weird, but fine. Running VBoxManage modifyhd --compact on this results in : 15729791488 bytes. This is more than what I started with ! Quite the opposite of what I expected.

comment:46 Changed 2 years ago by klaus

  • Description modified (diff)

labermo: in modern filesystems it doesn't work any more to create a file containing just zeroes to clear the free space. This is why you get this seemingly paradoxical behavior.

comment:47 Changed 11 months ago by aeichner

  • Status changed from reopened to closed
  • Resolution set to fixed

The shrink support for VHD is supported. We are aware that VMDK is not supported yet but we don't have time for this at the moment. Patches are welcome of course :) Closing this defect, if necessary create a new defect for vmdk images.

Note: See TracTickets for help on using tickets.

www.oracle.com
ContactPrivacy policyTerms of Use