[vbox-dev] Using xattrs to exclude disk images from time machine under OSX

[Klaus Espenlaub]

Hi Andrew,

On 18.09.2015 18:28, Andrew Melo wrote:
> Hi Klaus,
>
> Thanks for the response.
>
> On Fri, Sep 18, 2015 at 11:20 AM, Klaus Espenlaub
> <klaus.espenlaub at oracle.com> wrote:
>> Hi Andrew,
>>
>> On 11.09.2015 05:00, Andrew Melo wrote:
>>> Hello,
>>>
>>> I am looking to implement enabling a setting for Virtualbox to
>>> automatically set the
>>> "com.apple.metadata:com_apple_backup_excludeItem" xattr for newly
>>> created disk images, most probably hooked into the global/per-VM
>>> extradata somewhere under the VboxInternal tree. This xattr will cause
>>> time machine to ignore the file when backing up, without having to
>>> manually add the directories where Virtualbox is storing the images.
>>
>> Sounds like a nice option to me, but not something which should happen
>> unconditionally (as some people might wan to have their VMs handled by
>> time machine).
>
> Oh, of course not, I would keep the defaults as-is and a user could
> choose to change the behavior if they chose.
>
>>
>>> Why don't I just add ~/Virtualbox\ VMs to the exclusion? Because
>>> there's some applications which integrate kicking off a VM for
>>> testing, and the images are stored next to the rest of the application
>>> state.
>>
>> I don't follow why you think 'time machining' VM configs without the
>> corresponding disk images makes any sense.
>
> Think of the situation where you kick off a webserver in a VM, but the
> files served are on host mounts. I don't care about the VM image, but
> I want the other bits. (ideally the VM image would be somewhere else)
>
>>
>>> Looking through the source tree, the change looks to be
>>> straightforward, but I don't want to take the time to make a patch if
>>> this feature wouldn't be considered for inclusion.
>>
>> The real challenge from my perspective is to make this configurable in a
>> sensible way. I'm almost certain that a significant number of users
>> would complain if for all newly created disk images VirtualBox would
>> automatically exclude them from the backup. Feel free to give arguments
>> which support the opposite :)
>
> I agree 100%, the default behavior would stay the same. I would then
> implement Doing The Right Thing based on the global/per-VM setting.
> (You could set it globally to not back up and then invert it on a
> per-VM basis if you wanted)

The problem with associating the setting with a VM is that the 
correlation between VMs and disk images is actually rather loose. As you 
described, they could live anywhere. Multiple VMs can share a set of 
disk images (but have their own diff images) and so on. Which VM's 
setting counts in this case?

Also, when is the setting applied? Since this is about disk images I 
would assume it is relevant to creation (of course can be changed 
later), and for such options there is no existing infrastructure.

>> I do agree that the basic feature isn't complicated. The tricky part is
>> finding a sensible way to configure this, as no other platform so far
>> needed a similar configuration.
>
> I think as a first pass, I'd code the actual "tell the OS to not back
> this file up" logic in a way that should be abstracted for different
> platforms, if they have similar functionality. In the OSX case, it's
> just setting an extended attribute on the file, so it's not very
> heavyweight.

Certainly - we have a nicely abstracted runtime which hides away many 
platform details, and it should be possible to find a few places where a 
newly added flag would do its low level work. Triggering the operation 
at a sensible time is then the next question.

I actually know of another platform where users would like a similar 
flag (not quite the same!): Linux with btrfs. There we get reports that 
disabling the copy-on-write behavior for disk images gives a performance 
boost. The conditions are actually harder than for what you have in 
mind, as the btrfs developers' rules are: do this at file creation time 
as changing the flag later doesn't have any guaranteed effect.

>> Thanks for your proposal, we'll see what comes out of it...
>
> Thanks for listening. If the maintainers consider it reasonable, I'll
> put a patch together implementing it.

The way you intend to implement it sounds very reasonable to me.

No one expects that this is done in a single change. In fact it's best 
if you contribute the building blocks first (they'll need some review if 
they fit into the overall design of the runtime) and later the 
additional changes which put it in action at the right time.

Cheers,
Klaus

> Cheers,
> Andrew
>
>>
>> Klaus
>>>
>>> Thanks!
>>> Andrew