the virtualbox GUI isn't accessible to those using screen readers

[Gregory Nowak]

This issue has already been discussed on the vbox-users mailing list:

​http://vbox.innotek.de/pipermail/vbox-users/2007-July/001803.html ​http://vbox.innotek.de/pipermail/vbox-users/2007-August/001978.html ​http://vbox.innotek.de/pipermail/vbox-users/2007-July/001834.html

Dmitry from innotek has already said that time would be invested into this issue, but I didn't see that a ticket was created for it, and thought it would be a good idea to create one, so that this problem isn't put aside and forgotten.

An ideal solution if possible, would be access via screen readers to the vbox GUI on all supported platforms, (I.E. windows, linux, macos, ETC.).

[Gregory Nowak]

I've just upgraded to vbox 1.6.0, and would like to start off by saying thank you to Sun for making a step in the right direction, in terms of gui accessibility. As of 1.6.0, the gui isn't fully accessible though. I am running virtualbox on a winxp pro host, with window-eyes 6.1 as my screen reader. Here are the problems I found.

1. When pressing alt to access the menu bar, the menu items speak, but the

menu names don't. This is true for both when alt is pressed, as well as when left/right arrow are used to move from menu to menu.

2. When tabbing from control to control in the preferences and vm settings

dialogue, I've found that in order to know what control I'm on, I have to tab away from that control, and then tab back to it. If I don't do this, I just hear "custom control".

3. If I'm on a radio button, and I up/down arrow to the other radio button, I

hear custom control, instead of hearing the name of the radio button, and its status. In order to hear that radio button and its status, I have to tab away from the radio buttons, and shift+tab back to that group of radio buttons.

4. Some of the controls still say custom control, no matter what I do. These

seem to be list boxes. One example of such a control, is the control that let's one choose the type of guest os one is running (I.E. linux 2.4, linux 2.6, windows xp, ETC.), in the vm settings dialogue.

5. Finally, when tabbing around in the virtual disk manager, all I hear is

window border, instead of custom control, or the name of the control.

Thank you once again for the progress that has been made so far, and please continue to keep up the good work. I realize that most or all of these problems may not be possible to fix further, since skype, another qt application I think, doesn't behave perfectly with window-eyes either. However, I thought I'd throw out there the problems I'm seeing so far, in the hope that gui accessibility can be improved upon even more.

[Gregory Nowak]

I've just upgraded to virtualbox 2.0.0. The upgrade to 2.0.0 and qt4 seems to be a major step backward. When attempting to arrow/tab around in the vbox interface, I hear "custom control" at best, nothing at all in the worst case scenario. The best way to summarize this, is to say that virtualbox 2.0.0 behaves with window-eyes the same way that virtualbox used to behave, prior to vbox 1.6.0.

[James Teh]

I can confirm that accessibility is totally broken in the closed-source edition of 2.0.0, where 1.6.x was at least partially accessible.

Some research reveals that QT's accessibility support can be built as a plugin or directly into the QT library: ​http://doc.trolltech.com/4.0/qt4-accessibility.html Is it possible that this issue is simply due to the accessibility support being accidentally omitted from the build? If so, can another build of the closed-source edition be provided with this fixed?

[Michael Thayer]

We are looking at this on several host platforms. Do you know how we can best test accessibility support on a Linux host?

[James Teh]

Replying to michael:

We are looking at this on several host platforms. Do you know how we can best test accessibility support on a Linux host?

GTK2 is accessible under Linux, but this obviously isn't relevant due to VirtualBox's use of QT4. There is an effort to implement code in QT4 to use the same accessibility API (AT-SPI) as Gnome/GTK2, but this is not yet complete and, as far as I can discover, it is not yet usable by users. This unfortunately means that VirtualBox isn't going to be accessible under Linux until this work is complete.

I can certainly help with any testing required under Windows.

[Frank Mehnert]

Please check VirtualBox 2.0.4 if it improves accessible support for you.

[Bram Duvigneau]

Here it's still totally inaccessible. Windows host, tested with the screenreaders: Jaws for windows (8 and 9), Window-eyes 7, Nvda 0.6 p1.

[Gregory Nowak]

I've upgraded from vbox 2.0.6 to 2.1.0, and thought I'd post a report, since there has been a change on the accessibility front, and I haven't seen anyone else comment on it here. I've been in touch with someone who claims that 2.0.6 was accessible for him, but this wasn't the case for me since 1.6.x, until the upgrade to 2.1.0. My first round of testing in 2.1.0 was done with window-eyes (wineyes) 6.1, which isn't the newest version of wineyes as of now, but it's what I have to work with. Using wineyes, the accessibility of vbox 2.1.0 can best be compared to accessibility in vbox 1.6.x, which I described in this ticket before. The main difference in accessibility between vbox 2.1.0, and 1.6.x using wineyes, is that the boxes where you can choose the guest os don't say custom control anymore, they now read as combo box, but that's all. If I use the read current line command, or the speak summary command in wineyes, all I hear on those combo boxes are things like choose os, and choose version (in the case of a linux guest), but I don't have a way of finding out what choice I'm on.

The other major change regarding vbox 2.1.0 and wineyes, is that the controls in the virtual media manager now speak either "custom control", or they actually read the control, (provided I tabbed away from the control, and tabbed back to it). This wasn't the case in 1.6.x, as I stated in my report for that version.

My second round of testing in vbox 2.1.0 was done with nonvisual desktop access (nvda) 0.6p2. Unlike wineyes, nvda really shines when it comes to the vbox GUI, to the point that vbox can be described as being almost fully accessible when using nvda, in my humble opinion at least. The first difference I noticed when using nvda, is that the controls speak immediately, I don't need to tab away from a control, and tab back to it, to hear what control I'm on. The second immediate difference when using nvda, is that not only am I told what control I'm on, but I'm also told what I'm expected to do with that control. For example, when in the vbox preferences, on the box that let's you type the location of vdi files, with nvda, I'm told that this is an edit box, and that this is where I'm supposed to type the location of where vdi files are stored, and I'm also told the text currently typed into that edit box. When on that same control with wineyes on the other hand, all I'm told is that this is an edit box, and I have to use the read current line command to discover what is typed in to the edit box. In most cases, what is typed into the edit box gives me a clue as to what the box is for, but not always, and wineyes doesn't provide that piece of information, as far as I can tell. The only drawback to edit box controls when using wineyes and nvda, is that when arrowing left/right, I don't know what letter I'm on, this is also true when backspacing. However, since it's possible to hear the entire contents of the edit box, this isn't a big problem when trying to change what is typed into the edit box.

Another similar drawback is in the vm settings, in the combo boxes where you select the guest os. When arrowing up/down, I don't hear the choice I'm on, but in the case of nvda, tabbing away from that control, and tabbing back to it, reads the current choice. In the case of wineyes, there's no way to tell what the current selection is, as far as I can tell.

Next, I'd like to mention the menubars. When using nvda, the menubars (the menubar in the main vbox window, as well as the other menubars, such as in the virtual media manager), are fully accessible, (I.E. the menus themselves speak, as well as the menu items in those menus). When using wineyes, the behavior with the menubars is the same as in vbox 1.6.x, the items speak, but the menus don't. Also when using wineyes, it is necessary to press the alt key a few times before I hear "menubar" instead of "custom control", this isn't the case when using nvda; the menus speak right away, just like the other controls do.

There still are however a few areas of vbox which aren't accessible mostly, or totally with both wineyes, and nvda. One such area is the main application window. I have been able to determine that there are 3 tab controls here, and I'll describe the kind of feedback (or lack thereof) that I get in each of these. In the first tab, I know from past experience that we have the list of virtual machines currently configured. Wineyes describes this list as custom control, and nvda simply calls it a pane, (I'll have more to say about panes later). When arrowing up/down in the list of configured guests, there is no way of telling what virtual machine one is currently on, with either screen reader. The only way to tell this, is to use the settings item from the machine menu, and see what the name of the guest is. It also is possible to tell what machine one is on by looking at the text box in this same tab, which I'll mention below. In summary, while it is possible to tell with some effort, which vm one is currently on, this isn't ideal, this information should be readily available when arrowing up/down. I've been able to determine that after the list of configured virtual machines, the next control is a text box, which seems to be the equivalent of vboxmanage's showvminfo, and this is the textbox I mentioned earlier, which is one of the ways to determine which vm is currently selected. One thing worth noting here, is that once on this textbox, I've found it dificult to tab away from it. It takes a repeated number of tab/shift+tab presses to get off the textbox control with either screen reader.

The second tab in the main window just has 2 panes, or custom controls, depending on what screen reader you use. It would be nice to know what these are, and how to make use of them, but I have no clue what they're for as of now. The 3rd, and seemingly last tab of the main window has an edit button (which nvda says can be accessed with ctrl+e), and another mysterious pane/custom control. I tried activating this edit button, but no matter what I do while using either screen reader, I get no speech, so I don't know what I'm supposed to be editing. The only way to return to the main window seems to be by pressing ESC. I should also mention that tab controls in the main window, in the virtual media manager, in vm settings, and in the preferences are not accessible. All I know is that I'm on a tab control, but I don't know the name of the tab control. This is true when using both screen readers.

There is as well another set of inaccessible controls in the settings dialogue, accessed from the machine menu of the main window. This set of inaccessible controls is the 3rd tab within settings, (the tab after the tab where you get to configure acpi, and hd controller). This tab seems to have 4 controls in it, what nvda calls "edit blank" (wineyes calls this a custom control), another pane/custom control, and ok/cancel buttons. I tried activating this "edit blank" control, but it behaves like the edit button in the main screen that I described earlier, and the only way to get out of this again is by pressing ESC.

Finally, there are a few more mysterious panes/custom controls. In preferences, and vm settings, they are found right after the help button, and before the tab control, there is one in each window. The other such pane/custom control, exists in the virtual media manager, in between the tab control, and the other set of controls in each tab.

Ok, I think this sums up my findings regarding GUI access in vbox 2.1.0. I realize I covered a lot here, and some things I said may not be clear, or may require more details. If there is anything that needs clarification, please post, and I'll do my best. Thanks for reading, and thanks again to the vbox team for a fine product.

[James Teh]

Replying to gnowak:

The only drawback to edit box controls when using wineyes and nvda, is that when arrowing left/right, I don't know what letter I'm on, this is also true when backspacing.

This is because the screen reader has no way of determining where the caret (system cursor) is located for QT applications. MSAA does not provide a way of accessing this information and QT only uses MSAA to provide accessibility. Screen readers which use a video intercept or display hooks can sometimes read the text and locate the caret using display information, but it seems that wineyes also has trouble with these QT edit fields. The only way to fix this would be to implement a richer accessibility API such as IAccessible2 or UI Automation in QT. It would be good if Sun could make it known to Trolltech (or is it Nokia now?) that supporting IAccessible2 on Windows in QT would very much be desirable for VirtualBox.

Another similar drawback is in the vm settings, in the combo boxes where you select the guest os. When arrowing up/down, I don't hear the choice I'm on, but in the case of nvda, tabbing away from that control, and tabbing back to it, reads the current choice.

Sounds like a value change event isn't being fired here. This is probably a bug in QT accessibility.

One such area is the main application window. I have been able to determine that there are 3 tab controls here

One tab control containing three tabs. The tabs in 1.6.x are: Details, Snapshots and Description.

In the first tab, I know from past experience that we have the list of virtual machines currently configured. Wineyes describes this list as custom control, and nvda simply calls it a pane

This is a regression, as this list used to read mostly fine with VirtualBox 1.6.x. Was this the same for you? (I'll have more to say about panes

The second tab in the main window just has 2 panes, or custom controls, depending on what screen reader you use.

Another regression - these report as a tree view and a list in 1.6.x and were once again mostly accessible.

The 3rd, and seemingly last tab of the main window has an edit button (which nvda says can be accessed with ctrl+e), and another mysterious pane/custom control.

This tab seems to be different in 1.6.x, as there is just a list, no button. Nevertheless, there seems to be a pattern here: many lists and trees aren't accessible in 2.0.x, whereas they were in 1.6.x. Is there some custom control being used which wasn't being used before? If a custom control has been designed, QT accessibility support needs to be implemented for this control.

I haven't yet tested VirtualBox 2.1.0, but shall test it soon. Thanks for the comprehensive report.

Btw, I am one of the core developers of ​NVDA. If there's anything we can do to help in terms of providing technical information or testing, please let me know. I am very much interested in helping to improve the accessibility of VirtualBox.

[Gregory Nowak]

Replying to jteh:

This is a regression, as this list used to read mostly fine with VirtualBox 1.6.x. Was this the same for you?

Yes, I thought I remembered that being the case, but wasn't sure, and didn't want to spread misinformation.

Another regression - these report as a tree view and a list in 1.6.x and were once again mostly accessible.

That's also what I seemed to recall, but again, wasn't sure.

Btw, I am one of the core developers of [​http://www.nvda-project.org/ NVDA].

I thought your vbox site username was familiar. Thanks to your team as well for the great work you're doing. It's nice to have the option of using a windows screen reader that's under the GPL, instead of being limited only to commercial products, which often exceed the price of a low-end PC, and for which the upgrades aren't cheap either. Python is on my list of languages to learn, but once I get around to doing that, I'd be interested in helping with nvda's development. Please continue to keep up the great work you're doing.

[Gregory Nowak]

The upgrade to vbox 2.1.2 has reminded me of another accessibility issue. If the format of the xml files has changed, upon launching the virtualbox GUI after the upgrade, one is presented with a dialogue with ok, more info, and backup buttons. If I wasn't familiar with this box's content from back when qt3 was being used, I would have no idea what this is all about. This is true both when using wineyes, and nvda. If I choose the more info button, the situation is the same, with an ok button, an edit button, and there's one more button which I can't recall. Again, I have no idea, (and I really don't this time), what this second dialogue box is for.

My suggestion here would be that the information displayed in these boxes be put into a textbox control, such as the textbox control that shows the description of the selected guest in the first tab of the main window when the vbox GUI is first launched.

[James Teh]

I've filed several accessibility issues (some encompassing the issues described above) as separate tickets. They can be found using this query.

[Gregory Nowak]

I've just upgraded to virtualbox 2.2.0, and noticed a slight regression when it comes to accessibility. When one is running a virtual machine, now and then, information, or error messages pop up, explaining about mouse integration, how to use the host key, that vbox was unable to lock and allocate memory, and so on. All of these messages have a check box, saying don't show this message again, an ok button, and a text box containing the message.

In virtualbox 2.1.4, the message was in an edit box, which meant that it was accessible by reading the current line. However, in virtualbox 2.2.0, the edit box is described as a custom control by wineyes, and as a pane by nvda, which means that the message can no longer be read. I don't remember for sure if the informational messages are laid out like this or not in vbox 2.1.4, but I do know for sure that the error message about not being able to lock and allocate memory, was contained in an edit box in vbox 2.1.4, and it is now in a custom control/pane as of virtualbox 2.2.0.

I don't recall seeing anything specific to qt4 in the changelog for vbox 2.2.0, so I don't know why there would be a difference here, but there definitely seems to be one.

[Technologov]

Here is a new article regarding this problem:

​http://prutser.wordpress.com/2009/03/22/should-i-switch-to-virtualbox/

As for me -- it is very unexpected, because, AFAIK, Qt4 uses standard native-platform widgets, so logically it should be fully accessible, just like native programs.

-Technologov, 9.7.2009.

[James Teh]

Replying to Technologov:

As for me -- it is very unexpected, because, AFAIK, Qt4 uses standard native-platform widgets

This is incorrect. QT4 uses its own custom widgets; I don't think it uses any native widgets at all. Thus, it must implement its own API accessibility.

[aeichner]

Please reopen if still relevant with a recent VirtualBox release.

[jožef gregorc]

Hi,

I've tested several versions of VB and found when it was transitioned to qt5, the accessibility was improved but there are still problems. these are:

• the gui of vm (ui where you can navigate to iso/setup files of guest Os) is completely inaccessible. It means that with any screen reader (tested with NVDA, JAWS and Window-Eyes) you can not manipulate with vm, so you can not save a vm state, you can not create snapshot, you can not press a browse button to navigate to place where setup files of guest os are stored.
• also found problems in settings area, where some controls are not accessible. e.g. these are comboboxes or treeview items in extensions section.
• please improve a proces of adding extension.

[Frank Mehnert]

This feature is currently being worked on.

[lukaszgo1]

Any progress on this? I am blind, and I am using VirtualBox almost everyday, however I have to stick to the 5.1 series, due to the fact, that accessibility of more recent versions is dramatically worse to the point of being unusable with a screen reader. If you need any help i.e additional feedback, testing of something or really anything, which could result in improved a11y of VirtualBox just let me know. Doing everything from the command line isn't ideal, and using older versions of VB is also not a great solution.

[lukaszgo1]

Ah, one more thing. Replying to frank:

This feature is currently being worked on.

It isn't a feature. Imagine the reverse situation e.e the GUI is accessible, but cannot be used by sighted. I very much doubt, that fixing such problem would take 10 years to fix.

[Klaus Espenlaub]

Frank's quote was from the days when a huge boost of a11y was in the works as part of 5.2, which unfortunately turned out to be practically a noop for Windows host.

We couldn't find out so far why on Windows the (hardened) VM process is not accessible at all, which is practically hiding all a11y progress in 5.2 and 6.0. Which is a pity, because the GUI devs spent a lot of effort on it, and didn't realize all their work was practically for nothing because they ran non-hardened binaries, as usual for developers. Since only the VM manager UI's a11y was in fact usable, and the VM runtime UI's a11y (what I assume you're needing) is practically non-accessible it's a really tough situation. It's possible that there's only a small technical change necessary to improve the situation, but given that hardening also makes the VM process un-debuggable it's extremely hard to find out what's really blocking the Windows accessibility framework from doing its job.

From what I read it's supposed to be based on a couple of COM services, which makes it only more mysterious since VirtualBox itself uses COM in many ways so it can't be a simple breakage.

[Michael Thayer]

Have you tried to see if the command line interface (VBoxManage) helps you as a work-around?

[lukaszgo1]

Replying to klaus:

Frank's quote was from the days when a huge boost of a11y was in the works as part of 5.2, which unfortunately turned out to be practically a noop for Windows host.

We couldn't find out so far why on Windows the (hardened) VM process is not accessible at all, which is practically hiding all a11y progress in 5.2 and 6.0. Which is a pity, because the GUI devs spent a lot of effort on it, and didn't realize all their work was practically for nothing because they ran non-hardened binaries, as usual for developers. Since only the VM manager UI's a11y was in fact usable,

The problem is, that ad least with NVDA which is my main screen reader the VM manager UI is no longer usable from version 5.2 onward. With 5.1 it was working fine. May I ask with which screen reader are you testing this

and the VM runtime UI's a11y (what I assume you're needing) is practically non-accessible it's a really tough situation. It's possible that there's only a small technical change necessary to improve the situation, but given that hardening also makes the VM process un-debuggable it's extremely hard to find out what's really blocking the Windows accessibility framework from doing its job.

From what I read it's supposed to be based on a couple of COM services, which makes it only more mysterious since VirtualBox itself uses COM in many ways so it can't be a simple breakage.

I am not a programmer, but are you by any chance building the vm GUI with GCC rather than with VS? If so it is almost certainly the problem, QT GUI's builded with GCC and compilers other than the Visual Studio aren't accessible under Windows.

[lukaszgo1]

Replying to michael:

Have you tried to see if the command line interface (VBoxManage) helps you as a work-around?

Yes, and I am using it for some tasks, which cannot be done from the GUI. As I said above I am sticking to the 5.1 series, and it is accessible enough, so command line is a last resort for me. But if, or rather when I would have to switch to more recent VB version this would be my only option.

[Valiant8086]

6.1.30 still has a lot of the same issues. Can we get a breakthrough on accessibility? When I get the select startup disk when I first play a brand new vm, the controls all say "unknown". Ditto when I get into the menu when on a vm window. Also I sometimes can't get my list of vm's to focus, apparently pressing enter on the tools item in the main window will help but.

[Klaus Espenlaub]

Hello Valiant8086 and all others interested in accessibility improvements:

Unfortunately there are technical reasons preventing a breakthrough in VirtualBox 6.1.x, which originates in the bugs and limitations of the Qt library we're using. We're very sorry about that.

For the next major VirtualBox release we already have invested time into improving the state of accessibility (which is finally possible with the newer version of Qt). It likely isn't perfect straight from the start, but my expectation based on the status reports for the "improve accessibility" task is that it will be usable.

If it is feasible for you to go for the VirtualBox development test builds then you could already start testing and providing feedback where we need to spend more effort. This would be greatly appreciated!

One known issue is that on Windows the VM window isn't accessible if you go for the standard way of starting a VM (which combines the UI and VM in one process). In this case the hardening of VM processes is blocking accessibility. However, there is an alternative (which I'm hoping will become the only available option before long anyway): starting the VM with detachable UI (using the "Detachable Start" option in the drop-down associated with the start button). You could also first start the VM headless (holding the Shift key while starting the VM in the usual way), and then use the "Show" button (into which the Start button turns when a VM is already running in headless mode).

If there are question, please ask. There will be still numerous changes and improvements until the development snapshot will become a BETA version of VirtualBox 7. Probably also some bugs and regressions, but we're doing our best to only update the development snapshot when it is reasonably stable.

[Ferna\ndoSilva]

Vows that everything is fine with everyone. I start by apologizing, this text is translated by google translator. I'm Brazilian and I don't speak English. I'm visually impaired and I'm a computer instructor for people with visual impairments. I was interested in trying out the virtual box to demonstrate to my students the experience of installing windows. I installed virtual box and had a problem navigating through the virtual box window to choose the virtual machine boot disk. I got in touch with nv access developer of the screen reader I'm using and I was instructed to read this topic as well as try out the test versions of the virtual box. While trying out the test versions of the virtual box I found some errors when using it with the nvda screen reader, where I was instructed to open a ticket at this location to report my experiences. When opening the virtual box version VirtualBox.exe, Oracle VM VirtualBox 6.1.97.148976 from the screen reader nvda.exe, NVDA alpha-24329,d3444424 when walking with tabe the screen reader announces only list, I expect screen reader to report the name of the list. When walking with the arrows up or down through the lists, the screen reader is mute, I don't know if these lists are empty or it is the screen reader who cannot read what is in these lists. Also when opening settings within the file menu, the screen reader only announces the help text, but when navigating with the arrows through the list, nvda does not read even an item, although the items are being changed, which indicates by the screen content being changed and checked when walking with tabe. When doing the same tests using the narrator screen reader, the home screen reader announces a list label as the name of the virtual machine I created, but when walking with tabe but not even a list label is informed, as well as when trying to navigate with the arrows like this or down through the lists not even an information is read. When entering the virtual box settings the list items are read, unlike nvda which did not read them. When opening the virtual box with the jaws, the virtual box program locked and closed. jaws version 2022 21.12 I would like to ask you to please verify these occurrences. I'm going to talk more about the nvda screen reader, which I use the most in my daily life, and it's the tool we use to teach classes. I opened a ticket with nv access, number: #13182 ​https://github.com/nvaccess/nvda/issues/13182 If anyone can help me in this call with nv access passing more information so that together we can reach a solution I would appreciate it. Finally, I emphasize that I am not a technical user, I am not a programmer, I define myself as a user with a little more ease of access trying to help and teach other visually impaired users. I would also like to apologize for any mistakes made. Thank you very much.