the virtualbox GUI isn't accessible to those using screen readers

[Gregory Nowak]

This issue has already been discussed on the vbox-users mailing list:

​http://vbox.innotek.de/pipermail/vbox-users/2007-July/001803.html ​http://vbox.innotek.de/pipermail/vbox-users/2007-August/001978.html ​http://vbox.innotek.de/pipermail/vbox-users/2007-July/001834.html

Dmitry from innotek has already said that time would be invested into this issue, but I didn't see that a ticket was created for it, and thought it would be a good idea to create one, so that this problem isn't put aside and forgotten.

An ideal solution if possible, would be access via screen readers to the vbox GUI on all supported platforms, (I.E. windows, linux, macos, ETC.).

[Gregory Nowak]

I've just upgraded to vbox 1.6.0, and would like to start off by saying thank you to Sun for making a step in the right direction, in terms of gui accessibility. As of 1.6.0, the gui isn't fully accessible though. I am running virtualbox on a winxp pro host, with window-eyes 6.1 as my screen reader. Here are the problems I found.

1. When pressing alt to access the menu bar, the menu items speak, but the

menu names don't. This is true for both when alt is pressed, as well as when left/right arrow are used to move from menu to menu.

2. When tabbing from control to control in the preferences and vm settings

dialogue, I've found that in order to know what control I'm on, I have to tab away from that control, and then tab back to it. If I don't do this, I just hear "custom control".

3. If I'm on a radio button, and I up/down arrow to the other radio button, I

hear custom control, instead of hearing the name of the radio button, and its status. In order to hear that radio button and its status, I have to tab away from the radio buttons, and shift+tab back to that group of radio buttons.

4. Some of the controls still say custom control, no matter what I do. These

seem to be list boxes. One example of such a control, is the control that let's one choose the type of guest os one is running (I.E. linux 2.4, linux 2.6, windows xp, ETC.), in the vm settings dialogue.

5. Finally, when tabbing around in the virtual disk manager, all I hear is

window border, instead of custom control, or the name of the control.

Thank you once again for the progress that has been made so far, and please continue to keep up the good work. I realize that most or all of these problems may not be possible to fix further, since skype, another qt application I think, doesn't behave perfectly with window-eyes either. However, I thought I'd throw out there the problems I'm seeing so far, in the hope that gui accessibility can be improved upon even more.

[Gregory Nowak]

I've just upgraded to virtualbox 2.0.0. The upgrade to 2.0.0 and qt4 seems to be a major step backward. When attempting to arrow/tab around in the vbox interface, I hear "custom control" at best, nothing at all in the worst case scenario. The best way to summarize this, is to say that virtualbox 2.0.0 behaves with window-eyes the same way that virtualbox used to behave, prior to vbox 1.6.0.

[James Teh]

I can confirm that accessibility is totally broken in the closed-source edition of 2.0.0, where 1.6.x was at least partially accessible.

Some research reveals that QT's accessibility support can be built as a plugin or directly into the QT library: ​http://doc.trolltech.com/4.0/qt4-accessibility.html Is it possible that this issue is simply due to the accessibility support being accidentally omitted from the build? If so, can another build of the closed-source edition be provided with this fixed?

[Michael Thayer]

We are looking at this on several host platforms. Do you know how we can best test accessibility support on a Linux host?

[James Teh]

Replying to michael:

We are looking at this on several host platforms. Do you know how we can best test accessibility support on a Linux host?

GTK2 is accessible under Linux, but this obviously isn't relevant due to VirtualBox's use of QT4. There is an effort to implement code in QT4 to use the same accessibility API (AT-SPI) as Gnome/GTK2, but this is not yet complete and, as far as I can discover, it is not yet usable by users. This unfortunately means that VirtualBox isn't going to be accessible under Linux until this work is complete.

I can certainly help with any testing required under Windows.

[Frank Mehnert]

Please check VirtualBox 2.0.4 if it improves accessible support for you.

[Bram Duvigneau]

Here it's still totally inaccessible. Windows host, tested with the screenreaders: Jaws for windows (8 and 9), Window-eyes 7, Nvda 0.6 p1.

[Gregory Nowak]

I've upgraded from vbox 2.0.6 to 2.1.0, and thought I'd post a report, since there has been a change on the accessibility front, and I haven't seen anyone else comment on it here. I've been in touch with someone who claims that 2.0.6 was accessible for him, but this wasn't the case for me since 1.6.x, until the upgrade to 2.1.0. My first round of testing in 2.1.0 was done with window-eyes (wineyes) 6.1, which isn't the newest version of wineyes as of now, but it's what I have to work with. Using wineyes, the accessibility of vbox 2.1.0 can best be compared to accessibility in vbox 1.6.x, which I described in this ticket before. The main difference in accessibility between vbox 2.1.0, and 1.6.x using wineyes, is that the boxes where you can choose the guest os don't say custom control anymore, they now read as combo box, but that's all. If I use the read current line command, or the speak summary command in wineyes, all I hear on those combo boxes are things like choose os, and choose version (in the case of a linux guest), but I don't have a way of finding out what choice I'm on.

The other major change regarding vbox 2.1.0 and wineyes, is that the controls in the virtual media manager now speak either "custom control", or they actually read the control, (provided I tabbed away from the control, and tabbed back to it). This wasn't the case in 1.6.x, as I stated in my report for that version.

My second round of testing in vbox 2.1.0 was done with nonvisual desktop access (nvda) 0.6p2. Unlike wineyes, nvda really shines when it comes to the vbox GUI, to the point that vbox can be described as being almost fully accessible when using nvda, in my humble opinion at least. The first difference I noticed when using nvda, is that the controls speak immediately, I don't need to tab away from a control, and tab back to it, to hear what control I'm on. The second immediate difference when using nvda, is that not only am I told what control I'm on, but I'm also told what I'm expected to do with that control. For example, when in the vbox preferences, on the box that let's you type the location of vdi files, with nvda, I'm told that this is an edit box, and that this is where I'm supposed to type the location of where vdi files are stored, and I'm also told the text currently typed into that edit box. When on that same control with wineyes on the other hand, all I'm told is that this is an edit box, and I have to use the read current line command to discover what is typed in to the edit box. In most cases, what is typed into the edit box gives me a clue as to what the box is for, but not always, and wineyes doesn't provide that piece of information, as far as I can tell. The only drawback to edit box controls when using wineyes and nvda, is that when arrowing left/right, I don't know what letter I'm on, this is also true when backspacing. However, since it's possible to hear the entire contents of the edit box, this isn't a big problem when trying to change what is typed into the edit box.

Another similar drawback is in the vm settings, in the combo boxes where you select the guest os. When arrowing up/down, I don't hear the choice I'm on, but in the case of nvda, tabbing away from that control, and tabbing back to it, reads the current choice. In the case of wineyes, there's no way to tell what the current selection is, as far as I can tell.

Next, I'd like to mention the menubars. When using nvda, the menubars (the menubar in the main vbox window, as well as the other menubars, such as in the virtual media manager), are fully accessible, (I.E. the menus themselves speak, as well as the menu items in those menus). When using wineyes, the behavior with the menubars is the same as in vbox 1.6.x, the items speak, but the menus don't. Also when using wineyes, it is necessary to press the alt key a few times before I hear "menubar" instead of "custom control", this isn't the case when using nvda; the menus speak right away, just like the other controls do.

There still are however a few areas of vbox which aren't accessible mostly, or totally with both wineyes, and nvda. One such area is the main application window. I have been able to determine that there are 3 tab controls here, and I'll describe the kind of feedback (or lack thereof) that I get in each of these. In the first tab, I know from past experience that we have the list of virtual machines currently configured. Wineyes describes this list as custom control, and nvda simply calls it a pane, (I'll have more to say about panes later). When arrowing up/down in the list of configured guests, there is no way of telling what virtual machine one is currently on, with either screen reader. The only way to tell this, is to use the settings item from the machine menu, and see what the name of the guest is. It also is possible to tell what machine one is on by looking at the text box in this same tab, which I'll mention below. In summary, while it is possible to tell with some effort, which vm one is currently on, this isn't ideal, this information should be readily available when arrowing up/down. I've been able to determine that after the list of configured virtual machines, the next control is a text box, which seems to be the equivalent of vboxmanage's showvminfo, and this is the textbox I mentioned earlier, which is one of the ways to determine which vm is currently selected. One thing worth noting here, is that once on this textbox, I've found it dificult to tab away from it. It takes a repeated number of tab/shift+tab presses to get off the textbox control with either screen reader.

The second tab in the main window just has 2 panes, or custom controls, depending on what screen reader you use. It would be nice to know what these are, and how to make use of them, but I have no clue what they're for as of now. The 3rd, and seemingly last tab of the main window has an edit button (which nvda says can be accessed with ctrl+e), and another mysterious pane/custom control. I tried activating this edit button, but no matter what I do while using either screen reader, I get no speech, so I don't know what I'm supposed to be editing. The only way to return to the main window seems to be by pressing ESC. I should also mention that tab controls in the main window, in the virtual media manager, in vm settings, and in the preferences are not accessible. All I know is that I'm on a tab control, but I don't know the name of the tab control. This is true when using both screen readers.

There is as well another set of inaccessible controls in the settings dialogue, accessed from the machine menu of the main window. This set of inaccessible controls is the 3rd tab within settings, (the tab after the tab where you get to configure acpi, and hd controller). This tab seems to have 4 controls in it, what nvda calls "edit blank" (wineyes calls this a custom control), another pane/custom control, and ok/cancel buttons. I tried activating this "edit blank" control, but it behaves like the edit button in the main screen that I described earlier, and the only way to get out of this again is by pressing ESC.

Finally, there are a few more mysterious panes/custom controls. In preferences, and vm settings, they are found right after the help button, and before the tab control, there is one in each window. The other such pane/custom control, exists in the virtual media manager, in between the tab control, and the other set of controls in each tab.

Ok, I think this sums up my findings regarding GUI access in vbox 2.1.0. I realize I covered a lot here, and some things I said may not be clear, or may require more details. If there is anything that needs clarification, please post, and I'll do my best. Thanks for reading, and thanks again to the vbox team for a fine product.

[James Teh]

Replying to gnowak:

The only drawback to edit box controls when using wineyes and nvda, is that when arrowing left/right, I don't know what letter I'm on, this is also true when backspacing.

This is because the screen reader has no way of determining where the caret (system cursor) is located for QT applications. MSAA does not provide a way of accessing this information and QT only uses MSAA to provide accessibility. Screen readers which use a video intercept or display hooks can sometimes read the text and locate the caret using display information, but it seems that wineyes also has trouble with these QT edit fields. The only way to fix this would be to implement a richer accessibility API such as IAccessible2 or UI Automation in QT. It would be good if Sun could make it known to Trolltech (or is it Nokia now?) that supporting IAccessible2 on Windows in QT would very much be desirable for VirtualBox.

Another similar drawback is in the vm settings, in the combo boxes where you select the guest os. When arrowing up/down, I don't hear the choice I'm on, but in the case of nvda, tabbing away from that control, and tabbing back to it, reads the current choice.

Sounds like a value change event isn't being fired here. This is probably a bug in QT accessibility.

One such area is the main application window. I have been able to determine that there are 3 tab controls here

One tab control containing three tabs. The tabs in 1.6.x are: Details, Snapshots and Description.

In the first tab, I know from past experience that we have the list of virtual machines currently configured. Wineyes describes this list as custom control, and nvda simply calls it a pane

This is a regression, as this list used to read mostly fine with VirtualBox 1.6.x. Was this the same for you? (I'll have more to say about panes

The second tab in the main window just has 2 panes, or custom controls, depending on what screen reader you use.

Another regression - these report as a tree view and a list in 1.6.x and were once again mostly accessible.

The 3rd, and seemingly last tab of the main window has an edit button (which nvda says can be accessed with ctrl+e), and another mysterious pane/custom control.

This tab seems to be different in 1.6.x, as there is just a list, no button. Nevertheless, there seems to be a pattern here: many lists and trees aren't accessible in 2.0.x, whereas they were in 1.6.x. Is there some custom control being used which wasn't being used before? If a custom control has been designed, QT accessibility support needs to be implemented for this control.

I haven't yet tested VirtualBox 2.1.0, but shall test it soon. Thanks for the comprehensive report.

Btw, I am one of the core developers of ​NVDA. If there's anything we can do to help in terms of providing technical information or testing, please let me know. I am very much interested in helping to improve the accessibility of VirtualBox.

[Gregory Nowak]

Replying to jteh:

This is a regression, as this list used to read mostly fine with VirtualBox 1.6.x. Was this the same for you?

Yes, I thought I remembered that being the case, but wasn't sure, and didn't want to spread misinformation.

Another regression - these report as a tree view and a list in 1.6.x and were once again mostly accessible.

That's also what I seemed to recall, but again, wasn't sure.

Btw, I am one of the core developers of [​http://www.nvda-project.org/ NVDA].

I thought your vbox site username was familiar. Thanks to your team as well for the great work you're doing. It's nice to have the option of using a windows screen reader that's under the GPL, instead of being limited only to commercial products, which often exceed the price of a low-end PC, and for which the upgrades aren't cheap either. Python is on my list of languages to learn, but once I get around to doing that, I'd be interested in helping with nvda's development. Please continue to keep up the great work you're doing.

[Gregory Nowak]

The upgrade to vbox 2.1.2 has reminded me of another accessibility issue. If the format of the xml files has changed, upon launching the virtualbox GUI after the upgrade, one is presented with a dialogue with ok, more info, and backup buttons. If I wasn't familiar with this box's content from back when qt3 was being used, I would have no idea what this is all about. This is true both when using wineyes, and nvda. If I choose the more info button, the situation is the same, with an ok button, an edit button, and there's one more button which I can't recall. Again, I have no idea, (and I really don't this time), what this second dialogue box is for.

My suggestion here would be that the information displayed in these boxes be put into a textbox control, such as the textbox control that shows the description of the selected guest in the first tab of the main window when the vbox GUI is first launched.

[James Teh]

I've filed several accessibility issues (some encompassing the issues described above) as separate tickets. They can be found using this query.

[Gregory Nowak]

I've just upgraded to virtualbox 2.2.0, and noticed a slight regression when it comes to accessibility. When one is running a virtual machine, now and then, information, or error messages pop up, explaining about mouse integration, how to use the host key, that vbox was unable to lock and allocate memory, and so on. All of these messages have a check box, saying don't show this message again, an ok button, and a text box containing the message.

In virtualbox 2.1.4, the message was in an edit box, which meant that it was accessible by reading the current line. However, in virtualbox 2.2.0, the edit box is described as a custom control by wineyes, and as a pane by nvda, which means that the message can no longer be read. I don't remember for sure if the informational messages are laid out like this or not in vbox 2.1.4, but I do know for sure that the error message about not being able to lock and allocate memory, was contained in an edit box in vbox 2.1.4, and it is now in a custom control/pane as of virtualbox 2.2.0.

I don't recall seeing anything specific to qt4 in the changelog for vbox 2.2.0, so I don't know why there would be a difference here, but there definitely seems to be one.

[Technologov]

Here is a new article regarding this problem:

​http://prutser.wordpress.com/2009/03/22/should-i-switch-to-virtualbox/

As for me -- it is very unexpected, because, AFAIK, Qt4 uses standard native-platform widgets, so logically it should be fully accessible, just like native programs.

-Technologov, 9.7.2009.

[James Teh]

Replying to Technologov:

As for me -- it is very unexpected, because, AFAIK, Qt4 uses standard native-platform widgets

This is incorrect. QT4 uses its own custom widgets; I don't think it uses any native widgets at all. Thus, it must implement its own API accessibility.

[aeichner]

Please reopen if still relevant with a recent VirtualBox release.