VirtualBox

source: vbox/trunk/src/VBox/ValidationKit/docs/VBoxAudioValidationKitReadMe.html@ 103068

Last change on this file since 103068 was 98107, checked in by vboxsync, 2 years ago

Manual (C) year updates.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 17.7 KB
Line 
1<?xml version="1.0" encoding="utf-8" ?>
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4<head>
5<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
6<meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
7<title>Audio Testing of VirtualBox</title>
8<style type="text/css">
9
10/*
11:Author: David Goodger (goodger@python.org)
12:Id: $Id: VBoxAudioValidationKitReadMe.html 98107 2023-01-17 22:56:50Z vboxsync $
13:Copyright: This stylesheet has been placed in the public domain.
14
15Default cascading style sheet for the HTML output of Docutils.
16
17See https://docutils.sourceforge.io/docs/howto/html-stylesheets.html for how to
18customize this style sheet.
19*/
20
21/* used to remove borders from tables and images */
22.borderless, table.borderless td, table.borderless th {
23 border: 0 }
24
25table.borderless td, table.borderless th {
26 /* Override padding for "table.docutils td" with "! important".
27 The right padding separates the table cells. */
28 padding: 0 0.5em 0 0 ! important }
29
30.first {
31 /* Override more specific margin styles with "! important". */
32 margin-top: 0 ! important }
33
34.last, .with-subtitle {
35 margin-bottom: 0 ! important }
36
37.hidden {
38 display: none }
39
40.subscript {
41 vertical-align: sub;
42 font-size: smaller }
43
44.superscript {
45 vertical-align: super;
46 font-size: smaller }
47
48a.toc-backref {
49 text-decoration: none ;
50 color: black }
51
52blockquote.epigraph {
53 margin: 2em 5em ; }
54
55dl.docutils dd {
56 margin-bottom: 0.5em }
57
58object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
59 overflow: hidden;
60}
61
62/* Uncomment (and remove this text!) to get bold-faced definition list terms
63dl.docutils dt {
64 font-weight: bold }
65*/
66
67div.abstract {
68 margin: 2em 5em }
69
70div.abstract p.topic-title {
71 font-weight: bold ;
72 text-align: center }
73
74div.admonition, div.attention, div.caution, div.danger, div.error,
75div.hint, div.important, div.note, div.tip, div.warning {
76 margin: 2em ;
77 border: medium outset ;
78 padding: 1em }
79
80div.admonition p.admonition-title, div.hint p.admonition-title,
81div.important p.admonition-title, div.note p.admonition-title,
82div.tip p.admonition-title {
83 font-weight: bold ;
84 font-family: sans-serif }
85
86div.attention p.admonition-title, div.caution p.admonition-title,
87div.danger p.admonition-title, div.error p.admonition-title,
88div.warning p.admonition-title, .code .error {
89 color: red ;
90 font-weight: bold ;
91 font-family: sans-serif }
92
93/* Uncomment (and remove this text!) to get reduced vertical space in
94 compound paragraphs.
95div.compound .compound-first, div.compound .compound-middle {
96 margin-bottom: 0.5em }
97
98div.compound .compound-last, div.compound .compound-middle {
99 margin-top: 0.5em }
100*/
101
102div.dedication {
103 margin: 2em 5em ;
104 text-align: center ;
105 font-style: italic }
106
107div.dedication p.topic-title {
108 font-weight: bold ;
109 font-style: normal }
110
111div.figure {
112 margin-left: 2em ;
113 margin-right: 2em }
114
115div.footer, div.header {
116 clear: both;
117 font-size: smaller }
118
119div.line-block {
120 display: block ;
121 margin-top: 1em ;
122 margin-bottom: 1em }
123
124div.line-block div.line-block {
125 margin-top: 0 ;
126 margin-bottom: 0 ;
127 margin-left: 1.5em }
128
129div.sidebar {
130 margin: 0 0 0.5em 1em ;
131 border: medium outset ;
132 padding: 1em ;
133 background-color: #ffffee ;
134 width: 40% ;
135 float: right ;
136 clear: right }
137
138div.sidebar p.rubric {
139 font-family: sans-serif ;
140 font-size: medium }
141
142div.system-messages {
143 margin: 5em }
144
145div.system-messages h1 {
146 color: red }
147
148div.system-message {
149 border: medium outset ;
150 padding: 1em }
151
152div.system-message p.system-message-title {
153 color: red ;
154 font-weight: bold }
155
156div.topic {
157 margin: 2em }
158
159h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
160h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
161 margin-top: 0.4em }
162
163h1.title {
164 text-align: center }
165
166h2.subtitle {
167 text-align: center }
168
169hr.docutils {
170 width: 75% }
171
172img.align-left, .figure.align-left, object.align-left, table.align-left {
173 clear: left ;
174 float: left ;
175 margin-right: 1em }
176
177img.align-right, .figure.align-right, object.align-right, table.align-right {
178 clear: right ;
179 float: right ;
180 margin-left: 1em }
181
182img.align-center, .figure.align-center, object.align-center {
183 display: block;
184 margin-left: auto;
185 margin-right: auto;
186}
187
188table.align-center {
189 margin-left: auto;
190 margin-right: auto;
191}
192
193.align-left {
194 text-align: left }
195
196.align-center {
197 clear: both ;
198 text-align: center }
199
200.align-right {
201 text-align: right }
202
203/* reset inner alignment in figures */
204div.align-right {
205 text-align: inherit }
206
207/* div.align-center * { */
208/* text-align: left } */
209
210.align-top {
211 vertical-align: top }
212
213.align-middle {
214 vertical-align: middle }
215
216.align-bottom {
217 vertical-align: bottom }
218
219ol.simple, ul.simple {
220 margin-bottom: 1em }
221
222ol.arabic {
223 list-style: decimal }
224
225ol.loweralpha {
226 list-style: lower-alpha }
227
228ol.upperalpha {
229 list-style: upper-alpha }
230
231ol.lowerroman {
232 list-style: lower-roman }
233
234ol.upperroman {
235 list-style: upper-roman }
236
237p.attribution {
238 text-align: right ;
239 margin-left: 50% }
240
241p.caption {
242 font-style: italic }
243
244p.credits {
245 font-style: italic ;
246 font-size: smaller }
247
248p.label {
249 white-space: nowrap }
250
251p.rubric {
252 font-weight: bold ;
253 font-size: larger ;
254 color: maroon ;
255 text-align: center }
256
257p.sidebar-title {
258 font-family: sans-serif ;
259 font-weight: bold ;
260 font-size: larger }
261
262p.sidebar-subtitle {
263 font-family: sans-serif ;
264 font-weight: bold }
265
266p.topic-title {
267 font-weight: bold }
268
269pre.address {
270 margin-bottom: 0 ;
271 margin-top: 0 ;
272 font: inherit }
273
274pre.literal-block, pre.doctest-block, pre.math, pre.code {
275 margin-left: 2em ;
276 margin-right: 2em }
277
278pre.code .ln { color: grey; } /* line numbers */
279pre.code, code { background-color: #eeeeee }
280pre.code .comment, code .comment { color: #5C6576 }
281pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
282pre.code .literal.string, code .literal.string { color: #0C5404 }
283pre.code .name.builtin, code .name.builtin { color: #352B84 }
284pre.code .deleted, code .deleted { background-color: #DEB0A1}
285pre.code .inserted, code .inserted { background-color: #A3D289}
286
287span.classifier {
288 font-family: sans-serif ;
289 font-style: oblique }
290
291span.classifier-delimiter {
292 font-family: sans-serif ;
293 font-weight: bold }
294
295span.interpreted {
296 font-family: sans-serif }
297
298span.option {
299 white-space: nowrap }
300
301span.pre {
302 white-space: pre }
303
304span.problematic {
305 color: red }
306
307span.section-subtitle {
308 /* font-size relative to parent (h1..h6 element) */
309 font-size: 80% }
310
311table.citation {
312 border-left: solid 1px gray;
313 margin-left: 1px }
314
315table.docinfo {
316 margin: 2em 4em }
317
318table.docutils {
319 margin-top: 0.5em ;
320 margin-bottom: 0.5em }
321
322table.footnote {
323 border-left: solid 1px black;
324 margin-left: 1px }
325
326table.docutils td, table.docutils th,
327table.docinfo td, table.docinfo th {
328 padding-left: 0.5em ;
329 padding-right: 0.5em ;
330 vertical-align: top }
331
332table.docutils th.field-name, table.docinfo th.docinfo-name {
333 font-weight: bold ;
334 text-align: left ;
335 white-space: nowrap ;
336 padding-left: 0 }
337
338/* "booktabs" style (no vertical lines) */
339table.docutils.booktabs {
340 border: 0px;
341 border-top: 2px solid;
342 border-bottom: 2px solid;
343 border-collapse: collapse;
344}
345table.docutils.booktabs * {
346 border: 0px;
347}
348table.docutils.booktabs th {
349 border-bottom: thin solid;
350 text-align: left;
351}
352
353h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
354h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
355 font-size: 100% }
356
357ul.auto-toc {
358 list-style-type: none }
359
360</style>
361</head>
362<body>
363<div class="document" id="audio-testing-of-virtualbox">
364<h1 class="title">Audio Testing of VirtualBox</h1>
365
366<div class="section" id="overview-goal">
367<h1>Overview / Goal</h1>
368<p>The goal is to create a flexible testing framework to test the
369VirtualBox audio stack.</p>
370<p>It should be runnable with an easy-to-use setup so that also regular users
371can perform tests on request, without having to install or set up additional
372dependencies.</p>
373<p>That framework must be runnable on all host/guest combinations together with all
374audio drivers (&quot;backends&quot;) and device emulations being offered. This makes it a
375rather big testing matrix which therefore has to be processed in an automated
376fashion.</p>
377<p>Additionally it should be flexible enough to add more (custom) tests later on.</p>
378</div>
379<div class="section" id="operation">
380<h1>Operation</h1>
381<p>The framework consists of several components which try to make use as much of
382the existing audio stack code as possible. This allows the following
383operation modes:</p>
384<dl class="docutils">
385<dt>Standalone</dt>
386<dd>Playing back / recording audio data (test tones / .WAV files) in a
387standalone scenario, i.e. no VirtualBox / VMs required). This mode is using
388VirtualBox' audio (mixing) stack and available backend drivers without the
389need of VirtualBox being installed.</dd>
390<dt>Manual</dt>
391<dd>Performing single / multiple tests manually on a local machine.
392Requires a running and set up test VM.</dd>
393<dt>Automated</dt>
394<dd>Performs single / multiple tests via the Validation Kit audio test
395driver and can be triggered via the Validation Kit Test Manager.</dd>
396<dt>(Re-)validation of previously ran tests</dt>
397<dd>This takes two test sets and runs the validation / analysis on them.</dd>
398<dt>Self testing mode</dt>
399<dd>Performs standalone self tests to verify / debug the involved components.</dd>
400</dl>
401</div>
402<div class="section" id="components-and-terminology">
403<h1>Components and Terminology</h1>
404<p>The following components are in charge for performing the audio tests
405(depends on the operation mode, see above):</p>
406<ul>
407<li><p class="first">VBoxAudioTest (also known as VKAT, &quot;Validation Kit Audio Test&quot;):
408A binary which can perform the standalone audio tests mentioned above, as well
409as acting as the guest and host service(s) when performing manual or automated
410tests. It also includes the analysis / verification of audio test sets.
411VKAT also is included in host installations and Guest Additions since
412VirtualBox 7.0 to give customers and end users the opportunity to test and
413verify the audio stack.</p>
414<dl class="docutils">
415<dt>Additional features include:</dt>
416<dd><ul class="first last simple">
417<li>Automatic probing of audio backends (&quot;--probe-backends&quot;)</li>
418<li>Manual playback of test tones (&quot;play -t&quot;)</li>
419<li>Manual playback of .WAV files (&quot;play &lt;WAV-File&gt;&quot;)</li>
420<li>Manual recording to .WAV files (&quot;recording &lt;WAV-File&gt;&quot;)</li>
421<li>Manual device enumeration (sub command &quot;enum&quot;)</li>
422<li>Manual (re-)verification of test sets (sub command &quot;verify&quot;)</li>
423<li>Self-contained self tests (sub command &quot;selftest&quot;)</li>
424</ul>
425</dd>
426</dl>
427<p>See the syntax help (&quot;--help&quot;) for more.</p>
428</li>
429<li><p class="first">ATS (&quot;Audio Testing Service&quot;): Component which is being used by 1 and the
430Validation Kit audio driver (backend) to communicate across guest and host
431boundaries. Currently using a TCP/IP transport layer. Also works with VMs
432which are configured with NAT networking (&quot;reverse connection&quot;).</p>
433</li>
434<li><p class="first">Validation Kit audio test driver (tdAudioTest.py): Used for integrating and
435invoking VKAT for manual and automated tests via the Validation Kit framework
436(Test Manager). Optional. The test driver can be found at <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
437</li>
438<li><p class="first">Validation Kit audio driver (backend): A dedicated audio backend which
439communicates with VKAT running on the same host to perform the actual audio
440tests on a VirtualBox installation. This makes it possible to test the full
441audio stack on a running VM without any additional / external tools.</p>
442<p>On guest playback, data will be recorded, on guest recording, data will be
443injected from the host into the audio stack.</p>
444</li>
445<li><dl class="first docutils">
446<dt>Test sets contain</dt>
447<dd><ul class="first last simple">
448<li>a test manifest with all information required (vkat_manifest.ini)</li>
449<li>the generated / captured audio data (as raw PCM)</li>
450</ul>
451</dd>
452</dl>
453<p>and are either packed as .tar.gz archives or consist of a dedicated directory
454per test set.</p>
455<p>There always must be at least two test sets - one from the host side and one
456from the guest side - to perform a verification.</p>
457<p>Each test set contains a test tag so that matching test sets can be
458identified.</p>
459</li>
460</ul>
461<p>The above components are also included in VirtualBox release builds and can be
462optionally enabled (disabled by default).</p>
463<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
464<colgroup><col class="label" /><col /></colgroup>
465<tbody valign="top">
466<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>src/VBox/ValidationKit/tests/audio/tdAudioTest.py</td></tr>
467</tbody>
468</table>
469</div>
470<div class="section" id="setup-instructions">
471<h1>Setup instructions</h1>
472<ul>
473<li><p class="first">VM needs to be configured to have audio emulation and audio testing enabled
474(via extra-data, set &quot;VBoxInternal2/Audio/Debug/Enabled&quot; to &quot;true&quot;).</p>
475</li>
476<li><p class="first">Audio input / output for the VM needs to be enabled (depending on the test).</p>
477</li>
478<li><p class="first">Start VBoxAudioTest on the guest, for example:</p>
479<p>VBoxAudioTest test --mode guest --tcp-connect-address 10.0.2.2</p>
480<dl class="docutils">
481<dt>Note: VBoxAudioTest is included with the Guest Additions starting at</dt>
482<dd><p class="first last">VirtualBox 7.0.</p>
483</dd>
484<dt>Note: Depending on the VM's networking configuration there might be further</dt>
485<dd><p class="first last">steps necessary in order to be able to reach the host from the guest.
486See the VirtualBox manual for more information.</p>
487</dd>
488</dl>
489</li>
490</ul>
491</div>
492<div class="section" id="performing-a-manual-test">
493<h1>Performing a manual test</h1>
494<ul>
495<li><p class="first">Follow &quot;Setup instructions&quot;.</p>
496</li>
497<li><p class="first">Start VBoxAudioTest on the host with selected test(s), for example:</p>
498<p>VBoxAudioTest test --mode host</p>
499<blockquote>
500<dl class="docutils">
501<dt>Note: VBoxAudioTest is included with the VirtualBox 7.0 host installers and</dt>
502<dd><p class="first last">will be installed by default.</p>
503</dd>
504</dl>
505</blockquote>
506</li>
507<li><p class="first">By default the test verification will be done automatically after running the
508tests.</p>
509</li>
510</ul>
511</div>
512<div class="section" id="advanced-performing-manual-verification">
513<h1>Advanced: Performing manual verification</h1>
514<p>VBoxAudioTest can manually be used with the &quot;verify&quot; sub command in order to
515(re-)verify previously generated test sets. It then will return different exit
516codes based on the verification result.</p>
517</div>
518<div class="section" id="advanced-performing-an-automated-test">
519<h1>Advanced: Performing an automated test</h1>
520<ul class="simple">
521<li>TxS (Test E[x]ecution Service) has to be up and running (part of the
522Validation Kit) on the guest.</li>
523<li>Invoke the tdAudioTest.py test driver, either manually or fully automated
524via Test Manager.</li>
525</ul>
526</div>
527<div class="section" id="internals-workflow-for-a-single-test">
528<h1>Internals: Workflow for a single test</h1>
529<p>When a single test is being executed on a running VM, the following (simplified)
530workflow applies:</p>
531<ul class="simple">
532<li>VKAT on the host connects to VKAT running on the guest (via ATS, also can be a
533remote machine in theory).</li>
534<li>VKAT on the host connects to Validation Kit audio driver on the host
535(via ATS, also can be a remote machine in theory).</li>
536<li><dl class="first docutils">
537<dt>For example, when doing playback tests, VKAT on the host ...</dt>
538<dd><ul class="first last">
539<li><dl class="first docutils">
540<dt>... tells the Validation Kit audio driver to start recording</dt>
541<dd>guest playback.</dd>
542</dl>
543</li>
544<li>... tells the VKAT on the guest to start playing back audio data.</li>
545<li><dl class="first docutils">
546<dt>... gathers all test data (generated from/by the guest and recorded from</dt>
547<dd>the host) as separate test sets.</dd>
548</dl>
549</li>
550<li>... starts verification / analysis of the test sets.</li>
551</ul>
552</dd>
553</dl>
554</li>
555</ul>
556</div>
557<div class="section" id="current-status-limitations">
558<h1>Current status / limitations</h1>
559<ul class="simple">
560<li><dl class="first docutils">
561<dt>The following test types are currently implemented:</dt>
562<dd><ul class="first last">
563<li>Test tone (sine wave) playback from the guest</li>
564<li>Test tone (sine wave) recording by the guest (injected from the host)</li>
565</ul>
566</dd>
567</dl>
568</li>
569<li>Only the HDA device emulation has been verified so far.</li>
570<li>Only the ALSA audio stack on Debian 10 has been verified so far.
571Note: This is different from PulseAudio using the ALSA plugin!</li>
572</ul>
573</div>
574<div class="section" id="troubleshooting">
575<h1>Troubleshooting</h1>
576<ul class="simple">
577<li>Make sure that audio device emulation is enabled and can be used within the
578guest. Also, audio input / output has to be enabled, depending on the tests.</li>
579<li>Make sure that the guest's VBoxAudioTest's instance can reach the host via
580the selected transport layer (TCP/IP by default).</li>
581<li>Increase the hosts audio logging level
582(via extra-data, set &quot;VBoxInternal2/Audio/Debug/Level&quot; to &quot;5&quot;).</li>
583<li>Increase VBoxAudioTest's verbosity level (add &quot;-v&quot;, can be specified
584multiple times).</li>
585<li>Check if the VBox release log contains any warnings / errors with the
586&quot;ValKit:&quot; prefix.</li>
587</ul>
588<table class="docutils field-list" frame="void" rules="none">
589<col class="field-name" />
590<col class="field-body" />
591<tbody valign="top">
592<tr class="field"><th class="field-name">Status:</th><td class="field-body">$Id: VBoxAudioValidationKitReadMe.html 98107 2023-01-17 22:56:50Z vboxsync $</td>
593</tr>
594<tr class="field"><th class="field-name">Copyright:</th><td class="field-body">Copyright (C) 2021-2023 Oracle Corporation.</td>
595</tr>
596</tbody>
597</table>
598</div>
599</div>
600</body>
601</html>
Note: See TracBrowser for help on using the repository browser.

© 2024 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette