Changeset 86595 in vbox

Timestamp:

Oct 16, 2020 7:18:45 AM (4 years ago)

Author:

vboxsync

Message:

Audio/DrvAudio: Leave and re-enter the driver's critical section when calling drvAudioDevicesEnumerateInternal() via drvAudioStreamMaybeReInit() to avoid deadlocks with the IMMNotificationClient implementation callback. bugref:8658

File:

• trunk/src/VBox/Devices/Audio/DrvAudio.cpp (modified) (4 diffs)

trunk/src/VBox/Devices/Audio/DrvAudio.cpp

@@ -1116 +1116 @@
  * Re-initializes the given stream if it is scheduled for this operation.
  *
+ * @note This caller must have entered the critical section of the driver instance,
+ *       needed for the host device (re-)enumeration.
+ *
  * @param   pThis               Pointer to driver instance.
  * @param   pStream             Stream to check and maybe re-initialize.
@@ -1133 +1136 @@
             if (pThis->fEnumerateDevices)
             {
+                /* Make sure to leave the driver's critical section before enumerating host stuff. */
+                int rc2 = RTCritSectLeave(&pThis->CritSect);
+                AssertRC(rc2);
+
                 /* Re-enumerate all host devices. */
                 drvAudioDevicesEnumerateInternal(pThis, true /* fLog */, NULL /* pDevEnum */);
+
+                /* Re-enter the critical section again. */
+                rc2 = RTCritSectEnter(&pThis->CritSect);
+                AssertRC(rc2);
 
                 pThis->fEnumerateDevices = false;
@@ -2097 +2108 @@
  * VERR_NOT_SUPPORTED if not being supported.
  *
+ * @note Must not hold the driver's critical section!
+ *
  * @returns IPRT status code.
  * @param   pThis               Driver instance to be called.
@@ -2104 +2117 @@
 static int drvAudioDevicesEnumerateInternal(PDRVAUDIO pThis, bool fLog, PPDMAUDIODEVICEENUM pDevEnum)
 {
+    AssertReturn(RTCritSectIsOwned(&pThis->CritSect) == false, VERR_WRONG_ORDER);
+
     int rc;