Whisper swallows immediate retakes into one word's duration — how to flag them in davinci-resolve-mcp?
On macOS with davinci-resolve-mcp and mlx_whisper large-v3-turbo, when a speaker re-reads a sentence during recording, the transcript outputs the text once and absorbs the pause plus second take into a single word's timestamp (measured at 6.06s for one Chinese character). Silence detection can't catch it because the dead span peaks at -12.1 dB, louder than adjacent speech at -16.7 dB (recall was 3 of 17 instances). Is there a supported way to detect or flag these swallowed retakes?
Originally asked on GitHub: https://github.com/samuelgursky/davinci-resolve-mcp/issues/125
6 answers
Fixed in v2.83.0 (flag + ranking caveat) and v2.93.1 (media-analysis guide section).
The maintainer shipped a
possible_swallowed_retakeflag based on an absolute bar: any word with duration ≥1.2s is flagged. No per-language rate model was added — the English measurement showed the chars/s gate fires on 0 of 50 segments (the swallowed segment ran 8.81 chars/s against a 14.5 median), so the chars/s gate was retired rather than made configurable. The self-calibrating same-word-duration signal was also rejected: it produced 8 function-word false positives and missed the swallow, because content words (the words that absorb retakes) are too rare within a single take to form a distribution.The flag is wired into
rank_takesandplan_transcript_tighten. It reports a flag, never a cut point, and an empty result explicitly notes that no flags is weak evidence, not proof.rank_takesalso emitsRANKING_CAVEAT, stating that a take which stumbled and recovered can outscore one that did not. The media-analysis guide now documents the limitation and why silence detection cannot substitute (−12.1 dB dead span vs −16.7 dB speech; 3 of 17 recalled).Not shipped: the recursive re-transcription probe — three transcription passes per candidate window — the maintainer wants more ground-truth cases before wiring it in. Known structural blind spot: a swallow that smears across several words instead of one is invisible to the 1.2s bar.
Did this work for you?What a GitHub user tried on GitHub (Didn't work)
Silence-based pass using silencedetect-style energy thresholds. Failed because the dead span between takes measures -12.1 dB, louder than adjacent real speech at -16.7 dB; recalled only 3 of 17 ground-truth retakes.
Source: https://github.com/samuelgursky/davinci-resolve-mcp/issues/125 (opens external site)
Did this work for you?What a GitHub user tried on GitHub (Partly worked)
Recursive re-transcription probe: find implausibly slow words (≥2.5s, <4.5 chars/s), re-transcribe the window excluding the first take's span so Whisper cannot align onto it. Against 8 ground-truth cases: 4 exact hits (≤0.22s error), 3 flagged for manual review, 1 miss.
Source: https://github.com/samuelgursky/davinci-resolve-mcp/issues/125 (opens external site)
Did this work for you?What @chenyuxiaojin tried on GitHub (Didn't work)
Chars/s candidate gate (dur ≥2.5s and <4.5 chars/s) tested on English synthetic material with 15 planted retakes. Fired on 0 of 50 segments; the one genuine swallowed segment ran 8.81 chars/s. The gate does not generalize across languages.
Did this work for you?What @chenyuxiaojin tried on GitHub (Partly worked)
Words/s translation of the gate (dur ≥2.5s and <2.5 words/s) on the same English material: fired on exactly 1 of 36 long segments, the swallowed one (2.03 w/s). Recall 1/1, zero false positives, but n=1 — worth re-testing, not trusted as shipped.
Did this work for you?What @chenyuxiaojin tried on GitHub (Didn't work)
Self-calibrating same-word-duration signal (z ≥ 2 against the distribution of that word elsewhere in the take) as proposed by the maintainer. Missed the swallowed word entirely and produced 8 false positives, all function words from prosody variance.
Did this work for you?
Related questions
- scriptapp('Resolve') returns None with DaVinci Resolve 20 on macOS
- Are quit_app/restart_app and save_project in davinci-resolve-mcp dangerous for LLM use?
- davinci-resolve-mcp on Linux: "No valid Resolve scripting module path found" despite correct PYTHONPATH?
- Why does davinci-resolve-mcp report no QuickTime codecs for ProRes LT renders?
- Automate Fusion rectangle masks and title text in davinci-resolve-mcp? Audio conversion is an API gap