VAD-Gated KWS on ESP-32 Part 3: VAD Analysis

This is Part 3 of a four-part series on building a VAD-gated keyword spotting system on the ESP32-C3 Lyra. Part 1 provides an overview of the project, and Part 2 covers the four-feature voice activity detector (VAD) in detail. This post compares the VAD used to the other VAD approaches available for small microcontrollers(MCUs). Part 4 will discuss implementation on RISC-V architecure and summarize the results.

The Landscape

Currently VADs for MCUs today falls into four categories: bare energy thresholds, classical multi-feature detectors, ported statistical classifiers, and tiny neural networks. This project uses a multi-feature detector, which combines the Hadamard L1, energy, ZCR, and Golay features.

Energy Thresholding

The default VAD common in Arduino sketches, as well as the new ESP32-P4, implements its low-power hardware VAD using an energy threshold plus a passband-energy check. These energy—focused VADs are used for their low computational cost. They are cost efficient because they compute only one multiplication per frame, and don’t use any FFTs or AI inference. However, these VADs are lacking in accuracy, as they are susceptible to activation by high-energy background noise.

Classical Multi-Feature Detectors

To increase the accuracy of lightweight models, many VADs use low-power detectors in parallel. The VAD used in this project is one such example, as the VAD classifies a signal as speech if two of four detection thresholds are met simultaneously. This VAD is effective as it keeps compute costs low while achieving a high accuracy. It does this by using features that can operate solely on integer arithmetic, with no need for floating-point multiplication or division. On the C3’s RISC-V core, this keeps the VAD’s power consumption low enough to run as always-on.

WebRTC VAD: The Ported Standard

The most widely ported VAD on small systems is the WebRTC VAD, originally from Google’s real-time communication stack. It splits the signal into six sub-bands between 80 Hz and 4 kHz and classifies their energies with a Gaussian mixture model. This VAD operates smoothly on a microcontroller as it runs in fixed-point, and the frame buffers stay under 3 KB.

The main shortcoming of WebRTC, however, is false positives in noise. The cost of mistaking noise for speech is much greater in a power-gated KWS pipeline than it was in WebRTC’s original application of phone calls. This is because in our application every false trigger wakes the keyword spotter awhich in turn uses much more power. Espressif now positions its own neural VADNet as a replacement for WebRTC VAD for this reason.

Tiny Neural VADs

Most of the current VAD research is in this class of AI-based detectors, with a focus on reducing the models’ size. Many of these detectors are computationally heavy because they are spectrogram-based: each one first computes a spectrogram — requiring an FFT and a mel filterbank — before running neural network inference. Some examples of tiny neural VADs include:

• MarbleNet-class CNNs: (~88k parameters) achieve high accuracy, but expect a mel-spectrogram front end and tens of kilobytes of weights
• NAS-VAD: use neural architecture search to reach state-of-the-art accuracy at 151k parameters, however it is costly and spectrogram-based
• AtomicVAD: (2025) an end-to-end model that drops to roughly 0.3k parameters, a 99.7% cut compared to MarbleNet, with feature extraction learned inside the network rather than a separate spectrogram stage
• Espressif VADNet: trained on ~15,000 hours of speech, ships inside the ESP-SR audio front end, and achieves better noise rejection than WebRTC VAD.

While these models are accurate, they do not fit the ESP32-C3 Lyra cleanly. ESP-SR’s neural models assume a chip with PSRAM and SIMD, and Espressif had to ship a cut-down wake model (WakeNet9s) to make the family compatible with the C3. The four-feature VAD used in this project avoid the FFT and mel-filterbank computations that these AI-based detectors depends on, which is what keeps it light enough to run always-on.

The strongest alternative to our approach is AtomicVAD, an AI-based end-to-end neural network of roughly 0.3k parameters that learns its own feature extraction. It is similar to the 4-feature VAD in that it skips the spectrogram front end and runs on a bare microcontroller.

A major Drawback to AtomicVAD however, is that it requires a much larger sample size than our four-parameter VAD. AtomicVAD requires 10080 samples (630ms) of input per decision, while the 4-feature VAD requires only 256 samples(16ms). This is because AtomicVAD is a learned model that needs a training pipeline and a labelled dataset. Its accuracy is entirely based on the data it receives. Alternatively, our VAD is a deterministic four-feature majority vote. It involves no model training and instead uses pure integer arithmetic alonside manual tuning to known thresholds.

Both options are suitable for this application. The four-feature design was chosen because it is transparent and auditable. Every decision made can be traced to a specific feature and threshold rather than to learned weights, which make it the better fit for studying how a VAD actually behaves on RISC-V hardware.

Overall, we use the 4-feature VAD because it is fully transparent, training-free, and runs in pure integer arithmetic. In Part 4, we will assemble the complete system, gating the keyword spotter with this VAD on RISC-V and reporting full accuracy and CPU results.