VAD-Gated KWS on RISC-V Part 2 - VAD Structure

This is Part 2 of a series on Voice Activity Detection (VAD) gated Keyword Spotting (KWS) on the ESP32-C3 Lyra. Part 1 provides an overview of the system and the power reduction results achieved. this post examines the VAD in detail.

Four parameters are used simultaneously to create the entire four-feature VAD. The resulting VAD achieves high accuracy as it uses each of the four parameters to prevent noise from being falsely identified as speech.

1. Short-Time Energy (E)

Short-time energy E measures how loud a frame is by squaring each integer ADC sample. The squares are then summed across the frame. The squaring is done to seperate speech from noise in a non-linear manner. A frame with 4× the amplitude of noise yields 16× the energy, sharpening the contrast. Additionally, squaring removes the negative sign from any negative AC values, so that their amplitudes can be treated the same as their positive counterparts.

c

int64_t e = 0; for (int i = 0; i < n; i++) { e += (int64_t)frame[i] * frame[i]; }

This is the classic Short-Time Energy (STE) feature used throughout DSP-based VAD.

2. Zero Crossing Rate (ZCR)

ZCR counts how many times the waveform changes sign within a single frame. It separates broad classes of sound, as spoken vowels produce a moderate ZCR, unvoiced consonants produce a high ZCR, and silence produces a low ZCR. The frequency of a waveform is directly proportional to its ZCR.

c

int32_t zcr = 0; for (int i = 1; i < n; i++) { if ((frame[i] >= 0) != (frame[i-1] >= 0)) zcr++; }

ZCR is ambiguous on its own — noise can oscillate as rapidly as an unvoiced consonant — so it is interpreted alongside energy E. The possible combinations of ‘E’ and ZCR yield the following:

Condition	Interpretation
High ZCR + low E	Background noise
High ZCR + high E	Speech
Low ZCR + low E	Silence

This ZCR/energy combination is the well-established Rabiner & Sambur (1975) endpoint-detection approach.

3. Golay Correlation

For this feature, each frame is correlated against an approximated Golay reference sequence (Golay, 1961). These sequences are built from period-8 alternating blocks {1,1,1,1,-1,-1,-1,-1, ...}. Every audio sample is multiplied by the corresponding reference value. The resulting feature is the absolute value of the sum of these products. It measures how strongly the frame aligns with the reference pattern.

In practice the output separates cleanly into three regions:

Correlation sum	Interpretation
0–10	Silence (all samples near zero)
10–50	Noise (positive and negative samples cancel)
50–300+	Speech (structured signal produces a large sum)

c

// ±1 reference: period-8 alternating blocks { 1,1,1,1,-1,-1,-1,-1, ... } int64_t corr = 0; for (int i = 0; i < n; i++) { corr += (int64_t)frame[i] * golay_ref[i]; } if (corr < 0) corr = -corr; // absolute value

Restricting the reference to ±1 reduces the inner product to solely additions and subtractions. This lack of multiplication makes this feature deterministic and keeps computational costs low. The low compute cost allows for real-time execution on the ESP32-C3.

4. Fast Walsh-Hadamard Transform (FWHT)

The FWHT transforms the recorded audio samples into coefficients using only additions and subtractions. The L1 norm of the coefficients is then used to collapse them into a single value. Its butterfly structure is identical to the FFT except that the weights are restricted to ±1 — so there are no multiplications and no twiddle factors, making it integer-only and well suited to the ESP32-C3.

c

// Butterfly transform — additions only for (int len = 1; len < n; len <<= 1) { for (int i = 0; i < n; i += len << 1) { for (int j = 0; j < len; j++) { int32_t a = buf[i+j], b = buf[i+j+len]; buf[i+j] = a + b; // sum buf[i+j+len] = a - b; // diff } } } // L1 norm of coefficients acc += abs32_sat(buf[i]);

The L1 norm separates into three regions:

L1 norm	Interpretation
Low	Silence (all samples near zero)
Moderate	Noise (random, broadband, energy spread evenly)
High	Speech (structured signal, large coefficients)

The FWHT is an attractive feature for this application because of its low-power, multiplication-free pattern.

To identify speech, the VAD examines the values for the four. A frame is classified as speech if at least two of the four features exceed their respective thresholds.

c

bool vad_decide(const vad_features_t *f) { int score = 0; if (f->h >= THRESH_H) score++; // Hadamard L1 ≥ 10,000 if (f->e >= THRESH_E) score++; // Energy ≥ 200,000 if (f->g >= THRESH_G) score++; // Golay ≥ 10 if (f->zcr >= THRESH_ZCR) score++; // ZCR ≥ 2 return score >= 2; }

This majority-voting scheme provides robustness against false positives. For example, a transient noise spike may push one feature over threshold, but is unlikely to trigger two simultaneously. The thresholds were derived empirically from observed data on the ESP32-C3 Lyra V2.0 with direct microphone input.

Implementation on the ESP32-C3

All four VAD features are computed using integer arithmetic. This removes the high computational cost of floating-point arithmetic. For example, division is replaced throughout with right-shifts. This is present when energy uses a >> 16 to stay within int32 range, and the Golay correlation sum uses a >> 10 before threshold comparison. These operations keep the VAD entirely within the ESP32-C3’s integer pipeline.

Each frame consists of 256 samples taken directly from the microphone input buffer. The four features are computed sequentially per frame:

c

int32_t h = hadamard_compute_feature(g_audio_output_buffer, vad_workbuf, 256, HADAMARD_FEATURE_L1_SUM); int32_t e = vad_feature_energy(g_audio_output_buffer, 256); int32_t zcr = vad_feature_zcr(g_audio_output_buffer, 256); int32_t g_feat = golay_compute_feature(g_audio_output_buffer, 256);

The KWS model requires a 30 KB statically allocated tensor arena. The VAD, by contrast, requires only a 256-element int32 working buffer (~1 KB) for the Hadamard transform, which removes the need for dynamic allocation.

The system operates as a two-state machine: STATE_VAD and STATE_KWS. The VAD runs continuously in STATE_VAD. On a positive speech decision, the system transitions to STATE_KWS and begins running KWS inference. If no keyword is detected within 10 seconds, the system returns to STATE_VAD.

Standalone Power and Compute Cost

CPU utilization was measured on the ESP32-C3 Lyra V2.0 using esp_timer_get_time() to time each inference, reported as a duty-cycle percentage over a 5-second window.

Mode	Avg Inference	CPU Usage
Always-on KWS	~15.29 ms	1.64%
VAD only	~0.80 ms	0.08%

The VAD consumes approximately 20× less CPU per inference and runs at 0.08% CPU — roughly 20× lower duty cycle than always-on KWS at 1.64%.

In a realistic 1-minute usage scenario — 50 seconds of silence (VAD mode) and 10 seconds of active speech (KWS mode) — the VAD-gated system averages 0.34% CPU, compared to 1.64% for always-on KWS. This yields a 4.8× reduction in average CPU utilization, which directly corresponds to a reduction in active power draw over time.

In conclusion, the lightweight, integer-only VAD on the ESP32-C3 Lyra yielded a 4.8× reduction in CPU utilization over always-on KWS. Part 3 benchmarks it against alternative approaches, and Part 4 presents the full KWS on RISC-V results.