VAD-Gated KWS on RISC-V Part 4: Full System Results

Introduction

This is Part 4 of a series on VAD-gated Keyword Spotting on the ESP32-C3 Lyra. Part 1 introduced the system architecture, Part 2 detailed the four-parameter VAD, and Part 3 compared it to other VADs

System Architecture

Audio is captured via the ESP32-C3’s ADC1 CH0 and passed to the hybrid VAD, which extracts a four-feature vector. The four features are Energy, ZCR, Golay, and Hadamard. This four-feature vector is then used as input for a TFLite Micro neural network (4→8→1). When the NN classifies the frame as SPEECH, the KWS engine wakes and runs TensorFlow Lite for Microcontrollers inference, returning a Yes / No / Unknown response. During SILENCE, the KWS engine remains idle.

KWS Neural Network

The VAD gate is a fully integer, three-layer feedforward network trained on 994 labeled frames collected directly from the Lyra V2.0 microphone. The minimal architecture consists of four inputs (one per feature), a single hidden layer of eight neurons, and a binary output.

Parameter	Value
Architecture	4 → 8 → 1
Training frames	994 labeled
Model size	1,868 bytes
Runtime	TFLite Micro
Quantization	Integer-only

The four features fed to the network span complementary signal dimensions. The amplitude-domain structure is measured by Energy and the Golay feature, oscillation rate is captured by ZCR, and the FWHT-derived Hadamard L1 norm measures spectral complexity. Their observed ranges on the Lyra V2.0 are summarized below.

Feature	Silence	Speech
Hadamard (h)	5,800 – 9,800	10,000 – 176,000+
Energy (e)	63,000 – 150,000	83,000 – 2,200,000+
ZCR	0	0 – 73
Golay (g)	0 – 9	0 – 229

The separation between the silence and speech ranges confirms that each feature contributes an effective discriminative signal prior to the NN classification stage.

KWS Accuracy

The network achieved 96.98% test accuracy on the 994-frame labeled dataset, with inference running entirely on-device via TFLite Micro. At 1,868 bytes, the model fits comfortably within the ESP32-C3’s SRAM budget alongside the VAD feature extraction buffers.

Compute Performance

CPU utilization was measured on the ESP32-C3 Lyra V2.0 using esp_timer_get_time() to time each inference, reported as a duty-cycle percentage over a 5-second window.

Mode	Avg Inference	CPU Usage
Always-on KWS	~15.29 ms	1.64%
VAD only	~0.80 ms	0.08%

The VAD gate consumes approximately 20× less CPU per inference than the KWS engine and runs at 0.08% duty cycle during silence, compared to 1.64% for always-on KWS.

Full System Performance

The VAD-gated system operates as a two-state machine, with VAD mode during silence, and KWS mode upon speech detection. KWS mode has a 10-second inactivity timeout before returning to VAD. In a representative 1-minute scenario — 50 seconds of silence and 10 seconds of active speech — the weighted average CPU utilization is:

Terminal

(50 × 0.08% + 10 × 1.64%) / 60 = 0.34% average CPU

This yields a 4.8× reduction in average CPU utilization relative to always-on KWS at 1.64%, corresponding directly to a reduction in active power draw over time.

Future Directions

The pipeline demonstrates that an open, edge-side VAD layer can be inserted before cloud-dependent speech recognition with minimal resource cost. Existing voice assistants rely on proprietary cloud pipelines with no support for on-device voice authentication or arbitrary remote device access. The ESP32-C3 implementation establishes a local wake-word stage that reduces both latency and cloud dependency. It also serves as a foundation for voice-authenticated remote command execution. A system of this type could include locally run VAD and wake-word detection, as well as authenticated commands routed to the cloud and executed on a remote device.

Conclusion

The VAD-gated KWS pipeline on the ESP32-C3 Lyra V2.0 demonstrates that speech-aware inference is achievable at near-zero compute cost during silence. A 1,868-byte TFLite Micro model classifies four integer features with 96.98% accuracy, gating the KWS engine to an average of 0.34% CPU — a 4.8× reduction over always-on operation. The result confirms the viability of multi-stage, integer-only speech pipelines on RISC-V class microcontrollers without floating-point hardware or DSP extensions.

References

1. J. L. Walsh, “A closed set of normal orthogonal functions,” American Journal of Mathematics, vol. 45, no. 1, pp. 5–24, Jan. 1923.
2. J. Hadamard, “Résolution d’une question relative aux déterminants,” Bulletin des Sciences Mathématiques, ser. 2, vol. 17, pp. 240–246, 1893.
3. M. J. E. Golay, “Complementary series,” IRE Transactions on Information Theory, vol. 7, no. 2, pp. 82–87, Apr. 1961.
4. L. R. Rabiner and M. R. Sambur, “An algorithm for determining the endpoints of isolated utterances,” The Bell System Technical Journal, vol. 54, no. 2, pp. 297–315, Feb. 1975.

VAD-Gated KWS on ESP-32 Part 3: VAD Analysis

This is Part 3 of a four-part series on building a VAD-gated keyword spotting system on the ESP32-C3 Lyra. Part 1 provides an overview of the project, and Part 2 covers the four-feature voice activity detector (VAD) in detail. This post compares the VAD used to the other VAD approaches available for small microcontrollers(MCUs). Part 4 will discuss implementation on RISC-V architecure and summarize the results.

The Landscape

Currently VADs for MCUs today falls into four categories: bare energy thresholds, classical multi-feature detectors, ported statistical classifiers, and tiny neural networks. This project uses a multi-feature detector, which combines the Hadamard L1, energy, ZCR, and Golay features.

Energy Thresholding

The default VAD common in Arduino sketches, as well as the new ESP32-P4, implements its low-power hardware VAD using an energy threshold plus a passband-energy check. These energy—focused VADs are used for their low computational cost. They are cost efficient because they compute only one multiplication per frame, and don’t use any FFTs or AI inference. However, these VADs are lacking in accuracy, as they are susceptible to activation by high-energy background noise.

Classical Multi-Feature Detectors

To increase the accuracy of lightweight models, many VADs use low-power detectors in parallel. The VAD used in this project is one such example, as the VAD classifies a signal as speech if two of four detection thresholds are met simultaneously. This VAD is effective as it keeps compute costs low while achieving a high accuracy. It does this by using features that can operate solely on integer arithmetic, with no need for floating-point multiplication or division. On the C3’s RISC-V core, this keeps the VAD’s power consumption low enough to run as always-on.

WebRTC VAD: The Ported Standard

The most widely ported VAD on small systems is the WebRTC VAD, originally from Google’s real-time communication stack. It splits the signal into six sub-bands between 80 Hz and 4 kHz and classifies their energies with a Gaussian mixture model. This VAD operates smoothly on a microcontroller as it runs in fixed-point, and the frame buffers stay under 3 KB.

The main shortcoming of WebRTC, however, is false positives in noise. The cost of mistaking noise for speech is much greater in a power-gated KWS pipeline than it was in WebRTC’s original application of phone calls. This is because in our application every false trigger wakes the keyword spotter awhich in turn uses much more power. Espressif now positions its own neural VADNet as a replacement for WebRTC VAD for this reason.

Tiny Neural VADs

Most of the current VAD research is in this class of AI-based detectors, with a focus on reducing the models’ size. Many of these detectors are computationally heavy because they are spectrogram-based: each one first computes a spectrogram — requiring an FFT and a mel filterbank — before running neural network inference. Some examples of tiny neural VADs include:

• MarbleNet-class CNNs: (~88k parameters) achieve high accuracy, but expect a mel-spectrogram front end and tens of kilobytes of weights
• NAS-VAD: use neural architecture search to reach state-of-the-art accuracy at 151k parameters, however it is costly and spectrogram-based
• AtomicVAD: (2025) an end-to-end model that drops to roughly 0.3k parameters, a 99.7% cut compared to MarbleNet, with feature extraction learned inside the network rather than a separate spectrogram stage
• Espressif VADNet: trained on ~15,000 hours of speech, ships inside the ESP-SR audio front end, and achieves better noise rejection than WebRTC VAD.

While these models are accurate, they do not fit the ESP32-C3 Lyra cleanly. ESP-SR’s neural models assume a chip with PSRAM and SIMD, and Espressif had to ship a cut-down wake model (WakeNet9s) to make the family compatible with the C3. The four-feature VAD used in this project avoid the FFT and mel-filterbank computations that these AI-based detectors depends on, which is what keeps it light enough to run always-on.

The strongest alternative to our approach is AtomicVAD, an AI-based end-to-end neural network of roughly 0.3k parameters that learns its own feature extraction. It is similar to the 4-feature VAD in that it skips the spectrogram front end and runs on a bare microcontroller.

A major Drawback to AtomicVAD however, is that it requires a much larger sample size than our four-parameter VAD. AtomicVAD requires 10080 samples (630ms) of input per decision, while the 4-feature VAD requires only 256 samples(16ms). This is because AtomicVAD is a learned model that needs a training pipeline and a labelled dataset. Its accuracy is entirely based on the data it receives. Alternatively, our VAD is a deterministic four-feature majority vote. It involves no model training and instead uses pure integer arithmetic alonside manual tuning to known thresholds.

Both options are suitable for this application. The four-feature design was chosen because it is transparent and auditable. Every decision made can be traced to a specific feature and threshold rather than to learned weights, which make it the better fit for studying how a VAD actually behaves on RISC-V hardware.

Overall, we use the 4-feature VAD because it is fully transparent, training-free, and runs in pure integer arithmetic. In Part 4, we will assemble the complete system, gating the keyword spotter with this VAD on RISC-V and reporting full accuracy and CPU results.

VAD-Gated KWS on RISC-V Part 2 - VAD Structure

This is Part 2 of a series on Voice Activity Detection (VAD) gated Keyword Spotting (KWS) on the ESP32-C3 Lyra. Part 1 provides an overview of the system and the power reduction results achieved. this post examines the VAD in detail.

Four parameters are used simultaneously to create the entire four-feature VAD. The resulting VAD achieves high accuracy as it uses each of the four parameters to prevent noise from being falsely identified as speech.

1. Short-Time Energy (E)

Short-time energy E measures how loud a frame is by squaring each integer ADC sample. The squares are then summed across the frame. The squaring is done to seperate speech from noise in a non-linear manner. A frame with 4× the amplitude of noise yields 16× the energy, sharpening the contrast. Additionally, squaring removes the negative sign from any negative AC values, so that their amplitudes can be treated the same as their positive counterparts.

c

int64_t e = 0; for (int i = 0; i < n; i++) { e += (int64_t)frame[i] * frame[i]; }

This is the classic Short-Time Energy (STE) feature used throughout DSP-based VAD.

2. Zero Crossing Rate (ZCR)

ZCR counts how many times the waveform changes sign within a single frame. It separates broad classes of sound, as spoken vowels produce a moderate ZCR, unvoiced consonants produce a high ZCR, and silence produces a low ZCR. The frequency of a waveform is directly proportional to its ZCR.

c

int32_t zcr = 0; for (int i = 1; i < n; i++) { if ((frame[i] >= 0) != (frame[i-1] >= 0)) zcr++; }

ZCR is ambiguous on its own — noise can oscillate as rapidly as an unvoiced consonant — so it is interpreted alongside energy E. The possible combinations of ‘E’ and ZCR yield the following:

Condition	Interpretation
High ZCR + low E	Background noise
High ZCR + high E	Speech
Low ZCR + low E	Silence

This ZCR/energy combination is the well-established Rabiner & Sambur (1975) endpoint-detection approach.

3. Golay Correlation

For this feature, each frame is correlated against an approximated Golay reference sequence (Golay, 1961). These sequences are built from period-8 alternating blocks {1,1,1,1,-1,-1,-1,-1, ...}. Every audio sample is multiplied by the corresponding reference value. The resulting feature is the absolute value of the sum of these products. It measures how strongly the frame aligns with the reference pattern.

In practice the output separates cleanly into three regions:

Correlation sum	Interpretation
0–10	Silence (all samples near zero)
10–50	Noise (positive and negative samples cancel)
50–300+	Speech (structured signal produces a large sum)

c

// ±1 reference: period-8 alternating blocks { 1,1,1,1,-1,-1,-1,-1, ... } int64_t corr = 0; for (int i = 0; i < n; i++) { corr += (int64_t)frame[i] * golay_ref[i]; } if (corr < 0) corr = -corr; // absolute value

Restricting the reference to ±1 reduces the inner product to solely additions and subtractions. This lack of multiplication makes this feature deterministic and keeps computational costs low. The low compute cost allows for real-time execution on the ESP32-C3.

4. Fast Walsh-Hadamard Transform (FWHT)

The FWHT transforms the recorded audio samples into coefficients using only additions and subtractions. The L1 norm of the coefficients is then used to collapse them into a single value. Its butterfly structure is identical to the FFT except that the weights are restricted to ±1 — so there are no multiplications and no twiddle factors, making it integer-only and well suited to the ESP32-C3.

c

// Butterfly transform — additions only for (int len = 1; len < n; len <<= 1) { for (int i = 0; i < n; i += len << 1) { for (int j = 0; j < len; j++) { int32_t a = buf[i+j], b = buf[i+j+len]; buf[i+j] = a + b; // sum buf[i+j+len] = a - b; // diff } } } // L1 norm of coefficients acc += abs32_sat(buf[i]);

The L1 norm separates into three regions:

L1 norm	Interpretation
Low	Silence (all samples near zero)
Moderate	Noise (random, broadband, energy spread evenly)
High	Speech (structured signal, large coefficients)

The FWHT is an attractive feature for this application because of its low-power, multiplication-free pattern.

To identify speech, the VAD examines the values for the four. A frame is classified as speech if at least two of the four features exceed their respective thresholds.

c

bool vad_decide(const vad_features_t *f) { int score = 0; if (f->h >= THRESH_H) score++; // Hadamard L1 ≥ 10,000 if (f->e >= THRESH_E) score++; // Energy ≥ 200,000 if (f->g >= THRESH_G) score++; // Golay ≥ 10 if (f->zcr >= THRESH_ZCR) score++; // ZCR ≥ 2 return score >= 2; }

This majority-voting scheme provides robustness against false positives. For example, a transient noise spike may push one feature over threshold, but is unlikely to trigger two simultaneously. The thresholds were derived empirically from observed data on the ESP32-C3 Lyra V2.0 with direct microphone input.

Implementation on the ESP32-C3

All four VAD features are computed using integer arithmetic. This removes the high computational cost of floating-point arithmetic. For example, division is replaced throughout with right-shifts. This is present when energy uses a >> 16 to stay within int32 range, and the Golay correlation sum uses a >> 10 before threshold comparison. These operations keep the VAD entirely within the ESP32-C3’s integer pipeline.

Each frame consists of 256 samples taken directly from the microphone input buffer. The four features are computed sequentially per frame:

c

int32_t h = hadamard_compute_feature(g_audio_output_buffer, vad_workbuf, 256, HADAMARD_FEATURE_L1_SUM); int32_t e = vad_feature_energy(g_audio_output_buffer, 256); int32_t zcr = vad_feature_zcr(g_audio_output_buffer, 256); int32_t g_feat = golay_compute_feature(g_audio_output_buffer, 256);

The KWS model requires a 30 KB statically allocated tensor arena. The VAD, by contrast, requires only a 256-element int32 working buffer (~1 KB) for the Hadamard transform, which removes the need for dynamic allocation.

The system operates as a two-state machine: STATE_VAD and STATE_KWS. The VAD runs continuously in STATE_VAD. On a positive speech decision, the system transitions to STATE_KWS and begins running KWS inference. If no keyword is detected within 10 seconds, the system returns to STATE_VAD.

Standalone Power and Compute Cost

CPU utilization was measured on the ESP32-C3 Lyra V2.0 using esp_timer_get_time() to time each inference, reported as a duty-cycle percentage over a 5-second window.

Mode	Avg Inference	CPU Usage
Always-on KWS	~15.29 ms	1.64%
VAD only	~0.80 ms	0.08%

The VAD consumes approximately 20× less CPU per inference and runs at 0.08% CPU — roughly 20× lower duty cycle than always-on KWS at 1.64%.

In a realistic 1-minute usage scenario — 50 seconds of silence (VAD mode) and 10 seconds of active speech (KWS mode) — the VAD-gated system averages 0.34% CPU, compared to 1.64% for always-on KWS. This yields a 4.8× reduction in average CPU utilization, which directly corresponds to a reduction in active power draw over time.

In conclusion, the lightweight, integer-only VAD on the ESP32-C3 Lyra yielded a 4.8× reduction in CPU utilization over always-on KWS. Part 3 benchmarks it against alternative approaches, and Part 4 presents the full KWS on RISC-V results.

VAD KWS on RISC-V Part 1: Overview and Executive Summary

This post is Part 1 of 4 in a series covering Voice Activity Detection (VAD) gated Keyword Spotting (KWS) on the ESP32-C3 Lyra, a RISC-V based microcontroller. Precursor work solely focused on KWS on the Lyra can be viewed at Keyword Spotting on ESP32-C3-Lyra V2 Using ESP-IDF.

This is the subject of a project I completed for two of my M.S. courses, Digital Signal Processing and AI-accelerated computing, both taught by Dr. Adly Fam. The goal of this project is to reduce the power consumption of always-on KWS by gating the KWS model behind a lightweight VAD. Rather than running the full KWS model continuously, the VAD listens first — only waking the KWS model when speech is detected.

The VAD: Four-Parameter Approach

The VAD used in this project is based on four acoustic parameters:

1. Energy
2. Zero Crossing Rate (ZCR)
3. The Golay Feature
4. The Fast Walsh Hadamard Transform (FWHT)

Power Consumption Results

By running the VAD instead of the full KWS model during silence, the system achieves approximately a 4.8× reduction in average power consumption over time on the ESP32-C3.

This concludes part 1. Part 2 will explore the VAD's structure, Part 3 will discuss other existing methods for VAD, and Part 4 will summarize the results from testing the full system.

Keyword Spotting on ESP32-C3-Lyra V2 Using ESP-IDF

ESP32-C3 Keyword Spotting (“Yes/No”) with TFLite Micro micro_speech using the Onboard Mic (ADC)

This post documents how to run TensorFlow Lite for Microcontrollers (TFLM) (now branded as LiteRT for Microcontrollers) keyword spotting example (micro_speech) on an ESP32-C3 board, and how to adapt the example to use the onboard analog microphone routed through the ESP32-C3 ADC. To set up the ESP32-C3-lyra V2, see this post: Hello World on ESP-32-C3-Lyra V2.0

Environment / Versions

• Target board: ESP32-C3 (ESP32-C3-Lyra)
• ESP-IDF version: v6.x (or v6.0-dev)
• Example project: esp-tflite-micro:micro_speech (keyword spotting “yes/no”)

Note (ESP-IDF v6+): ESP-IDF v6 removed the legacy ADC header (driver/adc.h) and renamed some ADC attenuation enums. The ADC code below reflects those v6+ changes.

Command Log

1) Load the ESP-IDF environment

. $HOME/esp/esp-idf/export.sh

2) Create a new project from the micro_speech example

cd ~
idf.py create-project-from-example "espressif/esp-tflite-micro=1.3.0:micro_speech"
mv ~/micro_speech ~/keyword_spotting_tflm

3) Set the target to ESP32-C3

cd ~/keyword_spotting_tflm && idf.py set-target esp32c3

4) Install/verify ESP-IDF tools for ESP32-C3 (v6+ toolchain)

python3 $IDF_PATH/tools/idf_tools.py install --targets esp32c3
. $HOME/esp/esp-idf/export.sh

5) Build

cd ~/keyword_spotting_tflm && idf.py build

6) Flash and open the serial monitor

cd ~/keyword_spotting_tflm && idf.py -p /dev/ttyUSB0 flash monitor

Exit the monitor: Press Ctrl + ]

Troubleshooting

Issue: Default example tried I2S and failed

The upstream micro_speech project’s audio capture path attempted to configure I2S pins (I2S microphone). On this setup, we used the onboard mic through ADC instead. The following error occurs:

E (...) i2s_set_pin(...): bck_io_num invalid
E (...) TF_LITE_AUDIO_PROVIDER: Error in i2s_set_pin

Fix: Switch audio capture from I2S to ADC continuous sampling

Key points of the ADC implementation:

• Use esp_adc/adc_continuous.h continuous mode to sample at 16 kHz.
• Convert 12-bit unsigned ADC samples into signed 16-bit PCM-like samples centered around mid-scale.
• Write samples into the existing ring buffer so the model’s GetAudioSamples() continues to work.

Replace audio_provider.cc with the working ADC version (ESP32-C3 TYPE2) by replacing the entire contents of:

~/keyword_spotting_tflm/main/audio_provider.cc

with the following code:

/* ADC-based audio provider for ESP32-C3-Lyra (MIC_ADC on IO0 / ADC1 CH0) */

#include "audio_provider.h"

#include <cstring>

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

#include "esp_log.h"

#include "esp_adc/adc_continuous.h"

#include "ringbuf.h"
#include "micro_model_settings.h"

static const char* TAG = "TF_LITE_AUDIO_PROVIDER";

ringbuf_t* g_audio_capture_buffer;
volatile int32_t g_latest_audio_timestamp = 0;

constexpr int32_t history_samples_to_keep =
    ((kFeatureDurationMs - kFeatureStrideMs) * (kAudioSampleFrequency / 1000));
constexpr int32_t new_samples_to_get =
    (kFeatureStrideMs * (kAudioSampleFrequency / 1000));

const int32_t kAudioCaptureBufferSize = 40000;

namespace {
int16_t g_audio_output_buffer[kMaxAudioSampleSize * 32];
bool g_is_audio_initialized = false;
int16_t g_history_buffer[history_samples_to_keep];

adc_continuous_handle_t g_adc_handle = NULL;

// Read buffer (raw ADC frames)
static constexpr size_t kAdcReadBytes = 1024;
uint8_t g_adc_read_buf[kAdcReadBytes];

// Temporary PCM buffer (int16)
int16_t g_pcm_buf[kAdcReadBytes / sizeof(adc_digi_output_data_t)];

// ESP32-C3-Lyra MIC_ADC is routed to IO0 => ADC1 channel 0
static constexpr adc_unit_t kAdcUnit = ADC_UNIT_1;
static constexpr adc_channel_t kAdcChannel = ADC_CHANNEL_0;
static constexpr adc_atten_t kAdcAtten = ADC_ATTEN_DB_12;
static constexpr adc_bitwidth_t kAdcBitwidth = ADC_BITWIDTH_12;
}  // namespace

static void adc_init_continuous() {
  adc_continuous_handle_cfg_t handle_cfg = {
      .max_store_buf_size = 4096,
      .conv_frame_size = 1024,
  };
  ESP_ERROR_CHECK(adc_continuous_new_handle(&handle_cfg, &g_adc_handle));

  adc_digi_pattern_config_t pattern = {};
  pattern.atten = kAdcAtten;
  pattern.channel = kAdcChannel;
  pattern.unit = kAdcUnit;
  pattern.bit_width = kAdcBitwidth;

  adc_continuous_config_t dig_cfg = {};
  dig_cfg.sample_freq_hz = kAudioSampleFrequency;  // 16 kHz
  dig_cfg.conv_mode = ADC_CONV_SINGLE_UNIT_1;

  // ESP32-C3 DMA output uses TYPE2 layout
  dig_cfg.format = ADC_DIGI_OUTPUT_FORMAT_TYPE2;

  dig_cfg.pattern_num = 1;
  dig_cfg.adc_pattern = &pattern;

  ESP_ERROR_CHECK(adc_continuous_config(g_adc_handle, &dig_cfg));
  ESP_ERROR_CHECK(adc_continuous_start(g_adc_handle));
}

static inline int16_t adc12_to_pcm16(uint16_t adc12) {
  int32_t centered = (int32_t)adc12 - 2048;
  int32_t pcm = centered << 4;  // scale 12-bit to ~16-bit
  if (pcm > 32767) pcm = 32767;
  if (pcm < -32768) pcm = -32768;
  return (int16_t)pcm;
}

static void CaptureSamples(void* arg) {
  adc_init_continuous();

  while (true) {
    uint32_t out_bytes = 0;
    esp_err_t ret = adc_continuous_read(
        g_adc_handle, g_adc_read_buf, kAdcReadBytes, &out_bytes, pdMS_TO_TICKS(200));

    if (ret == ESP_OK && out_bytes > 0) {
      const size_t n_frames = out_bytes / sizeof(adc_digi_output_data_t);

      for (size_t i = 0; i < n_frames; i++) {
        const adc_digi_output_data_t* p =
            (const adc_digi_output_data_t*)(g_adc_read_buf +
                                            i * sizeof(adc_digi_output_data_t));

        // ESP32-C3 uses type2 layout (type1 will not compile)
        uint16_t raw = (uint16_t)(p->type2.data);

        g_pcm_buf[i] = adc12_to_pcm16(raw);
      }

      const int bytes_to_write = (int)(n_frames * sizeof(int16_t));
      const int bytes_written = rb_write(g_audio_capture_buffer,
                                         (uint8_t*)g_pcm_buf,
                                         bytes_to_write,
                                         pdMS_TO_TICKS(200));

      if (bytes_written > 0) {
        const int samples_written = bytes_written / (int)sizeof(int16_t);
        g_latest_audio_timestamp += (1000 * samples_written) / kAudioSampleFrequency;
      }
    }

    if (ret != ESP_OK && ret != ESP_ERR_TIMEOUT) {
      ESP_LOGE(TAG, "adc_continuous_read failed: %s", esp_err_to_name(ret));
      vTaskDelay(pdMS_TO_TICKS(50));
    }
  }
}

TfLiteStatus InitAudioRecording() {
  g_audio_capture_buffer = rb_init("tf_ringbuffer", kAudioCaptureBufferSize);
  if (!g_audio_capture_buffer) {
    ESP_LOGE(TAG, "Error creating ring buffer");
    return kTfLiteError;
  }

  xTaskCreate(CaptureSamples, "CaptureSamples", 1024 * 4, NULL, 10, NULL);

  while (!g_latest_audio_timestamp) {
    vTaskDelay(1);
  }

  ESP_LOGI(TAG, "Audio Recording started (ADC continuous)");
  return kTfLiteOk;
}

TfLiteStatus GetAudioSamples1(int* audio_samples_size, int16_t** audio_samples) {
  if (!g_is_audio_initialized) {
    TfLiteStatus init_status = InitAudioRecording();
    if (init_status != kTfLiteOk) {
      return init_status;
    }
    g_is_audio_initialized = true;
  }

  int bytes_read =
      rb_read(g_audio_capture_buffer, (uint8_t*)(g_audio_output_buffer), 16000, 1000);
  if (bytes_read < 0) {
    ESP_LOGI(TAG, "Couldn't read data in time");
    bytes_read = 0;
  }
  *audio_samples_size = bytes_read;
  *audio_samples = g_audio_output_buffer;
  return kTfLiteOk;
}

TfLiteStatus GetAudioSamples(int start_ms, int duration_ms,
                             int* audio_samples_size, int16_t** audio_samples) {
  if (!g_is_audio_initialized) {
    TfLiteStatus init_status = InitAudioRecording();
    if (init_status != kTfLiteOk) {
      return init_status;
    }
    g_is_audio_initialized = true;
  }

  memcpy((void*)(g_audio_output_buffer), (void*)(g_history_buffer),
         history_samples_to_keep * sizeof(int16_t));

  int bytes_read =
      rb_read(g_audio_capture_buffer,
              ((uint8_t*)(g_audio_output_buffer + history_samples_to_keep)),
              new_samples_to_get * sizeof(int16_t), pdMS_TO_TICKS(200));

  if (bytes_read < 0) {
    ESP_LOGE(TAG, "Model could not read data from Ring Buffer");
  }

  memcpy((void*)(g_history_buffer),
         (void*)(g_audio_output_buffer + new_samples_to_get),
         history_samples_to_keep * sizeof(int16_t));

  *audio_samples_size = kMaxAudioSampleSize;
  *audio_samples = g_audio_output_buffer;
  return kTfLiteOk;
}

int32_t LatestAudioTimestamp() { return g_latest_audio_timestamp; }

Issue: Missing header esp_adc/adc_continuous.h

After adding the include, the build failed with:

fatal error: esp_adc/adc_continuous.h: No such file or directory

Fix: Add the esp_adc component dependency

Edit main/CMakeLists.txt to include esp_adc to PRIV_REQUIRES (or REQUIRES):

nano ~/keyword_spotting_tflm/main/CMakeLists.txt

idf_component_register(
  SRCS ...
  INCLUDE_DIRS .
  PRIV_REQUIRES esp_adc
)

Issue: adc_digi_output_data_t had no type1 on ESP32-C3

Build error:

error: 'const struct adc_digi_output_data_t' has no member named 'type1'

Fix: Use the ESP32-C3 struct layout (TYPE2)

Make the following changes in the file keyword_spotting_tflm/main/audio_provider.cc:

• ADC_DIGI_OUTPUT_FORMAT_TYPE1 → ADC_DIGI_OUTPUT_FORMAT_TYPE2
• p->type1.data → p->type2.data

Next, rebuild and reflash:

cd ~/keyword_spotting_tflm && idf.py build
cd ~/keyword_spotting_tflm && idf.py -p /dev/ttyUSB0 flash monitor

Issue: Toolchain version mismatch on ESP-IDF v6+

If the build fails with a toolchain mismatch (e.g., expected esp-15.2.0_20250929), install the ESP32-C3 toolchain:

python3 $IDF_PATH/tools/idf_tools.py install --targets esp32c3
. $HOME/esp/esp-idf/export.sh

Issue: idf.py fullclean refuses

If idf.py fullclean refuses to delete the build directory, delete it manually:

cd ~/keyword_spotting_tflm
rm -rf build
idf.py build

After switching the audio provider to ADC and aligning the ADC DMA output format for ESP32-C3, the application ran successfully and recognized the keywords “yes” and “no” over serial output. The next post will include customization for keyword spotting with additional words.

Hello World on ESP32-C3-Lyra V2.0 Using ESP-IDF

Flashing “Hello World” to an ESP32-C3-Lyra V2.0 on Linux (ESP-IDF)

This post serves as a tutorial for how to build and flash the official ESP-IDF hello_world example to an ESP32-C3-Lyra-V2.0 from a Linux machine, then verify output over UART. This tutorial works for any ESP32-C3. Additionally, this is a precursor to the next post in this series: Keyword Spotting using ESP32-C3-Lyra V2.0

prerequisites:

Hardware

• ESP32-C3 development board
• USB cable
• USB-to-UART bridge presented as CP2102N (common on many ESP32 boards)

Software

• Ubuntu 22.04
• ESP-IDF v5.2.3 (installed from source)

Step 1: Verify the board appears as a serial device

Plug the board in over USB and check the kernel log:

sudo dmesg -T | tail -n 30

You should see something indicating a USB-to-UART bridge and the assigned device node, for example:

• CP2102N USB to UART Bridge Controller
• ... now attached to ttyUSB0

That means your UART port is likely /dev/ttyUSB0.

Exit the screen using Ctrl+A then K then y

Step 2: Install required packages

Install the typical ESP-IDF build dependencies:

sudo apt update && sudo apt install -y git python3 python3-venv python3-pip cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0

Step 3: Install ESP-IDF (v5.2.3) + ESP32-C3 toolchain

Clone ESP-IDF and install only what you need for ESP32-C3:

cd ~ && git clone -b v5.2.3 --recursive https://github.com/espressif/esp-idf.git && cd esp-idf && ./install.sh esp32c3

Then load the environment into your current terminal session:

. ./export.sh

From this point, idf.py should be available.

Step 4: Create a local hello_world project

Copy the example out of the ESP-IDF tree (so you can edit safely later):

cd ~ && cp -r ~/esp-idf/examples/get-started/hello_world ~/hello_world

Step 5: Set the target to ESP32-C3

cd ~/hello_world && idf.py set-target esp32c3

Step 6: Build, flash, and monitor

Flash to the detected serial port and open the monitor immediately:

idf.py -p /dev/ttyUSB0 flash monitor

If you get a permissions error on /dev/ttyUSB0, rerun with sudo:

sudo idf.py -p /dev/ttyUSB0 flash monitor

To exit the ESP-IDF monitor: press Ctrl + ].

Expected output

After flashing, the monitor should show hello_world output similar to:

Hello world!
This is esp32c3 chip with 1 CPU core(s), WiFi/BLE, silicon revision v0.3, 2MB external flash
Minimum free heap size: 328280 bytes

Troubleshooting

1) idf.py: command not found

Re-run:

. ./export.sh

2) Wrong serial port

Re-run:

sudo dmesg -T | tail -n 30

Look for ttyUSB0 vs ttyACM0, then update the -p argument accordingly.

3) Permission denied on /dev/ttyUSB0

• Use sudo as shown above, or add your user to the serial group (commonly dialout) and re-login.