Introduction to Machine Learning on Arduino - Micro Speech - Part 1

The beginning of this guide is based off of the post Get Started With Machine Learning on Arduino | Arduino Documentation. Up through the speech recognition example, it follows mostly the same steps. I've added steps for clarity and detail, for anyone starting from scratch.  Additionally, I encountered an issue with the Windows Machine I'm using crashing due to the GameInput Application, which will be addressed at the end.

The examples in this guide are part of the field of TinyML. TinyML is machine learning done on low-power devices such as the Arduino Nano we are using. 

I have written this guide so that the program can be run and tested in the Arduino IDE as of June 2024.

This is Part 1 of 2 in this series of posts. In part 2, I will show how to change the keywords using Edge Impulse.

1 Creating a New Arduino Sketch

1.1 Prerequisites

The software installation requires a stable network connection. The host machine should have a 64-bit processor.  You will also need about 16GB of hard drive space.

1.2 Install Arduino IDE

Arduino IDE 2.3.2 is current at time of writing. Here's the link to Download the installer for your machine: Software | Arduino. 

When you click on the installer, you can select "Just Download" on the following two pages that pop up.

Once the .exe file is downloaded, open it and click on "I agree" to the installer.

Continue and Click "Next" after selecting which users to install the IDE for.

A system dialogue screen may appear (not pictured in this guide). Click "Yes" to "Allow Changes" then click on "I agree" once more.

Afterwards, click "Install"

The installation will take a minute. Once it is done, click "Finish".

1.3 Install Git

Download the correct version of Git for your machine. The latest Windows Release as of writing is 2.45.2. Click on the "Download For Windows" Button.

Here is the link to the site to download: Git (git-scm.com)       

Next, click on "Click here to download" the latest version

Open the .exe file and click "Next".

I chose all of the default selections through the rest of the installation. Feel free to make any changes necessary for your installation. For each screen, Click "Next" until you get to the "configuring "experimental options."  and can click "Install"

Click "Install". The installation will run for a few minutes.

Click "Finish". The Release notes will open in a browser and can be disregarded for now

1.4 Install TensorFlow Arduino Libraries

Open the 'Ternimal' (AKA Windows Powershell) application. There may be an icon on the start ribbon at the bottom of your desktop, if not search for 'Terminal' in the start menu.

Open the file explorer and locate your Arduimo Libraries Folder. Click on it so that its highlighted, then press Ctrl+Shift+C to copy as path. (Alternatively, you can also left click on it then click "Copy as Path".)

Once you've copied the path, return to the terminal and paste the path in the command line:

C:\Users\maxcl>:~# "C:\Users\maxcl\Arduino\libraries"

Next: Use the following command to install the TensorFlow Lite library

C:\Users\maxcl>:~# git clone https://github.com/tensorflow/tflite-micro-arduino-examples Arduino_TensorFlowLite

Once the library is installed, use the following command to change diretory to the new TensorFlow Library

C:\Users\maxcl>:~# cd Arduino_tensorFlowLite

Lastly use the git pull command to update the library

1.5 Working in the Arduino IDE

Open the Arduino IDE, either by searching for it from the Windows start menu ("Arduino IDE"), or clicking on a desktop icon. (It also may be open from the installation earlier.

When it opens, there will be a message that appears in the bottom left of the screen prompting you to install updates. Click "Install all". There also may be an update message for boards. Click "Install All" to this too.

After everything is installed, verify that the Arduino_TensorFlow Lite library is installed by looking under "File" Then "Examples" in the top right-hand corner of the IDE. You might have to scroll down in the "Examples" section to find it because it will be found under "Examples from Custom Libraries"

1.6 Attaching the Arduino Nano

Connect the Arduino Nano 33 BLE to your computer using a USB A to Micro USB cable. Note: if you have the Arduino Tiny Machine Learning kit, the Arduino Nano 33 BLE is in foam (see picture below).

Also, a USB A to Micro USB cable is provided in the kit.

In the Windows start menu, search for the "device manager".  Open it and click on "USB Serial Device(COMX)" under Ports. The Com number may differ; it is COM7 in this example.

When you click on it, it should say "This Device is working properly" under General.

When the board is Connected for the first time, A pop-up will appear in the bottom right corner of the Arduino IDE, prompting you to manually install the board package. Click "Install" then "Next". A banner will appear on the right side of the screen. Click the green "Install" button on the one that says, "Arduino Mbed OS Nano Boards." 

Once the board package is installed, click on the white text box in the top right of the screen. Note: it may say "None" or "Unknown" at first. Click on the pen icon to open the board/port selector.

When the Board Selector Opens, search for "Arduino Nano 33 BLE" and select the one that reads 'Arduino Nano 33 BLE - Arduino Mbed OS Nan...'. Under 'Ports' Select the COM device that was listed for 'USB SERIAL DEVICE' in the Device Manager. Click the Green 'OK' Button.

1.7 The Micro Speech Example

In the Arduino IDE, under File>Examples>Arduino_TensorFlowLite, click on 'micro_speech'. This will open the "micro_speech" example which does basic speech recognition. Note: Make sure to Click on 'Examples' as opposed to just hovering over it.

Once micro_speech is open, check to make sure that the proper board and port are selected. (You may need to reselect the correct COM port and the board.)

Next, click the checkmark in the top right of the screen. (The "Verify" button.) This will compile the code. (Which may take a few minutes and uses a lot of processing power). 

After the board is verified, click the arrow next to it, the 'Upload' Button. This programs the board with the code.

The board is now programmed with speech recognition. You can test it by saying "Yes" aloud and watching the LED turn green, then saying "No" for it to turn red.

While verifying and uploading the code, my computer kept crashing. After looking into the event viewer, the shutdowns were due to the 'GameInput' Application. To address this I followed the steps in the following video up to1:59. At this point, I had to have the GameInput startup to 'Manual' instead of 'Automatic' which was shown in the video. 

 

The "Manual' startup should be the default setting. If you need to change startup mode, the way to open the GameInput properties differs than in the video.

To access the properties, first, right click on GameInput in Task Manager, and click on the 'Open Services' Option. Note: Tasks are sorted alphabetically

Next, right click on the GameInput Service and choose 'Properties'. Note: Sorting is again alphabetical 

Once you are in the properties, select 'Manual' for Startup Type. If it is running, select 'Stop'. Afterwards, click on 'OK'. Now that the GameInput is not running, you should be able to run the micro_speech example without crashing.

In part 2, I will show how to change the words being used for speech recognition using Edge Impulse.