Chapter 14: Designing Your Own TinyML Applications
What Model Architecture Should I Use?
| Problem Type | Preprocessing | Architecture | Post-processing |
|---|---|---|---|
| Image Classification | - Resize images to consistent size - Data augmentation (e.g., rotation, scaling) - Normalization |
Convolutional Neural Networks (CNN) | Softmax for class probabilities |
| Object Detection | - Resize images to consistent size - Data augmentation - Normalization |
- Faster R-CNN - SSD - YOLO |
Non-max suppression for bounding boxes |
| Audio Classification | - Convert to spectrogram or MFCC - Normalization |
- CNN - Recurrent Neural Networks (RNN) - LSTM/GRU |
Softmax for class probabilities |
| Time Series Forecasting | - Normalization - Sequence windowing |
- LSTM/GRU - 1D CNN |
- |
| Natural Language Processing | - Tokenization - Sequence padding - Embeddings |
- RNN - LSTM/GRU - Transformers (e.g., BERT, GPT) |
- |
| Machine Translation | - Tokenization - Sequence padding - Embeddings |
Sequence-to-Sequence models with attention | Beam search, greedy decoding |
| Accelerometer Data | - Normalization - Sequence windowing |
- CNN (for multi-axes data as channels) - LSTM/GRU |
Softmax for class probabilities (if classification task) |
| Semantic Segmentation | - Resize images to consistent size - Data augmentation - Normalization |
- U-Net - SegNet - FCN |
Pixel-wise softmax or argmax |
| Anomaly Detection | - Normalization (if required) | - Autoencoders - One-Class SVM (though not deep learning) |
Thresholding on reconstruction error |
Wizard of Oz-ing
This technique is about testing design ideas without heavy technological investment. Given that determining the right requirements is the hardest part of engineering and that building machine learning models is time-intensive, it's valuable to validate ideas before deep dives. The Wizard of Oz approach involves creating a system mock-up where, instead of software, a human mimics the decision-making process. This helps identify issues early and refine system requirements.
Example: Biosignal Processing Application: Heart Rate Monitor for Stress Detection
Imagine you're designing a wearable device that detects a user's stress levels based on their heart rate variability (HRV). The aim is for the wearable to alert users when their stress levels reach a threshold, prompting them to take a break or engage in relaxation exercises.
Instead of immediately investing in a fully integrated sensor system with real-time HRV calculation and stress level prediction algorithms, start with a prototype setup.
-
Mock-up Setup:
- Use an off-the-shelf heart rate monitor to continuously measure a user's heart rate.
- Feed this heart rate data to a laptop or mobile device where a human "wizard" is observing the data stream.
-
Human Decision Making:
- Provide the human observer (the "wizard") with basic training on how HRV works and its relation to stress. Equip them with simple tools or software to manually compute HRV.
- The "wizard" continuously monitors the HRV and, based on the observed patterns and predetermined criteria, decides when the stress threshold is crossed.
- Whenever the "wizard" deems the stress level has reached the threshold, they send a manual alert to the user's device (a simple notification).
-
User Interaction:
- The user, believing that the system is fully automated, will interact naturally with the alerts.
- Gather feedback from users on the accuracy and usefulness of the alerts, how they felt, and if they noticed any specific triggers before receiving an alert.
Get It Working on the Desktop First
Before deploying a model on embedded platforms, which can be time-consuming, it's advantageous to first test it on a desktop or cloud machine. By streaming sensor data to these machines for processing, rapid experimentation and iteration become possible. While this method might be energy-intensive for actual deployment, it provides invaluable feedback on the machine learning solution's efficacy in the overall product.
A significant advantage of this approach is the ability to record and reuse sensor data streams for model evaluations. This repeated testing can identify and correct high-impact errors that might be missed in standard metrics—like misclassifying a baby's photo as a dog, which could be distressing for users despite high overall accuracy.
To implement this: 1. Start by gathering data using platforms like Raspberry Pi, which offers good sensor support. 2. Transfer this data to a desktop or cloud for processing. 3. Initially, use standard TensorFlow in Python for offline training and evaluation. 4. Once a promising model is developed, take steps like converting the model to TensorFlow Lite but still test it on the desktop. 5. For a more realistic environment, integrate the desktop TensorFlow Lite application with a simple web API, allowing communication with devices closer to the intended final product.