I’m not a big fan on the idea of synthetic data. I don’t have a good feeling about it. It’s a bit like training llms on the output of llms. I don’t think good things will come of it. Each iteration would loose information content.

Especially when there’s so much real data around. That’s on the road map to do. Really looking forward to it. But some other things to do first.

I have to train new models in connection with a project with Lars.

I had similar feelings when I first heard about it but listening to the edge impulse people has made me think it has more potential than I initially thought. Obviously real data is best but for niche applications where training data is limited I can see it being helpful to get models started.

Yeah. In the sense that I can also get captures of monkeys by looking for all the animal types in the default coco dataset and setting the tolerance low.

But it should be stressed that it’s only a bootstrap procedure.

In the case of edge impulse the memory in the models is very small anyway. There’s not enough memory to store the true essence of what a specific animal is so maybe comparatively it might work okay. But in something like yolov6 large model there is enough memory to capture detail so training to detect the animals versus training to detect llm generated images might have more of a difference.

Yolov6 has around 59 million parameters. Do you know how many parameters are involved in the edge impulse models you are referring to ?

This is chatgpt's answer:

"Here’s a rough idea of the number of parameters in different types of models commonly used in Edge Impulse:

1. Neural Networks for Classification (e.g., MLP, CNN)
   • Small, shallow models: These can contain as few as 10,000 to 50,000 parameters when dealing with simple tasks like basic image recognition or sensor data classification.
   • Medium complexity models: For more advanced tasks, such as keyword spotting or more detailed image recognition, models may contain 50,000 to 200,000 parameters.
   • Larger models (Edge Optimized CNNs): For tasks like more complex object detection, models can go up to 200,000 to 500,000 parameters while still being optimized for edge devices.
2. Recurrent Neural Networks (RNN, LSTM)
   • These are often used for time-series data like accelerometer or audio signals. A typical RNN model for edge deployment might have 10,000 to 100,000 parameters, depending on the length and complexity of the sequences being analyzed.
3. Transfer Learning Models (e.g., MobileNet, EfficientNet variants)
   • Transfer learning models like MobileNet, when adapted for edge deployment, are typically pruned and quantized to reduce their size. These models may contain anywhere from 1 million to 3 million parametersin their full form, but when trimmed for edge deployment, they could be reduced to 100,000 to 1 million parameters.
4. Classical Machine Learning Models (e.g., Decision Trees, KNN)
   • These models usually don’t have parameters in the same way as deep learning models, but they are lightweight and typically optimized for embedded use. They might only use a few kilobytes of memory or less, depending on the dataset size and number of features.

Overall, the model sizes in Edge Impulse are kept small to run efficiently on microcontrollers with limited memory (usually in the range of 256 KB to 1 MB of RAM and a few megabytes of flash). The number of parameters is carefully balanced to ensure good accuracy while maintaining small memory footprints for edge deployment."

Disclaimer: I can't guarantee that all of the above is not a hallucination.

So, not even 1 million parameters. yolov3 tiny model back in the day was 6 million parameters. It's my understanding that the idea behind edge impulse was that it's retrained up at the edge itself. Training requires a lot of memory than inference so by it's nature the models would be further constrained by this.

It's important to be aware of what your model is (How advanced it is), what size it is and how many parameters it contains when you consider your use case to manage your expectations. Later I will post you a few of the terrrible false positives that even the very best and largest models have made to give you an idea of just what AI is actually learning and by extension not learning. So if the best of the best can make terrible mistakes, what will small early models do?

In general though, a large advanced model can do much better at inference with a relatively small number of annotated training images than a really small model with potentially thousands and many more images.

For example, I decided way back in 2019 that yolov3 tiny model was absolutely useless for security because of the boy who cried wolf effect. 
 

The biggest use case I see for very small models would be as a hopefully better replacement to using a PIR sensor. I say hopefully, because both PIR sensors and very small models can have de-generate environmental conditions that cause large number of false positives. At a certain threshold, any perceived gain in power usage may disappear.

I welcome edge impulse to correct anything above that I may have said that was untrue as I personally have never used edge impulse, but I was involved in a project whereby yolov6 was pitted against and compared against a system using edge impulse.

Yes baboons. But this area has a few different primate species. In the weekend three species were present.

Yes, the thermal ones will be spectacular. I wondering what other species will jump into the picture when we start with that. The 640x512 thermal camera I'm testing is really amazing. Now I'm onto testing a 1280x1024 resolution thermal camera. That's going to be super interesting! As soon as I have some nice images I'll post them.