Hi Lars, thanks for your reply!

your project looks interested!, have you had a chance to test it out yet?

Yes i've had problems with "annotation pollution" as well and am working on ways to automate the relabeling etc. 

Thanks for sharing those datasets as well, i've been looking for more sources! 

@Lars_Holst_Hansen and Luke,

I've been avoiding GBIF an creating my own bounding boxes and annotations by animating about 15 museum specimens from jars, and spitting out the bounding boxes in Adobe to a CSV for import/JSON convert for training. I label per batch with whatever annotations are relevant for each batch production of a couple hundred training images (e.g., occluded subject, strong shadows, leaves, moss.)

Here's link to our research paper on Authorea pre-prints:
https://www.authorea.com/users/608968/articles/639400-animating-the-dead-creating-site-specific-synthetic-datasets-to-train-machine-learning-models-for-ecology-and-conservation

Overview presentation and GIT link
https://github.com/Automated-Wildlife-CIG/animating-the-dead/blob/main/STD%20Creation%20Presentation.pdf

https://github.com/Automated-Wildlife-CIG/animating-the-dead

Hi Luke!

The system is set up in a Zoo at the moment for testing and capture of additional footage.

It will get field deployed later this year.

That looks very interesting, Constance! 

I have played a little with background synthesis as well, but let photoshop's Generative AI create something. I am still uncertain how usefull it may be. I have also played with grabbing 360 degree images from Google StreetView and used those as backgrounds.

Original image from Animals with Attributes 2 

Hi Ed,

Sorry for the late reply!

Let me know if you ever need help! 

So Ed, if you are looking at the rpi here does this mean that you are okay with the power usage for your use case ?

@LukeD Thanks. Here's a start: have you run MegaDetector on the RaspPi + AI HAT+?

@kimhendrikse Yes, this is not for a battery-powered use case, but rather a mini-server.

Hi Ed, 

I haven't tried a MegaDetector model yet, however MegaDetector models are just trained YOLO models, so it shouldn't be be too hard (or any harder than a normal YOLO) to get it running. What have you tried so far?

Regards

I thought the Israeli Hailo stuff only worked with models for which they have built in support for ?

You can use any model architecture that uses the network layer-types they support. They have pre-defined a bunch of configurations for well known architectures which makes it easier to compile your own trained model that uses this architecture. The compellation process can be difficult, so most people just use the available, pre-trained, already complied models.

Have you tried this with any model that has greater than 15 million parameters in it ? If so, how does it compare with CPU on the same platform?

Yes, You can load large YOLO models with hundreds of millions of parameters, the Hailo 8 has 26TOPS and can run these models over 30FPS easily. A Raspberry Pi CPU would take several seconds to process a single frame...

Hmm, I don't know any object detector models with hundreds of millions of parameters. Yolov6 Large model at 1280x1280 image size, which is only 140 million parameters. A Jetson Orin NX with 1024 CUDA cores (I thought hailo was just a vector processor, so not as powerful as an nvidia GPU I would expect) could not run that at 30 fps. Maybe with 8 bit quantization that the hailo is doing though. Not sure if you can quantize that to 8-bits. I understand that yolov5s is around 7M params, yolov5M around 21M.

I believe that something like yolov5s or yolov5m with 8-bit quantization might be able to run that fast though. I've never run models with quantization as large as to 8-bits. It would be interesting to see what affect that has in accuracy in practise. Certainly, if the model was restricted to yolov5 or lower architecture it would impact accuracy.

But thanks for the insights.

Sorry, I meant ONE hundred million parameters.

The Jetson Orin NX has ~25 TOPS FP16 Performance, the large YOLOv6 processing 1280x1280 takes requires about 673.4 GFLOPs per inference. You should therefore theoretically get ~ 37fps, you're unlikely to get this exact number, but you should get around that...

Also later YOLO models (7+) are much more efficient (use less FLOPs for the same mAP50-95) and run faster.

Most Neural network inference only accelerators (Like Hailo's) use INT8 models and, depending on your use case, any drop in performance is acceptable. 

Ah I see, thanks for clarifying.

BTW yolov7 actually came out earlier than yolov6. yolov6 has higher precision and recall figures. And I noticed that in practise it was slightly better.

My suspicion is that it's not trival to translate the layer functions from yolov6 or yolov9 to hailo specific ones without affecting quality in unknown ways. If you manage to do it, do tell :)

The acceptability of a drop of performance depends heavily on the use case. In security if I get woken up 2x a night versus once in 6 months I don't care how fast it is, it's not acceptable for that use case for me.

I would imagine that for many wild traps as well a false positive would mean having to travel out and reset the trap.

But as I haven't personally dropped quantization to 8-bits I appreciate other peoples insights on the subject. Thanks for your insights.

@LukeD, I am looping in @Kamalama997 from the TRAPPER team who is working on porting MegaDetector and other models to RPi with the AI HAT+. Kamil will have more specific questions.