I am looking for open source camera trap management software within which I can integrate the models we are developing. We also want to help build out such an application to make it easier for ecologists to use and maintain.
I believe Camelot was a project funded by Flora and Fauna, so I want to chat with you anyway. Happy to have a call to discuss/clarify. :)

Ah, okay that is helpful. Great question. 

I'm not sure the current status of camelot - @cshclm @heidi_h, maybe you could provide an update? 

The two projects that came to mind may not be relevant, but I'll flag them in case they are? First is Pytorch-wildlife, which @zhongqimiao  will be speaking about this afternoon at Variety Hour. The other is achille (dont know if I spelt that right?) - @EricSchmidt might be able to pop in here an see if what his team is working on is relevant? 

Hope this helps! 

Steph 

I'm not doing anything with video though. But that's likely just a python opencv video consumer that then processes the images (Then as a service) and writes them out as a video. So very similar.

@pvlun - I finally had a chance to try EcoAssist. Great work! This is a great starting point for the conservationists I'm working with to utilize MegaDetector. I'm mainly working with researchers in the Americas (North and South). We may need to train some classifiers (or find some), but we may also want to integrate models from animl-py. 

For videos, is there a way to see the frame by frame detections? The results json file only seems to indicate 1 bounding box per category in the video.

Ultimately, we may still want to go for a full camera trap data management platform like TRAPPER. Supporting the Camtrap DP format may enable compatibility with TRAPPER.

@bluevalhalla There will be a amazon classifier (PyTorch Wildlife) added to the list soon (hopefully this week). 

Adding the animl-py models is on my TODO list, but unfortunately the TODO list is quite long....

At this point there is no option of visualising video's. What you are seeing is the direct result of MD on videos. It would be possible if we keep track of which bounding box corresponds to which frame. Perhaps a feature which exports the frame with the highest confidence per video would be sufficient. But then again, at this point I don't have a lot of time to work on that. 

Same with CamtrapDP, it's on my TODO list!

@pvlun Glad to hear about the Amazon classifier. I will be in Ecuador for the next month with access to some camera trap data from their Amazon region.

I may be able to help with animl-py if you would like. In addition to the Amazon model, they are also working on one in the Andes. This will help with th researchers we will work with on Andian bears. I want to look into integrating them to TRAPPER as well.

Video visualization is a pain. I will eventually want the frame level MD results. We are using sequences of an individual in a video to help boost the individual identification accuracy. Can MD output the results for every frame it sees?

@bluevalhalla EcoAssist now has the Microsoft AI4G Amazon model incorporated. If you update to v5.6 you will have automatic access.

Read "🚨 New open-source AI model for camera trap images available! 🚨 🌳 🐿… | Addax Data Science" on

🚨 New open-source AI model for camera trap images available! 🚨 🌳 🐿… | Addax Data Science

🚨 New open-source AI model for camera trap images available! 🚨 🌳 🐿 𝐀𝐦𝐚𝐳𝐨𝐧 𝐑𝐚𝐢𝐧𝐟𝐨𝐫𝐞𝐬𝐭 🐆 🌿 EcoAssist has now incorporated the Microsoft AI4G Amazon classification model for camera trap images, capable of recognizing 35 genera. • Dasyprocta • Bos • Pecari • Mazama • Cuniculus • Leptotila • Human • Aramides • Tinamus • Eira • Crax • Procyon • Capra • Dasypus • Sciurus • Crypturellus • Tamandua • Proechimys • Leopardus • Equus • Columbina • Nyctidromus • Ortalis • Emballonura • Odontophorus • Geotrygon • Metachirus • Catharus • Cerdocyon • Momotus • Tapirus • Canis • Furnarius • Didelphis • Sylvilagus Follow Addax Data Science to stay up-to-date for more free models to come. Install EcoAssist for free and try it out - both the model and the software are open-source: https://lnkd.in/gDcCYBMp The model is developed by the AI For Good Lab, Microsoft. The model is trained to classify animals into their genus taxonomic group. The training data are collected by the Department of Biological sciences at Universidad de los Andes. All the images come from the 'Magdalena Medio' region in Colombia. The dataset contains 41,904 images across 36 labeled genera, and it is distributed into 33,569 images in the training set and 8,335 images in the validation set. In our inference of the Amazon Rainforest dataset, we implement a 98% confidence threshold as part of a human-in-the-loop procedure. We observe that the model predicts 90% of the data with recognition confidence exceeding this threshold. Furthermore, within this high-confidence subset, the model achieves an average classification accuracy of 92%. This means that, after filtering out empty images with MegaDetector, only 10% of the detected animal objects require human validation. Video credit: fragments taken from Chucini Amazon Jungle Eco Tours Bolivia (https://lnkd.in/eXHsf2UB), and visualised with EcoAssist.

With regards to the video processing. EcoAssist takes the MegaDetector output as a starting point. For video's, that output JSON contains one detection per class per video. @dmorris Can you shed some light into how this is calculated? Would it be possible to get the frame number in that JSON as well?

Thank you for the update, @pvlun! I have installed and started using the Amazon model. It looks like the model is genus rather than species classifier. It's a shame they don't seem to have puma or panthera classes. Those are key genus for the researchers in Ecuador. Anyway, I should have access to more data when I get the the university in Quito next week. I'll test more then and see what they think of the classes.

@bluevalhalla those are indeed only genera. And yes, I also found that there are some crucial genera missing. Nevertheless might it still be handy for you to make a rough separation before doing any human verification. 

Regarding the video processing: I didn't catch that you're looking for an MD results file that does all frames. MD creates that, but EcoAssist deletes it again (to avoid confusion with the video-level detection file). If you want the frame-level detection file, just out comment these lines. Then you'll have both. 

Answering Peter's question to me above, about how MegaDetector's video processing stuff decides which detection to include in the video-level output file:

• By default, the function that matters just chooses the highest-confidence detection for each category (person/animal/vehicle) as the "canonical detection" for that video (for posterity, that happens here).
• The frame number corresponding to that detection was not previously included in the output (since 99.99% of users would have no way of using this, but it sounds like you're in the other 0.01% :) ), so Peter just submitted a PR to the MegaDetector repo to include this in the output for a video; I'll merge that PR today, so I expect you'll see this in a forthcoming version of EcoAssist.

But ditto what Peter said, it sounds like you actually want the frame-level output file, and Peter gave you a tip about how to get that.

Also FWIW you will almost never want to process every frame of a video, nor will you likely want to train your models on every frame of a video; you're not getting a lot of new information from adjacent frames.  Typically when MegaDetector users process videos, I advise sampling videos down to ~3fps, which is usually every 10th frame.

@bluevalhalla The latest EcoAssist version now includes the frame_number in the video detections.

{
 "images": [
  {
   "file": "animal_2.mp4",
   "detections": [
    {
     "category": "1",
     "conf": 0.975,
     "bbox": [
      0.2609,
      0.4944,
      0.5531,
      0.4194
     ],
     "frame_number": 168			# <- NEW
    }
   ]
  }
 ],

@pvlun Thank you for the pointer to keep the frame level detections. I'll give it a try next week.

@dmorris Thanks for the description of MDs video processing. I didn't even realize MD had video processing. I was extracting frames with ffmpeg and running those through MD as images. I don't typically run every frame. I'm usually in the 1-5 fps range. I should have said I was looking for the frame level details for the all the extracted frames (as opposed to all frames).

Clarifying: MD handles video only in the sense that you're suggesting, i.e. MegaDetector only knows about individual images, but there is supporting code to split videos into frames, run MD on frames, and do something reasonable at the end to create video-level confidence values that let users work with videos in Timelapse (typically the video-level confidence for a category is just the maximum confidence on any frame in the video for that category).

For small jobs with just a few videos, I would typically just do this with process_video or process_video_folder, but for very large/complex video jobs (multiple TB, or cases where I really need to do custom postprocessing on the frames), I do this via this notebook (actually its .py cousin, but they are the same content).  But if you want to tinker with this from your own code, take a look at:

• Documentation for the megadetector.detection.process_video module
• Documentation for the megadetector.detection.video_utils module

The latter includes stuff like splitting into frames, re-assembling frames into videos, etc.

@pvlun , I have a few more questions for you, but I will follow up via email to save this thread. :)