discussion / AI for Conservation  / 5 December 2024

Mirror images - to annotate or not?

We are pushing hard to start training our custom model for the PolarBearWatchdog! soon.

This includes lots of dataset curation and annotation work.

A question that has come up is what to do about mirror images regarding annotation.

Should they be annotated: 

polar bear on sea ice

or not:

polar bear on sea ice

Images from iNaturalist by boogan_boy (cc-by-nc)

These are MegeDetector5b detections run via EcoAssist btw. (and the exclusion is just a matter of confidence threshold).

Sometimes a mirror image can be of higher quality than a standard depiction so will it not confuse the model to tell it to treat it as "background"?

Any thoughts? @dmorris @eugenegalaxy @HugoMarkoff 

 




Lars Holst Hansen
@Lars_Holst_Hansen
Aarhus University
Biologist and Research Technician working with ecosystem monitoring and research at Zackenberg Research Station in Greenland
Conversation starter level 3
Popular level 3
Poster level 2
Involvement level 3
Commenter level 4

BTW sometimes the otherwise tremendous MegeDetector gets confused:

Polar bear on edge of sea ice

Image from iNaturalist by boogan_boy (cc-by-nc)

 

 

Oh, I do love edge cases!  Personally I would not recommend annotating the reflections in most situations, but it's a close call.  It almost definitely doesn't matter unless you have lots of images like this relative to your total data set, but in your case, you might, so I could be convinced.  Maybe I'm 51% in favor of annotating them in your case?

One of my all-time favorite camera trap images is the thumbnail we used on LILA for the NACTI dataset, I like it so much that I also use it as the fun image at the bottom of the main MegaDetector README:


water bird with a reflection

This one is interesting, because IIRC both MDv5a and MDv5b identify the reflection when I run the reduced-size image (640px) that's used as a thumbnail, but not when they can see the full resolution of the original image (which MD would see at 1280px)... here's MDv5b run on the 640px version:


water bird with a reflection and bounding boxes
 

Other edge cases like this that have presented interesting issues in various datasets:

  • Based on my very non-scientific anecdotal experience with suburban camera traps in England, English people seem to enjoy placing animal-shaped statues in their yards.  English WILDLABS folks, is that a thing?  In any case, they sure do look like animals if you're an object detector, in fact they often look like animals to my human eyes in a single image also.
  • Lots of businesses in urban environments like to have giant animal painted on the sides of their trucks.  Sometimes those trucks are related to the animals (like a dog grooming business), but usually it's more like "we're the big elephant of local roofing companies", with a gratuitous elephant on the side of the truck.
  • Carcasses set out as bait are a constant challenge... we usually don't annotate them as animals, but they are animals, just... messy ones.

 

Lars Holst Hansen
@Lars_Holst_Hansen
Aarhus University
Biologist and Research Technician working with ecosystem monitoring and research at Zackenberg Research Station in Greenland
Conversation starter level 3
Popular level 3
Poster level 2
Involvement level 3
Commenter level 4

Hi Dan!

Thanks for your comments!

I guess statistics will save me (no matter what I decide to do) as these kinds of images will not be very common in the total training set.

It is quite fun to see how the rolling shutter has resulted in quite a different mirror image of the heron in your example! 

Speaking of edge examples, in this one, I most probably will NOT annotate the mirror image:

Polar bear on sea ice

Image from iNaturalist by boogan_boy (cc-by-nc)

I guess it will be somewhat subjective.

Regarding carcasses, I have noticed that too.

Here is an example from our setup in Copenhagen Zoo:

Polar bear in a zoo

In most cases, the carcass got a very low confidence - but not in all!

Funny you mention the garden figurines!  While going through the "bear" class of the COCO dataset, I found so many Teddy bears, bear sculptures, postcards, drawings and even an image of gummy bears! I guess they represented "bear" as a concept - although some of the images were obviously misplaced as there is a "Teddy bear" class too. There were also some animal species misplaced as bears - including dogs, wolverine, koala and even an ostrich! The same is somewhat true of Goggle's Open Images Dataset v7, but at least the socalled "depictions" (photos of images, drawings, sculptures etc.) are tagged as such. 

polar bears from Goggle's Open Images Dataset V7

A white tiger has sneaked into the "polar+bear" class in the Animals with Atributes 2 dataset! These public images Including Wiki Commons also often contain a lot of stuffed animals - which I filter out. 

Lars Holst Hansen
@Lars_Holst_Hansen
Aarhus University
Biologist and Research Technician working with ecosystem monitoring and research at Zackenberg Research Station in Greenland
Conversation starter level 3
Popular level 3
Poster level 2
Involvement level 3
Commenter level 4

Here is a case where MegaDetector gives a lot of false detections:

Polar bear mum and cub amongst red rocks

Image from iNaturalist by cintylee (cc-by-nc)

Not so surprising when comparing to this one:

Polar bear in front of big group of walrusses on a beach

Image from iNaturalist by cotinga (cc-by-nc)