Hi WILDLABS Community,

My colleagues and I are currently using Insta360 cameras to gather underwater video footage of beluga whales in the Churchill River estuary (Canada). While this has given us amazing footage of these whales, we are struggling to use the 360 environment to extract relevant scientific information beyond simple behavioural observations, or without having to painstakingly edit and flatten each video using the proprietary software included with the cameras.

Our original workflow utilizes a single direction camera providing us with 'flat' videos of beluga whales when in view. We split these videos into individual frames. Empty frames are dropped by a custom machine learning algorithm to retain only frames with beluga in them (read more here). These frames are then fed into our Zooniverse project, Beluga Bits, for further classification through a variety of workflows to extract information (ages, sexes, markings, etc.) and make observations on other species present, like jellyfish. 

Essentially, the advantage of these 360 cameras is that we can track individuals for more than just a narrow window into the sea. However, we are struggling to find a solution that might allow us to track individuals as they move through the 360-degree videos, and then extract frames or a series of frames of that individual in an encounter. 

I'm somewhat aware of computer vision efforts, and a few others, but we find ourselves at a loss to overcome this challenge. I just thought I'd find out if anyone knows of a solution or can point us in the right direction.

Thanks! Any help would be greatly appreciated!

Courtney