WILDLABS logoWildlabs.net : The conservation technology network

WILDLABS logoWildlabs.net : The conservation technology network

Group

Data management and processing tools / Feed

Conservation tech work doesn't stop after data is collected in the field. Equally as important to success is navigating data management and processing tools. For the many community members who deal with enormous datasets, this group will be an invaluable resource to trade advice, discuss workflows and tools, and share what works for you.

discussion

unsupervised machine learning to infer syntax and temporal organisations of animal vocalizations

Luigi

7 September 2023 3:02pm

I wish to share an update on my MSc thesis project, that contributes to the field of decoding animal communication. In my work I conducted a factorial experiment  to...

1 2
Kim Hendrikse

26 November 2024 3:41pm

Super interesting work! Maybe one day you will also be able to have a career as a science fiction writer. Lot of interesting outcomes can come of this. ❤️

See full post
discussion

Need advice on data for an app that recommends plants.

Colleen

21 November 2024 1:47pm

This is going to show how little I know, but I'm doing my best! Basically I could use help understanding what I should do from a data standpoint in order to enable a...

1 2
Frank van der Most

21 November 2024 8:41pm

It's a really good question, Colleen!

Ideally, I would work with two developers, or at least one developer and one other party who knows what developing an app like this would mean. This will lower the risk that the developer answers your question too much in their advantage. So this is one reason why it is a good question, and maybe this is why you ask it. However, if you really trust your developer not to take advantage, then go with just her/him/they.

It's also a good question because there is no easiest way to go about this if you're on a budget. You mention that you do not have a tech background, so here comes some explaining. If I misunderstood, then please skip and continue at "Back to 'the easiest way to go about this'"

There are at least two things you should be really aware of when it comes to software development.

The first is that once the basic data structure is defined and the software built around it, it is extremely costly to change the data structure. It's like deciding after the car has been built, that the engine should go to the back of the car instead of the front.

The second thing is that the basic data structure is dependent on ( among other things, but I'd say these are the two most important factors ) the complexity of what needs to be achieved and the speed at which it needs to be done.

The difficulty is that the required complexity and the speed may change over time, which brings one back to the car and engine situation. Changes in complexity and speed requirements may be the result of many things, one of which is success. You get far more clients than anticipated, so the system needs to be scaled up. In addition, with more users come more feature requests ( this can work both ways: new features result in more users, and more users may result in more feature requests ).

There is no real solution to this problem ( well, except not growing beyond the point that the first design can handle ). When it comes to scaling up, one vendor may claim that their database back bone easily scales up. Maybe so - but it may come at a price and they may also underestimate your and their own future needs. When it comes to changes in complexity, additional features can in the beginning probably be added on without changing the basic data structure. Maybe an additional row of seats at the back of the car, a trailer hook, bigger lamps, a roof-rack, a trailer, suitcases on the rack. At some point the car will need a new and bigger engine to carry all those add ons and keep at the same speed.

Here is a prediction : the more you stress cheap and efficient at the beginning, the bigger these problems will be later on. But when the business is successful, there will be more money to invest in scaling up and redesigning. Obviously yes, but in terms of the car metaphor, you may find that you want the car to be kept running with all its added on features, while the engine is replaced and moved to the back. It may be possible, but perhaps out of reach of patience and the increased income.

 

Back to 'the easiest way to go about this':

Invest a little effort to find out not only what are the minimum requirements for the MVP, but also what else you or your clients may want in the future. The developer should then have these future requirements in mind (and future upscaling) when they start developing for the minimum ones. This means, develop a somewhat more generic data structure than what is necessary for the MVP. This will cost some more at the beginning but should save a lot later on. I'm writing 'should' not 'will' on purpose. It's a balancing act because taking too much into account has the risk of over-engineering for a future that may not happen, or develop differently than expected. Like I said, there is no easiest way out.

 

A few more detailed comments

If there is no API for a source, try to go around it if possible at the beginning. API's are made with some long term stability in mind. Websites and web pages not necessarily so or less so. They will require more monitoring and maintenance on the web scraping routines.

Perhaps the developer may not be aware of the necessary pre-work that needs to be done if the pre-work depends on biological knowledge needed to transform the data from your data sources into data that allows easy ( and fast ) calculation of results to the users. 
 

See full post
discussion

Automatic extraction of temperature/moon phase from camera trap video

Lucy Dimitrova

29 November 2023 1:15pm

Hey everyone, I'm currently trying to automate the annotation process for some camera trap videos by extracting metadata from the files (mp4 format). I've been tasked to try...

10 1
John Loehr

7 September 2024 9:44am

I just noticed that TrapTagger has integrated AI reading of timestamps for videos. I haven't had a chance to try it out yet, but it sounds promising. 

John Loehr

28 October 2024 7:30pm

Small update. I uploaded >1000 videos from a spypoint flex camera and TrapTagger worked really well. Another program that I'm currently interested in is Timelapse which uses the file creation date/time. I haven't yet tried it, but it looks promising as well. 

Michal Smielak

1 November 2024 12:29pm

Hi Lucy,



I now realised it is an old thread and you most likely have already found a solution long ago but this might be of interest to others.

As mentioned previously, it is definitely much better to take moon phase from the date and location. While moon phase in general is not a good proxy for illumination, that moon phase symbol on the video is even worse as it generalises the moon cycle into a few discreet categories. For calculating moon phase you can use suncalc package in R but if you want a deeper look and more detailed proxy for moonlight intensity, I wrote a paper on it 

 with accompanying R package called moonlit 

When it comes to temperature I also agree that what is recorded in the camera is often very inconsistent so unless you have multiple cameras to average your measurements you are probably better off using something like NCEP/NCAR Reanalysis (again, there is an R package for that) but if you insist on extracting temperature from the picture, I tried it using tesseract and wrote a description here: 

 

Good luck!

See full post
discussion

How do you fund or finance your long-term data infrastructure?

Frank van der Most

12 September 2024 5:28pm

Today at the Bioacoustics Data Networks and Platforms webinar during the after hour, there was a great discussion about long term data infrastructure. Not just how to...

6 2
Frank van der Most

24 September 2024 2:35pm

Hej @kimhendrikse,

Not really the kind of answer I was looking for ( because we now still don't know how you finance your setup ), but it is an interesting answer to the question which is also practiced here and there. But the financing problem will not disappear. It will just be distributed as well. The nodes still need funding. Less so per node, which might make it a lot easier, but there is still funding involved. Perhaps there is even a way to distribute the 'central' catalogue. I wouldn't insert a central repositoring chaching data because then we're back at something big and expensive again. 

Also, there will be costs involved in developing and maintaining standards, which also need to be paid - through membership fees perhaps. Lastly one would need a way to deal with those nodes that cannot fund themselves anymore. It would lead to fewer and bigger nodes and should not happen too much because then we're back at square one.

 

Kim Hendrikse

24 September 2024 2:44pm

"because we now still don't know how you finance your setup"

I pay for everything myself out of the salary from my full time job :-(

But... long term I'm seeing a future in starting a company that sells products. (Maybe not that long term).

Frank van der Most

24 September 2024 3:42pm

That's a really honest answer Kim! Thank you for clarifying

You're likely not the only one. In fact it would be interesting to find out how much conservation tech use and development is funded through personal means. I made a note. Perhaps we'll do a poll sometime

See full post
event

Design-led innovation for nature

Catherine Greene

20 September 2024 2:24pm

How can space-enabled data and services be translated into information that is trusted and actionable, fuelling responsible behaviours towards nature? A new programme where SMEs will be paired with design consultancies...

0
See full post
discussion

efficient methods for labeling clusters of fish in images for segmentation purposes 

Tongfei Yang

16 July 2024 9:43pm

Hi everyone,Our team is working on a project involving the segmentation of individual fish within clusters (groupers specifically). We have images with hundreds of groupers...

2 0
Lampros Mouselimis

5 August 2024 9:58am

hi, if your intention is to use programming and a programming language like Python then one option which can give good results is the SAM (Segment Anything) or SAM-2 algorithm. The following weblinks include more information:

 

Larry O'Brien

17 August 2024 12:34am

Agreed. I have recently begun using SegmentAnything as a replacement for Detectron and have been very pleased. However, for a fish school I don't know how well it would do out of the box with the many overlapping fish. 

See full post
discussion

Rapid Camera Trap labeling with ChatGPT

Sara Olsson

17 July 2024 3:19pm

Hi all, I've published a blog post on rapidly labeling camera trap data using ChatGPT for species identification and a simple object detection model to get the bounding boxes....

3 6
Herdhanu Jayanto

20 July 2024 3:28pm

super interesting, thank you for sharing! definitely will watch!

Sarah Mshanga

3 August 2024 7:46pm

Really interesting, I will take a look

See full post
discussion

WILDLABS AWARDS 2024 - Mobilizing Motus: optimizing a wildlife tracking data pipeline for aninternational, collaborative conservation technology community

Lucas Berrigan

13 July 2024 10:04pm

We’re delighted to introduce our project, Mobilizing Motus. Motus, a Latin word for movement, is a collaborative wildlife tracking system that uses automated radio telemetry...

2 4
See full post
discussion

Firetail 12.2 released

Tobias Petri

12 June 2024 7:01pm

Hi all!I'm happy to announce that we just released Firetail - release 12.2A few new things in a nutshellA new evaluation module for evaluating the performance of predicted...

2 2
See full post
discussion

Behavioral logging from video

Kyler Abernathy

12 July 2024 6:01pm

Hi everyone.  I wanted to ask if anyone has found a tool that they'd recommend for logging/coding behavior from video.  I've used BORIS and do like that (and it's free...

2 2
Rob Appleby

26 July 2024 1:59pm

Hi @KylerA long time no speak! I've used BORIS mostly too, but I've also heard good things about CowLog, so could be worth a look. I also found this (now 4-year-old) paper that discusses a few different options, in case you hadn't seen it before. 

All the best,

Rob

See full post
discussion

Simple Data Standards for wildlife reserve management scenarios?

HeinrichS

23 July 2024 7:16am

Hi - I've been lurking on Wildlabs for a while, consuming as much info as humanly possible, but I seem to miss specific info for my current needs.I'm the self appointed in house...

2 1
Carly Batist

24 July 2024 5:48pm

Just as a note, you might also be interested in specific protected area management software like SMART or EarthRanger as well!

HeinrichS

25 July 2024 10:11am

Thanks!

I'm definitely looking at Smart and EarthRanger for overall management eventually, but for now I just want to track what we have better - especially as a base map to support further advanced management activities. Funding and available human resources to effectively manage a tech based scientific and best practice focused solution is in short supply - but will hopefully be better addressed within the next year or two. 

 

See full post
discussion

Auto ID Software for NA Amphibians?

Cortney Dean

20 July 2024 3:57am

Hi All!So I'm currently running 4 Wildlife Acoustic Song Meter MiniBats at my beaver research and control sites as a means to measure species richness. 2 units at each site, one...

2 0
Vanesa Reyes

24 July 2024 5:12pm

Hi Cortney,

I recommend exploring the tools developed by Kitzes Lab, such as OpenSoundscape, which can be used for the automatic detection of anurans. Here's a reference paper that might be useful. You might also consider reaching out directly to Justin Kitzes @jkitzes  for more specialized advice.
Also here is a another paper that use BirdNet to automatically detect the bullfrog presence.
Best of luck with your analyses!

Carly Batist

24 July 2024 6:15pm

Retweet on OpenSoundscape - great package and documentation that allows you to build your own CNNs! Note that this is in Python though. 

There are tons of bioacoustics software and packages in @tessa_rhinehart 's helpful list too! Some more recent options that I don't think are on that yet are Whombat and gibbonNetR (which uses the 'Torch for R' ecosystem). 

Also, Raven has template- and energy-based detectors you could try as well. The monitoR and warbleR packages allow you to do template-based detections too. BirdNET is integrated into Raven now as well.

You can also try Kaleidoscope's native clustering function.

Agree with you that there's definitely not enough resources for frog detection though! I'm assuming the Australian one you found is FrogID? There is also AnuraSet, but this is for neotropical frogs.  

See full post
discussion

Move BON Development: Follow up discussion

Lacey Hughey

19 June 2024 12:02pm

Hey Biologging Community! We just launched a new initiative to mobilize animal tracking data in support of national and global scale conservation goals (learn more here!). If you...

6 6
Akiba

12 July 2024 5:38am

Hi Talia! 

I feel like the topic is so broad that it might help to put some constraints around things, see what works, and then broaden those out. I have a lot of ideas regarding the data monitoring and collection side based on the other sensor and observation networks we've set up in the past. 

There may also be some potential scope to incorporate things like data collection and integrated monitoring to the Build Your Own Datalogger series where the system is updated to feed data into the observation network. 

It'd probably take a bit of discussion and coordination. Let me know if interested. I'm fine to jump on a call or discuss via email too.

CathyNj

22 July 2024 9:24pm

@cmwainaina please take a look

See full post
discussion

Looking for literature materials/any useful data on HHC(Human-Hippopotamus Conflict) mitigation and coexistence.

Loveness L Mutungi

12 June 2024 7:24pm

Hello Wildlabbers,I am looking for anyone who have done some research on human-hippopotamus conflict mitigation and coexistence or anyone who might have some scientific info/...

8 7
Nyangeta Magesa

7 July 2024 10:07am

Hi loveness,

Your doing great here are some research that may be helpful on what your looking for

search for article of Human-Hippopotamus Conflict: Impacts and Mitigation Strategies published in the Journal of Wildlife Management.

https://www.google.com/search?client=ubuntu-sn&sca_esv=23430e1df8f38b82&channel=fs&q=Assessing+Human-Wildlife+Conflict+with+Hippopotamuses+in+the+Context+of+Wildlife+Conservation+from+the+African+Journal+of+Ecology.&sa=X&ved=2ahUKEwigkNqHx5SHAxXi0gIHHaOBA-EQgwN6BAgFEAE

 

 

See full post
Link

Rapid Camera Trap labeling with ChatGPT

Sara Olsson

17 July 2024 3:02pm

Hi all, I've published a blog post on rapidly labeling camera trap data using ChatGPT for species identification and a simple object detection model to get the bounding boxes. While presented in Edge Impulse, this approach can be replicated outside the platform using Python...

1