Good point, Kim!

I don't know about Arbimon. But in general size does matter a lot here. I believe I heard the word petabytes during one of the presentations on bio acoustics.

For other technologies data output size may be different of course.

Interestingly, size matters, but only within a certain time frame. In the late 70s we talked about kilobytes, a megabyte was a lot and I guess no-one knew what to do with petabytes.

So perhaps, just talking in terms of size, funding the infrastructure may just be a matter of waiting a couple of years.

Or as Bob Planqué @rplanque pointed out, using compressed file types keeps overall storage requirements and related costs low ( my summary, please correct me if I'm wrong here, Bob ).

With my systems I'm storing 44khz flac files, it's around 95GB per month, so about 1.1TB per year per device.

A raspberry pi installed with 2x nvme 4TB drives can be a tiny device and would store 7 years of data for a single device. Here I have a tiny box (5x4x3 cm) that can store 8TB on encrypted nvme drives with a pi. (Although I use mirroring for data resilience, so that's 4TB). It would still be interesting to know what the local storage requirements could be for people to get the full picture :) Additionally, standards relating to the file storage and metadata would need to be decided upon. This is likely one of the more complex aspects as it involves people agreeing with each other. But... with agreed upon standards, a cloud boosted, distributed system is fully viable and would be cost limited by nature.

For me, I think that a large repo to store everything will not have a long term future, wikipedia are struggling to survive at the moment. We have seen in the past a distributed delivery system for files. Remember? The illegal music sources.

So maybe one way to go would be that the organisations that produce the data maintain their own repos and that there exists a large central repo that caches and distributes these datasets. That way the size of the central repo grows no larger that the popularity of the datasets in questions. Not everything is as popular as others. The repo size can be fixed and by using a caching approach the data streaming load is limited for the supplier of the data source.

Hej @kimhendrikse,

Not really the kind of answer I was looking for ( because we now still don't know how you finance your setup ), but it is an interesting answer to the question which is also practiced here and there. But the financing problem will not disappear. It will just be distributed as well. The nodes still need funding. Less so per node, which might make it a lot easier, but there is still funding involved. Perhaps there is even a way to distribute the 'central' catalogue. I wouldn't insert a central repositoring chaching data because then we're back at something big and expensive again. 

Also, there will be costs involved in developing and maintaining standards, which also need to be paid - through membership fees perhaps. Lastly one would need a way to deal with those nodes that cannot fund themselves anymore. It would lead to fewer and bigger nodes and should not happen too much because then we're back at square one.

"because we now still don't know how you finance your setup"

I pay for everything myself out of the salary from my full time job :-(

But... long term I'm seeing a future in starting a company that sells products. (Maybe not that long term).

That's a really honest answer Kim! Thank you for clarifying

You're likely not the only one. In fact it would be interesting to find out how much conservation tech use and development is funded through personal means. I made a note. Perhaps we'll do a poll sometime