Download image patches of annotated objects from VIAME?

VIAME & DIVE
1

Hi Kitware community,

I have what is likely to be a very quick and simple question about the online version of VIAME.

Do we have the ability to download our manual annotations – the actual image patches of the objects – such that we could use them elsewhere? That is to say, not just the .csv file of the metadata, but the actual image patches of the annotated objects?

For reference, we are using small ROVs to conduct photo/video surveys of kelp forests (you can learn more about our work on GitHub here - we’re trying to make as much information as possible available to the public).

Also, as an example, see here in the public domain for an example of our imagery and our manual annotations - this sequence of images is a training set to help our volunteer get comfortable in VIAME. Note that no tracking is involved - these are “single observations” (non-overlapping still images).

So, my question - can we download our manually annotated image patches?

Thank you very much!

Zach

2

Under the ‘Download’ or ‘Export’ tabs are the options to download either a CSV or JSON. Either of these files have all of the box/species/polygon/attributes, whatever was generated in the interface, in a single file (one per sequence). They can be downloaded in the annotator for a single sequence, or in bulk across multiple sequences in the file viewer. The format of the CSV file is specified here:

https://viame.readthedocs.io/en/latest/section_links/detection_file_conversions.html

1 Like
3

Thank you @matt_dawkins, for your quick response!

Okay, so it sounds like we can’t export the “raw” objects that have been detected themselves in pixel form. BUT, with either the .csv or .json, we can write a script to line up the bounding box coordinates from the output file with the specific images that we annotated. That should provide the same endpoint products that we want.

If useful, we’ll share that script here once we have it so that others may use it, if desired (though building this functionality is a slow-burn, so it may be a little while before I circle back with the script).

Thanks again,

Zach

4

If you want images for just the bounding box regions (without the rest of the image) there are some scripts that do this in the VIAME desktop version given the images/video and a truth csv alongside it. This can be run in bulk over multiple video/csv pairs at once.

See VIAME/examples/annotation_and_visualization/extract_chips_from_detections.bat at main · VIAME/VIAME · GitHub or .sh for linux

AKA [install]/examples/annotation_and_visualization in an installation. If you edit the file you will see options like input directory.

You can also run object reclassifier model trainers and it will do something similar as a pre-step. Note this method has more diversity in the way objects are ‘chipped’, some preserve aspect ratio and crop a square box for instance, other use box as-is.

1 Like
5

Ah yes, thank you, @matt_dawkins, this is indeed exactly what we want! We’ll dig into the script you linked to, and we’ll let you know if we have any questions or issues.

Thanks again!
Zach