– a national implementation of a
PLatform for ANalysis of SUBSea IMages
Table of Contents
Latest in PLAN-SUBSIM:
- 2 March 2023, implementation demonstration #2 done (Milestone 2.2), “Towards a national Platform for Analysis of Subsea Images“.
- 21 September 2022, user workshop #3 done (Milestone 1.3), focusing on methods for identifying invasive fish, using video from underwater observatories.
- September 2022, Be Geospatial AB joins the work in PLAN-SUBSIM. Be Geospatial AB will work closely with Geological Survey of Sweden on an operational tool for video interpretation, suitable for field activities.
- August 2022, PLAN-SUBSIM has started working and come some way on methods for identifying fish (and other moving species) with fixed observatories.
- June 2022, PLAN SUBSIM delivers first object detection models for benthic fauna and habitat builders. Bachelor thesis by Rasmus Thorslund, in Swedish.
- 17 May 2022, user workshop #2 (Milestone 1.2), together with SBDI, info & registration.
- 25 February 2022, visit to Medins AB in Mölnlycke. Medins will join our work.
- 20 January 2022, user workshop #1 done (Milestone 1.1), with reference group.
The coastal marine environment is under enormous pressure, from legacy of overexploitation and from the growing blue economy. A credible sustainable strategy needs balance the protection and the socio-economic development of near-coast environments. We need methods to monitor and analyse status and changes of subsea habitats and understand these changes and their causes.
Much of the technology used in research is mature and ready for a wider use by the industry and the public sector. The combination of camera-based sensors and image analysis resources allows us to extract information from marine environments with unprecedented quantity and quality. However, the abilities also impose a massive challenge to end users as they require infrastructure and expertise for data processing and analysis. Hence, big data approaches are instrumental for scalable and repeatable use of image-based information from subsea environments.
This project will leverage existing methods, knowledge, and infrastructure in the field of subsea image analysis and implement these for applications in marine resource management. Specifically, we will develop seamless linkage between existing data archives, thematic eLabs for data processing, HPC infrastructure, and data portals and thereby make valuable data and data products available to societal end users. The suggested activities (work packages) will include i) formulation of user specification, ii) implementation trials, and iii) scaling operations.
Work packages and timeline
Work package 1. User specifications
- M1.1 First user workshop [month 2]
- M1.2 Second user workshop [month 5]
- M1.3 Final user workshop [month
- D1.1 Specifications from users [month
Will be finalized after final user workshop, M1.3
- D1.2 Description of developmental cycles [month 6]
August 2022, descriptions for three teams, see D1.3 below.
- D1.3 Work teams established [month 9]
August 2022, three teams established: Team Seapens (Medins AB), Team Mussel banks (Geological Survey of Sweden), Team Invasive fish (University of Gothenburg).
The objective of WP1 is to identify and document the specifications for the system for new end user groups. Some major end users are already part of this project as partners or in the initial reference group, while additional new ones have already expressed interest in the approach, e.g. several County administrations. We will approach and consult these groups in specific user workshops. During these workshops we will describe the existing functionalities in our system with examples and thereafter discuss the customisation of the method to suit the needs of new users, including data archiving, citizen science-based training, model customisation and testing, high-performance computing, and any other potential analytical demands. The workshops will also be used to set up small work teams of 4-5 people, including system developers, data scientists, marine scientists, and managers. These groups will then start to translate the specifications into development cycles.
Work package 2. Implementation
- M2.1 First implementation demonstration [month 9]
August 2022, KSO Detector web app demonstrated, identifying four marine classes; Service deployment and cloud computing demonstrated on distributed Nordic cloud resources.
- M2.2 Second implementation demonstration [month 13]
March 2023, “Towards a national Platform for Analysis of Subsea Images”. First assembly of a national consortium around subsea image analysis for marine biology research and monitoring.
- M2.3 Final implementation demonstration [month 17]
- D2: Functional platform for image analysis [month 18]
In WP2 we will do implementation in use cases. Alla use cases will relate to specific priorities in the national research programmes, especially the programme for Ocean and Water. (The following two use cases will be complemented during work in WP1.)
- Use case on long-term monitoring of Baltic vegetation. SGU has more than 10 years of highly standardised and well-annotated transects with movie footage and associated data from several areas in the Baltic. Analysing these data manually is extremely time consuming and here we will use our machine-learning approach to speed up and scale out the analysis. This use case addresses the priority in the programme for Ocean and Water, to tackle challenges towards national goals.
- Use case on status assessments for the Marine Strategy Framework Directive (MSFD). SwAM is the responsible agency for monitoring Sweden’s marine environment, with SMHI as one of most important executors of the tasks. SMHI is also Sweden’s National Oceanographic Data C entre and by SwAM assigned as the data host for subsea image data. Implementing MFSD requires monitoring agencies to use new methods for analysing marine data, typically by using combinations of both traditional sensor data and image data. This use case addresses the priority in the programme for Ocean and Water, to implement European visions, directives and conventions.
Work package 3. Scaling and sustainability
- M3.1 First scaling up hackathon [month 15]
- M3.2 Second scaling up hackathon [month 18]
- M3.3 Third scaling up hackathon [month 21]
- M3.4 Final scaling up hackathon [month 24]
- D3 National system for subsea image analysis [month 24]
The objective of WP3 is to deploy an Application Programming Interface (API) to the analytical resources developed in WP2. The API will expose the machine-learning algorithms through HPC clusters and can be used by any party – commercial, public, or academic. We have the pilot version (KSD) available. The API will have improved documentation explaining for each of the exposed models how the algorithm works, for which species
and habitats it is trained, etc.