Francis Freire works at the governmental agency Geological Survey of Sweden (SGU) and was interviewed by the ODF Sweden Team Members Yixin Zhang (Department of Applied IT at Gothenburg University, Responsible for WP 2 Continuous evaluation & innovation) and Felicia Ridderbjelke (Community Curator at ODF) for his contribution to the project PLAN-SUBSIM. The project is a national implementation of a PLatform for ANalysis of SUBSea IMages to develop methods for monitoring and analysing the status of the subsea habitats. The project will leverage existing methods, knowledge and infrastructure in the field of subsea image analysis and implement these for applications in marine resource management.
Francis is a marine geologist who surveys the Swedish coastal waters using hydroacoustic techniques and collecting sediment samples and high-resolution underwater images to produce full coverage benthic habitat maps. Analysing these data is extremely time consuming and ODF will therefor use our machine-learning approach to speed up and scale out the analysis of the surveys.
Yixin Zhang: Can you tell us a little bit about yourself and the project?
Francis Freire: I’m a marine geologist by background and use marine geophysical methods to study seafloor geology. Together with people from my department, we survey the Swedish coastal waters to determine what type of seafloor can be found where. We do a habitat mapping based on the data that we collect for the survey and that’s basically how we came in contact with the ODF. Swedish Geological Survey is the government agency assigned to do the geophysical surveys in the territorial waters within Sweden and we wanted to effectivese the surveying and to characterize the sea for geology.
When we go out into the field, we use different geophysical equipment and have our own boat. The trips can be around 2 weeks and we are about 10 people on the boat to map a particular place. We also do geophysical sampling, which means that we use acoustic multi-beam systems to determine and characterize the seafloor. We use other acoustic systems to investigate what’s underneath the seafloor, the subseafloor. We take samples of the sediments in the coastal waters and do a lot of chemical analysis to determine the level of toxins.
I am probably more interested in the habitat mapping part of the seafloor, the part where we record underwater videos and acoustic data. When we collect the data, we use machine learning and to come up with habitat maps of the area that we survey. For example, we had a big project in Hoburgs bank, which is on the Baltic part of Swedish water. It was a very comprehensive survey and we were there for almost two months.
Yixin Zhang: I once read that we actually know more about planet Mars compared to the seafloor. How little or much do we actually know about the seafloor?
Francis Freire: That is probably true. We try to map as much as we can and we already know quite a lot about Swedish waters. But overall, there are still a lot of areas that are not mapped. The biggest gap is probably not in Europe or in the US but in the bigger ocean areas, for example in the middle of the Atlantic.
Yixin Zhang: What are habitat mapping and geophysical surveying?
Francis Freire: Habitat mapping is when you try to identify all the habits that can be found in the waters. There are around 40 different classified habitats depending on the coverage of the area. An example of this is the project called Helcom in the Baltic Seawhere where the key component is habitat mapping. There are so many ways to define the habitats and I think it is our ethical mandate to also create this to follow this directive and create habitat maps.
Regarding geophysical surveying, that is when we go out into the field and collect geophysical data, mostly acoustic data. We send out a caustic pulse which will bounce back from the seafloor to the boat and give us information about the seafloor. We also measure the amount of sound that comes back which gives us an idea of the seafloor material. Regarding this acoustic system, we send out 10 or 15 samples for every square meter. We then get very dense data and information for even up to a decimetre.So every point one decimeter in the seafloor, we get an acoustic pole. Wa also collect underwater videos and photos and sometimes also just collect “real samples” by just going out in the ocean and grabbing anything that is there to see what materials can be collected.
Finally, we then use machine learning to interpolate all this data to get a more clear picture of the habitat maps. But briefly, from the acoustic data, we collect information for the whole area. But for the underwater videos, pictures, and samples, we only receive information for specific points. Then we interpret and combine all this data.
A report that describes the habitat mapping process: in the HELCOM project: High-resolution benthic habitat mapping of Hoburgs bank, Baltic Sea (2020), Gustav Kågesten, Finn Baumgartner, and Francis Freire. Available at:
A figure from the report (Kågesten, Baumgartner, and Freire, 2020) illustrates ocean surveying and the instruments.
Figures from the report (Kågesten, Baumgartner, and Freire, 2020) illustrate the photo mosaic of seafloor habitat.
Yixin Zhang: What motivated you to join the Subsim-project?
Francis Freire: I think the motivation for the whole team was that we wanted to facilitate the processing of our collected underwater pictures and videos. The idea behind the project is to use an algorithm and a fast computer to do the identifying work for us. To collect our data and feed it all into a computer would save us a lot of time.
Yixin Zhang: Considering all the data that you are gathering, processing and analyzing, how many hours do you actually invest in analyzing a five or ten-minute long video?
Francis Freire: For example, we have collected close to 600 sampling points around the Hoburgs bank. For each of the photos, we need to find a way to upload it to our software and then identify the percentage for everything that covers the seafloor. For example, how large is the percentage of mussels? It probably takes around 30 minutes to one hour to analyze one picture. Then you also have to do some cross-checking afterward, so another person clarifies that the identification is right.
Figures from the report (Kågesten, Baumgartner, and Freire, 2020) illustrate camera and sensor set up, and underwater images mosaic.
Yixin Zhang: If I understand it right, you monitor 600 different observation points. How many pictures do you take per site?
Francis Freire: Depends on the area. The more diverse areas require more pictures. When the area is small, the resolution is high and the pictures become very clear. Before we start to photograph, we receive information from our multi-beam system or backscatter system that can give an idea about the depth.
Yixin Zhang: What challenges do you expect for similar projects in the future?
Francis Freire: For now, I think the biggest challenge is to be able to use artificial intelligence in our algorithm so we can make the identification even faster. If the algorithm can identify with good enough confidence, we can monitor more and bigger areas by feeding more pictures into our software and then process it effectively. This algorithm can then identify how much percent of the area was covered by for example algae so that we later can do the habitat mapping for the sites.
Yixin Zhang: Has machine learning been used in the work you do?
Francis Freire: I do not think we have enough manpower to do that just yet. We have been looking for some partners who can help us with this for a long time now, so we were happy to get in contact with ODF.
Felicia Ridderbjelke: From all of the photos you have collected, is there anything in particular that has surprised you?
Francis Freire: One interesting finding is that many seafloor covers are not permanent. The seafloor changes depending on the season. However, it is quite repetitive and looks similar for every season. We have also found a high percentage of mussels in Baltic areas, that differ from the species on the West Coast. Something that is not part of the project, but that interests me, is some of the shipwrecks that we have found.
Felicia Ridderbjelke: During which months have you done these trips?
Francis Freire: We started with the project in 2016 and our survey season starts from April until October.
Yixin Zhang: What are the challenges with your ocean trips?
Francis Freire: Well, you work 24 hours a day and if you get the night shift you work from 8 pm to 5 am. There was this one time, I think I threw up like three times during one trip. Also, the problem with our ship is that sometimes it makes a lot of noise which makes it hard to sleep. Sometimes the boat is shaking too much which makes it hard to eat. But when you have collected the data and produced the product, you get really satisfied and happy.
