Integrating xView3 Machine Learning Algorithms Into the SeaVision Maritime Domain Awareness Platform is Helping to Detect Illegal, Unreported and Unregulated Fishing at Global Scale
August 1, 2023 (Mountain View, CA)—The vastness of the ocean requires new tools to provide data analysis to detect and characterize “dark” vessels operating without an automatic identification system (AIS) or a vessel monitoring system. Often, these dark vessels are involved in illegal activity and do not broadcast their position or appear in public monitoring systems, allowing them to evade detection and enforcement efforts.
In late 2021, the Defense Innovation Unit (DIU) ran a machine learning prize challenge called xView3 in partnership with Global Fishing Watch (GFW) to find the best open-source AI algorithms that could use Synthetic Aperture Radar (SAR) imagery to detect and characterize dark vessels likely to be engaged in illegal, unreported, and unregulated (IUU) fishing activity. Since announcing the winners of the xView3 Challenge in January 2022, DIU worked with its military and civilian partners to deploy the winning open-source algorithms into U.S. government maritime domain awareness platforms to better detect and characterize dark vessels.
“The U.S. Coast Guard was a developmental partner for xView3 and has found tremendous use for the developed set of algorithms. From tracking IUU fishing to helping with search and rescue, xView3 has enabled us to get a better view of large portions of the ocean in an automated fashion,” said U.S. Coast Guard D11 Commander, RADM Andrew Sugimoto. “Traditionally, sailors manually review all obtained intelligence such as satellite or aerial imagery for a vessel of interest. With the accurate automation provided by xView3, analysts can now simply filter detected vessels to find the ones of interest.”
xView3 effort has involved building an end-to-end pipeline that retrieves real-time imagery from the European Space Agency’s Sentinel-1 SAR constellation, efficiently running that imagery through the winning xView3 models, and integrates the outputs into SeaVision’s platforms such as the SeaVision (Navy, Department of Transportation), and Proteus (Naval Research Lab), and several systems specific to individual Combatant Commands.
Example of xView3 Model applied over European Space Agency Sentinel-1 Synthetic Aperture Radar Imagery; blue bounding boxes indicate vessel detections (Source: European Space Agency)
Over the past year, the winning xView3 models were executed over the vast majority of Sentinel-1 imagery and are currently being output to maritime domain awareness platforms via joint funding partnerships with the U.S. Coast Guard, National Oceanographic and Atmospheric Administration, USINDOPACOM Maritime Security Initiative, and USAFRICOM. Applications include IUU fishing vessel detection and transshipment prevention.
USAFRICOM in particular has observed substantial utility from the model outputs, stating that, "xView3 has provided USAFRICOM previously unavailable insight (both in accuracy and volume) into dark vessel activities in regions of interest around the continent of Africa. This information can now be used to help inform and shape reporting and partner engagements on these critical activities in the maritime domain.”
While users from DoD, USCG, and international partners have observed promising results from xView3 to date – including motivating several concrete actions that have improved maritime security – much work remains to combat IUU fishing using remote sensing data. Fortunately, collaboration with the private sector, NGO networks, and international partners has enabled promising work on top of the foundation built by xView3.
“The xview3 challenge was a great opportunity for us to advance state-of-the-art technology for detecting dark vessels,” said Paul Woods, chief innovation officer at Global Fishing Watch. "We are leveraging the learnings from the competition to advance our efforts in creating a free and open map that reveals all industrial human activity at sea."
In the end, fighting the environmental and economic scourge of IUU fishing and tracking illegal shipping will require a concerted and collaborative effort amongst nations, NGOs, and the private sector. xView3 is a prime example of how the U.S. security community can lead the way in making progress on these objectives while also addressing some of its own intelligence needs for pennies on the dollar.
The xView3 dataset was published at the 2023 Conference on Neural Information Processing Systems. And for more information about DIU's xView Computer Vision Challenges, visit here.