New AI/ML technology has shown to reduce natural disaster assessment timelines from days to seconds thereby saving lives
Mountain View, CA (April 17, 2024) — In an effort to improve natural disaster responses, the Defense Innovation Unit (DIU) and the National Guard Bureau (NGB) partnered to prototype commercial AI and machine learning (ML) technologies to enhance natural disaster response awareness and analysis solutions. This technology is now going into production with a capability developed by X Development, LLC (known as X, part of Alphabet), called Bellwether.
NGB operates as a federal agency responsible for the administration of the National Guard and oversees the manning, training and equipping of Guard units across the 54 states, territories and the District of Columbia. While the Guard’s first mission as the primary combat reserve of the Army and the Air Force is to fight and win the nation’s wars, another core responsibility, and one with which people are likely most familiar, is assisting states and communities with domestic incident response. Domestic incident awareness and assessment (IAA) requires multiple sources of ephemeral knowledge and a complex set of imagery analysis. Additionally, as IAA technology evolves, new commercial AI/ML-supported situational awareness tools could help the Guard better serve emergency management partners during domestic operations (DOMOPS).
“It’s all about saving lives in our communities,” said COL Brian McGarry, Chief of Operations, Plans, and Training Division for NGB. “Right now, our analysts have to spend time sorting through images to find the ones that cover the areas most affected by natural disasters. They then have to correlate those images to surrounding infrastructure, label all the relevant features, and only then can highlight the significant damage and send it forward to first responder teams. Using AI/ML to do the routine tasks of georectification, identification, and labeling will greatly speed up how quickly we can get important information to the folks that need it most.”
To enhance natural disaster response awareness and analysis, in 2022, the DoD issued a solicitation in partnership with NGB calling for AI/ML solutions to unclassified processing, assessment, and dissemination of electro-optical aerial images of varying nadirs (collection platforms can range from handheld devices and/or aircraft mounted instruments). Sixty-five companies responded to DIU’s solicitation, ultimately resulting in prototype awards to two companies. X Development, LLC’s Bellwether group is finalizing its prototype contract to develop solutions for processing images from aerial platforms to detect and label critical infrastructure and key resources for the humanitarian assistance and disaster response mission of the NGB.
“Formerly serving in the California Air National Guard, I can attest to the significant amount of time that can be spent in combing over large amounts of imagery in domestic response scenarios,” said Dr. Nirav Patel, DIU’s Senior Scientist & Program Manager for Remote Sensing. “The technology we’re prototyping allows us to make sense of overhead aerial imagery taken from different angles, which can reduce limited response time to our local communities who find themselves in emergency situations with imminent needs.”
The technology developed by X automatically ingests images taken by the Civil Air Patrol (CAP) and uses AI to query geospatial assets. The technology will then identify the positioning of the original CAP image and reveal critical infrastructure contained in the snapshot, allowing the National Guard to rapidly coordinate relief efforts. The tool finds matches even when the source image contains a heavy amount of damage — such as after a tornado — or the landscape is dramatically different - such as after a flood.
The technology developed by X automatically ingests images taken by the Civil Air Patrol (CAP) and uses AI to query geospatial assets.
“First responders spend a lot of time understanding damage, both where damage has occurred and to what degree that damage presents itself. This process is time-intensive, critical for many downstream processes, and in many cases automatable,” added Ritwik Gupta, Deputy Technical Director for Autonomy at DIU and an AI researcher at UC Berkeley specializing in AI for disaster response. “We’ve demonstrated the power of machine learning for damage assessment at DIU in the past with xView2, but many challenges remain to generalize to any real-world situations; the work with X gets us much closer.”
The new tool takes just minutes to geolocate entire batches of images from Civil Air Patrol flights, saving hundreds of human hours per disaster event. The tool runs 24/7, with no ‘human in the loop’, freeing up critical downstream resources for relief efforts.
“Before the Bellwether team built this solution, we were spending, on average, 12 hours to analyze images from a single Civil Air Patrol flight,” according to Master Sergeant Kenneth Houck of the Georgia Air National Guard. “For a single image, it could take several minutes just to orient ourselves, find the infrastructure of interest, and label it for operational purposes. With this technology, we are literally moving from days of work to mere seconds. This means the Guard can now react faster than ever in the critical moments after a natural disaster."