The combination of AI-powered software and drones is incredibly effective for the environment. In this first in a two-part series, read how San Francisco researchers use drone imagery to clean up the San Francisco estuaries.
Researchers are using drones, machine learning algorithms, to tackle trash
By Jim Magill
(Part one of a two-part series on the use of drone images and machine learning software to cleanse the environment. Part two will examine a program run by the Danish Ministry of Climate Change that uses aerial drones to take pictures of polluted waterways and unmanned sailing robots to remove rubbish remove from the water.)
Researchers in California are combining images captured by drones with the latest artificial intelligence (AI) technology to solve an age-old problem. You will find plastics and other trash strewn along the banks of creeks and creeks before it can be swept away to add to the growing pollution of a cove or ocean.
Funded by the California Ocean Protection Council, the California Department of Health, and the U.S. Environmental Protection Agency, the San Francisco Estuary Initiative (SFEI) flies a DJI Mavic 2 Pro drone over designated areas along streams and creeks across the state to capture multiple images. Using machine learning tools developed by software company Kinetica, these images can be analyzed in real time to identify trash pieces much faster and more efficiently than more traditional methods.
Tony Hale, Program Director of SFEI, initiated the drone flight project to modernize the institute’s existing garbage detection program, which previously relied largely on people with waders walking along the riverbank.
“We thought, ‘This is very old-fashioned. Is there anything we can do to demonstrate what is possible with the use of new technologies? ‘”, He said. “By picking up a drone, we knew we could cover more space faster with fewer people, which translates into time and money savings.”
Another benefit of using a drone instead of a team of people on foot to survey an assessment area is that researchers can visit the same location multiple times. “Instead of being able to monitor a particular site once a year or twice a year at most, you can go out much more often to get a better picture of what’s going on,” said Hale.
The drone flights usually take place over assessment areas that can be up to several hundred meters long and up to 100 meters wide. The drone pilot typically flies the UAV at an altitude of around 100 feet, high enough to stay above the tree line but close enough to the ground to get sharp and easily analyzable photographic images.
At the beginning of the program, Hale said SFEI first flew a DJI Phantom but then switched to the Mavic 2 Pro. “We found the Mavic 2 Pro to be much more reliable. It was smaller, more portable, easier to use, and a reliable flyer, ”he said. “The Phantom has had some battery life issues. The Mavic 2 Pro was a real workhorse for us. “
The researchers had another reason to choose the Mavic 2 Pro. By using the same relatively inexpensive drone, other agencies that do not have a large budget to carry out garbage identification work could duplicate SFEI’s program with similar results. “We wanted to demonstrate the functionality of using types of vehicles that could not be bought off the shelf for a lot of money,” said Hale.
For the same reason, the project developers decided not to add any additional sensors or modify the drone itself in any way. However, they chose Esri Site Scan, a premium software package designed to optimize planning, data collection and data distribution functions. “It made our job easier so we didn’t spend all of the time planning flights and maintaining the data,” said Hale.
Kinetica’s partnership with SFEI also helped move the project forward. “We had some challenges refining our algorithm,” said Hale. “Partnering with Kinetica has helped make the ongoing iteration of machine learning tools and algorithms easier so that we can reach conclusions more quickly about the right direction for improvement.”
Nick Alonso, director of global solutions engineering at Kinetica, said the company’s machine learning algorithm accelerates the workflow in collecting, sorting and analyzing the thousands of images collected by the drone to locate and identify individual pieces of trash.
“We enable these users to hook up directly to these streaming feeds, stream them directly to a target table, and once they reach that table they create a seamless end-to-end machine learning workflow,” he said. Without the Kinetica software, analyzing the image data can take “days, weeks, or even months”.
Rapid data analysis is vital when it comes to identifying trash for later pickup, he said. A variety of factors could likely move the garbage from where it was first spotted by the drone: environmental factors like rain, wind, and wildlife, as well as human intervention, including vehicle and pedestrian traffic.
“Even within five or six hours, the likelihood that this piece of junk will be in the same area is pretty slim,” Alonso said.
SFEI’s work with drone-generated images and AI software to identify relatively large pieces of trash, such as plastic bottles, has led to the launch of another project to identify much smaller pieces of debris from the air. With funding from the California Department of Health, the institute developed another algorithm that can be used to analyze images collected by drones to detect cigarette butts.
Using the new algorithm and other software tools from partners Kinetica and Oracle, as well as a drone that takes images from a height of 60 feet, researchers were able to successfully identify cigarette butts on hardscape surfaces such as asphalt parking lots 90% of the time. The institute is currently working on refining the learning algorithm to enable the detection and identification of cigarette butts in other likely locations such as parks and along sidewalks.
Miriam McNabb is editor-in-chief of DRONELIFE and CEO of JobForDrones, a marketplace for professional drone services, and a fascinating observer of the emerging drone industry and the regulatory environment for drones. Author of over 3,000 articles focusing on the commercial drone space, Miriam is an international speaker and recognized figure in the industry. Miriam graduated from the University of Chicago and has over 20 years experience in high-tech sales and marketing for new technologies.
For advice or writing in the drone industry, email Miriam.
Subscribe to DroneLife here.