Crop loss is a huge problem for sustainability and food security. I’m using AI and computer vision to help fix it.

When we think about food waste, we usually think on the scale of not finishing our lunch. In reality, agricultural waste is a much bigger problem. An estimated 20% to 40% of crops are lost to pests and diseases, which affects our food systems and the planet; greenhouse-gas emissions from food that is never eaten account for 6% of total emissions.

I knew none of this about five years ago. I was working in bioinformatics, developing an AI-based platform that assists oncologists in diagnosing cancer, and I knew so little about agriculture that I couldn’t even keep a houseplant alive.

But someone who ran a commercial greenhouse reached out to discuss how AI could solve those crop-loss challenges. I quickly saw the opportunity to translate my work in caring for human health into caring for plant health, and Fermata was born.

Using AI and computer vision where humans fail

Right now, many farmers try to detect pests and diseases early by sending workers into the fields daily to look at every leaf on every plant for early signs of issues. But people get tired, get distracted, and overlook the tiny changes that suggest an issue.

It reminded me of the specialists who review X-rays all day and miss early signs of cancer. We were using AI analysis of imagery to solve that problem, and I felt the same solution could apply here.

Initially, we thought about building a robot that would move through the fields but realized we didn’t have the expertise to achieve that — and that a simpler solution existed.

We developed the Croptimus platform, which uses standard security cameras in greenhouses and fields to take pictures of every single plant several times a day.

Our artificial intelligence processes these images to detect any abnormalities. If it does, it notifies farmers via our app and suggests what it thinks the issue is. Using data science, we can also provide an overview of what’s going on in a facility at large, whether problems are increasing or decreasing, and whether treatments are working.

With this technology, farmers need less labor and can identify issues sooner than they would with humans, meaning they can apply fewer pesticides. This saves a lot of money and helps produce more sustainable and healthy food.

Fermata’s software uses imaging to evaluate a plant’s health and detect any issues.

Doing things manually while we build the dataset

One of the biggest challenges we had to overcome was establishing ground truth to build a reliable dataset to train our models. Every agronomist, or crop scientist, has their own opinion and will make mistakes as well. We needed to not only compile a decent dataset from the ground up but also build machine-learning models that could adapt to mistakes.

We did a few things to help. First, we built a research lab, where we grow plants, infest them with different things, and record them on videos. We also hired an in-house team of agronomists to help us label these images.

Additionally, we launched our product to the public before it was automated, doing identifications manually and encouraging farmers on the app to give us feedback. This helps us better understand the problem and gives us a more robust dataset. Even as we’ve shifted to relying on AI for identification, this feedback loop helps us continually improve our models.

Building relationships to find future possibilities for the technology

It’s so important that any technologist trying to solve a problem in a new-to-them industry — especially more conservative fields like agriculture — stay humble and develop real relationships with the people they’re hoping to help. If I marched in as a tech-industry outsider and told these farmers who have been doing this for decades that I could teach them how to operate and be more efficient, it wouldn’t work.

Instead, I work on building relationships and trust within the industry. I approach it from the perspective of wanting to learn from my customers and understand how my tech knowledge can help them.

Ultimately, this has helped me see even more potential in what we’re doing. I’ve learned that there’s so much visual data in farming, from understanding whether bees are pollinating to seeing how employees are treating plants. The vision that Fermata has is to build a new visual-data layer in the agricultural industry that helps all stakeholders — from farmers to people who sell fertilizer to pesticide companies — be more efficient.