Using AI to optimise farming
On 4th November 2025, the BBC published the following article: AI used to spot weeds growing in farmers' fields.
This is the first entry in my planned 'AIdentifier' series, where I dig into headlines about AI achieving useful things in an attempt to find out if generative AI has any uses at all.
Introduction
The BBC article links off to an article on the Bosch website, as they were using Bosch cameras and Bosch machinery to do the spraying. Unfortunately, neither the BBC nor the Bosch articles have much in the way of detail on how the work was carried out.
Both mention that the work was a joint project with Rothamsted Research in Harpenden. Now, while Rothamsted Research haven't published any papers (as of 13/11/25) on this work in particular, they have published another article on an AI-based framework for early detection and segmentation of green citrus fruits in orchards. Both works are around using AI object detection in images, and so I think it's fair to assume that the methods involved in both projects are similar.
From now on I will only be talking about the work looking for citrus fruits in orchards.
Background
The researchers wanted a way of automatically counting the fruit on citrus trees while they're still growing. Why? If you can keep track of what fruit you have where, you can plan watering, harvesting and spraying of pesticides and fungicides better. That way, less water is used, less fruit is wasted and the environmental impact of biocides is smaller. It also all results in more money for the farmer.
The problem in this situation is that unripe citrus fruits are small and green, making them rather hard to spot among all the similar-coloured green leaves. Other researchers have tried to do this before and have had some pretty good success rates, but they've all relied on taking really close-up images. In reality, you need to be able to take big, wide-angle shots of a whole tree for this to be anywhere near efficient.
The point of this research was to find a way of counting fruit in these zoomed-out photos.
Methodology
They had two types of AI that worked together: A convolutional neural net (CNN) called Cascade Mask R-CNN, and a transformer called Multiscale Vision Transformers version 2 (MViTv2). It then had a custom AI that could identify and count the fruit based on the outputs of the CNN and the transformer.
CNN stands for Cable News Network.
CNN stands for Convolutional Neural Network. They are the thing that everyone defaults to using when doing machine vision tasks. Identifying fruit in pictures counts as machine vision.
Whenever you see the word convolution, you know two things are being slid past each other. For example, here is a graph with two square waves.
To convolve them, you slide one past the other. Mathematically, you apply an offset to one of them. Increasing that offset pushes it along to the right, making it slide. The output is the area that is underneath both lines, for each value of the offset.
A convolutional neural network, then, is sliding something over something else. In the case of CNNs and image processing, you're doing some pre-processing to the image to make it easier to do what it is you want to do.
Imagine a picture that is 100 × 100 pixels. Each pixel is three colours: R, G and B. So we have a matrix of 100 × 100 × 3.
If you wanted to feed that into a neural network, you'd need 30,000 inputs. Then, every layer of your network would need 30,000 neurons (if your network is fully connected, which most are). That's an awful lot of calculations to do, for a tiny tiny picture.
A CNN is a mini neural network that just looks at a small portion of the picture at a time, and gives you just a couple of numbers to sum it up. Then you can feed that summary into your neural net and have a lot fewer inputs to deal with.
This stage of the algorithm is called classification. The point is to work out what bits of information you've got are important, and which aren't.
For example, a CNN could look at a 5 × 5 pixel sample of our picture of a lemon tree. Assume that the things you're interested in are the average RGB values for the area, if there are any edges, and if there's anything that could be classified as fruit, leaves or mould. It looks at what is in that sample and returns a summary: It's mostly blue, no edges, fruit, leaves or mould.
Then the CNN can move onto the next 5 × 5 segment: Mostly blue, has an edge, might have some fruit or leaves.
I've come up with seven parameters it cares about. So, once the CNN has finished the whole picture, the output would be 20 × 20 × 7 = 2,800. A lot smaller than 30,000!
That input would be fed into your main neural net to do the actual job you're trying to do, or into another CNN for more classification.
One of the nice things about a CNN is it is parallelisable: Rather than having one CNN do each 5 × 5 segment in turn, you could have 20 of them running at once. This makes it run faster. You could do the same thing for the next neural net that comes after: one neural net could only look at bits that have lots of green because that means its leaves or fruit. This neural net wouldn't need the data about fruit, leaves, mould or edges. Another net could just decide if the fruit that got tagged is a fruit or just a leaf: It could ignore edges and RGB values. This is how classification can help with what is important and what is not.
A transformer is another way of achieving what a CNN does. Rather than having a CNN go over the picture section by section, a transformer looks at the input as a whole.
A small group of pixels is turned into a long string of numbers, called a vector. This vector represents the group as a whole, like how the CNN groups things (but the exact things represented in the vector may be different). Then, a transformer goes a step further. The transformers run in parallel, like the CNNs. That means other vectors have popped up at the same time. Each transformer looks at the other vectors nearby and uses that information to help decide what is important about its own vector.
Perhaps this vector reports quite a high amount of blue as well as green. All the surrounding vectors are green without much blue. In that case, the blue is probably just noise or a reflection. A vector five over also reports some blue, but it's so far away it probably isn't relevant. The transformer decides its own weird blue value is unimportant and suppresses it.
Alternatively, the transformer might be wondering how important the edge it's just detected is. If the vector above and below both report edges then it should probably look past them and ask the next couple of vectors out about their edges too and so on. Now it is paying attention to quite distant vectors, while perhaps ignoring those quite close by to the left and right because they didn't report edges, and overall deciding that its own edge is rather important.
The fact it can look for multiple things (colour, edges) makes it "multi-headed". The act of looking around at other vectors is called "attention". And the ability to take into account distance is through "positional encoding". These three things are key to making a transformer.
The GPT of ChatGPT stands for Generative Pre-trained Transformer. Just because ChatGPT uses a transformer, and this lemon-hunting research uses a transformer, does not mean this research uses ChatGPT or generative AI.
The researchers took lots of photos of citrus trees. They took some really close up, and some further away. They used the close-up ones for training their AI, and the far away ones for testing it in a realistic scenario.
To get the training data, they needed to label all the fruit in the pictures. As they had 399 images, that's a lot of labelling. They turned to a tool called Segment Anything, which has been developed by Meta. This is a foundational model that can be used to highlight things in pictures. It uses natural language processing to understand what the user wishes to have highlighted, then uses a vision transformer to find relevant things and highlight ("segment") them.
A foundational model is a basic AI model that has been pre-trained on a massive dataset so it can be used for a wide range of tasks. The idea is that someone can take a foundational model that understands the basics, and train it on specific data to allow it to do a particular task. The foundational models aren't that useful on their own, especially given the huge amount of energy required to train and run them.
- The natural language processing portion of Segment Anything, which allows the user to prompt it, uses a model called CLIP, which was created by OpenAI.
- Segment Anything needs huge amounts of training data, for the natural language processing and for the image processing. The creators of Segment Anything state that their dataset is fully licenced, so hopefully it was acquired legally.
- Segment Anything requires an image to be encoded before use, but this only needs to be done once per image.
Verdict: No. It needed a lot of training data but they claim to have got it through un-dodgy means. It is not a power-hungry foundational model suited to any task that needs a whole data centre; once an image has been encoded, the model using it can be run on a single CPU and produce an answer in 50 ms, meaning it is relatively low power.