Macquarie University neuroethologist Andrew Barron and his colleagues have been looking at how bees play the equivalent of poker machines (colloquially known as ‘one-armed bandits’). Odd though that may sound, this is research with a very serious purpose.
WATCH: Professor Andrew Barron presents a TED talk explaining why he is building a model of the bee brain.
In an article published in Proceedings of the Royal Society B: Biological Sciences, they show what bees most likely use to weigh probability. This has potential applications in several surprising areas.
Foraging honeybees need to gather concealed pollen and nectar from different flower species. These flowers vary in the amount of pollen and nectar available, and even in whether they will contain any.
The bee does not know what the probabilities are, but it has to rank the available options.
Bees learn remarkably quickly and can store information for the rest of their lives after doing a task as little as three times.
This kind of problem is known as a ‘multi-armed bandit’ task. There are multiple options available, all with unknown potential for payouts. How the bees choose has real-world consequences for them. It also has consequences for the broader thrust of Barron’s research.
The researchers presented honeybees with a learning task using a five-armed bandit – different coloured feeders with different probabilities of reward and punishment – to see how they solved multiple comparisons. The phrase ‘learning task’ is accurate. Bees learn remarkably quickly and can store information for the rest of their lives after doing a task as little as three times.
One solution to the five-armed bandit problem is to find the option most likely to give a reward and choose it every time (probability maximising). For example, if a bee learned that blue pays off 60 per cent of the time and yellow the other 40 per cent, and could rank and compare these alternatives, choosing blue every time would give the greatest return.
Alternatively, she could simply learn the reward properties of each option and choose according to those (probability matching). Trained bees would choose blue 60 per cent of the time, and yellow for the remainder. They would still show a colour preference, but without comparison or ranking being involved.
Both these strategies have advantages, depending on the situation, Barron explains. Where a decision is based on probabilities of different options that are known up front, probability maximising offers the greatest possible reward. But if the decision must be based on reward probabilities that have to be estimated from sampling and/or change during sampling, probability matching is a better solution to the trade-off between making the most of what is available and exploring other alternatives.
"In situations where a foraging animal needs to sample a range of options while harvesting from what appears to be the best choice, probability matching may be a better strategy," Barron says.
Reward and punishment
Barron’s experiment was designed so the bees could use either approach. Two groups of bees both had 18 training trials. In each training trial, one particular feeder colour rewarded them with a sugar solution, and another gave them quinine, which is not a bee favourite.
Jackpot: Confronted with a five-armed bandit, bees matched their choices to the odds of a reward, the research found. Image credit: AppAdvice
The study involved five different colours that gave a variable proportion of reward and punishment, but reward and punishment colours were consistent.
The trained bees then performed three more tests. In the learning test, bees were given the colours that had always been rewarded and always punished in training. In the transfer test, they were faced with a colour combination they had not seen in training. The novel test featured a new colour and a colour that had been rewarded in half the training trials.
Bees display evidence of meta-cognition: a bee without enough information to complete a task will stop rather than make a choice without information.
They picked the colours they had learned more often than the new one.
"The results indicate that with a five-armed bandit, bees matched their colour choices in tests to the odds of a reward in the training exercises – that they learned the properties of each choice, and did not compare between options," Barron says.
"The speed with which they learned and the effectiveness of this strategy in foraging may mean they seldom need more complex strategies."
Tiny brains, big concepts
All this is interesting in itself, but it has a greater importance in the context of Barron’s work – both studying bee cognition and building a model of the way a bee brain functions as an information flow network. And modelling a bee brain is only a first step.
Clever critters: Professor Barron (right) says bees are robust, reliable, and amazing navigators over very large distances – all current challenges in autonomous robotics.
All brains are made of neurons – about a million for a bee, about 80 thousand million for a human – and they all make connections in much the same way. Bees are Barron’s first step towards learning what intelligence is.
“Bees are capable of abstract learning, and they display evidence of meta-cognition: a bee without enough information to complete a task will stop rather than make a choice without information," Barron says. "The obvious implication is that they are capable of knowing they don’t know something.
“How can an animal with just one million neurons do that? It forces us to rethink our assumptions about the minimal computational architecture that could do this.”
Barron and his colleagues have modelled the neurons that connect when honey bees learn an abstract concept – and, he says, can do the same with parts of mammalian brain.
“To be able to translate what we’ve learned from bees as an hypothesis to help us analyse the mammalian brain, the human brain – that’s the value of the work I’m doing.”
Bee-inspired drones of the future
But there’s more.
“We can take insights from these models and translate them directly into technological applications," Barron says.
"Bees have evolved for millions of years to be fantastic autonomous behavioural control systems. They’re robust, they’re reliable, they’re amazing navigators across very large distances. These are all current challenges in autonomous robotics, yet the bee is doing it with incredible computational efficiency.”
- How critical can you be in online reviews without being sued for defamation
- Scam slam: how to keep identity thieves at bay
He foresees autonomous drones modelled on bee cognition being used in exploration, agriculture, mining, and disaster recovery. It is, he says, “the holy grail for so much of robotics, and bees have solved it with a brain the size of a sesame seed".
“What interests me is the capacity for safe robotics," Barron says. "If we’re starting our robotic system on the basis of a deeply understood system like the bee brain, we have a system that is more intrinsically understood, and therefore potentially safer than some other approaches.”
For those applications, we need to know how bees make their foraging decisions, so autonomous drones can forage for minerals, or even for disaster survivors. Quite a payout from the five-armed bandit.
Professor Andrew Barron is a neuroethologist in the Department of Biological Sciences at Macquarie University.
