Bees in two-armed bandit situations: Foraging choices and possible decision mechanisms

In multi-armed bandit situations, gamblers must choose repeatedly between options that differ in reward probability, without prior information on the options' relative profitability. Foraging bumblebees encounter similar situations when choosing repeatedly among flower species that differ in food rewards. Unlike proficient gamblers, bumblebees do not choose the highest-rewarding option exclusively. This incomplete exclusiveness may reflect an adaptive sampling strategy. A cost-benefit analysis predicts decreased sampling levels with increasing differences in mean profitability between the available food sources. We simulated two-armed bandit situations in laboratory experiments to test this prediction. Bumblebees (Bombus terrestris L.) made 300 foraging visits to blue and yellow artificial flowers that dispensed sucrose solution according to seven probabilistic reward schedules. Reward schedules varied in profitability differences between the two feeding options. As predicted, the bees specialized more on the higher-rewarding food type (and thus sampled the alternative less) when the mean reward difference between the feeding options was larger. Choice ratios of individual bees were linearly related to the reward ratios they had experienced. It has been suggested that the behavioral mechanism underlying incomplete exclusiveness may involve simple rules of thumb that do not require long-term memory. However, the bees' response to recent foraging experience (rewarded and non-rewarded visits) differed between the beginning and the end of observation sessions and between treatments. Simulations of the Rescorla-Wagner difference learning rule reproduced the main trends of the results. These findings suggest that the observed incomplete exclusiveness results from associative learning involving long-term memory.