Understanding how associative learning is embedded in behavioural control

We want machines that are able to learn as quickly and effectively as animals. Learning in biological systems is not just acquisition of information, but the ability to change – improve – behaviour. A mechanism for improved action choice seems apparent for operant reinforcement learning, i.e., actions leading to reward should be increased in frequency or probability. It is less clear why or how Pavlovian associations [1], in which stimuli are paired with reinforcers independently of the animal’s actions, should alter subsequent behaviour [2]. Explaining this as simple stimulus substitution, as is still common in ‘behaviour-based’ robot architectures, is not in fact adequate to account for the complexities in behavioural expression seen in real animals, even simple ones such as fly larvae. Animals do not simply start to respond to the conditioned stimulus as though it was the reinforcer [3].
On the other hand, a cognitive interpretation involving explicit prediction of future reinforcement to make action choices, which underpins many vertebrate brain architecture models [4], seems implausible for miniature-brained animals. Standard computational accounts, in which stimuli or stimulus states linked to reinforcement acquire ‘value’, either assume a direct relationship of value to expressed behaviour [5], or rely on a complementary operant process to alter behaviour (the actor-critic architecture, [6]), neither of which can fully account for the behavioural data [7]. Moreover, such accounts generally neglect the actual details of behaviour execution, and their embedding in ongoing behavioural control, which may be crucial to structure the learning experiences that make acquisition so rapid and effective in biology.
To answer these questions, so that we can reproduce the flexibility of biological behaviour in
machines, requires fine-scale analysis of when and how an individual animal alters its behaviour as it learns under Pavlovian conditions (WP1 Objective 1.1 – To gain an in-depth understanding of the associative link formed in odour-taste conditioning). Ideally this should be in a naturalistic scenario for the animal (unrestrained, and with ecologically relevant stimuli), but still a situation where we can track precise motor output, test multiple individuals quickly, and have tight control over the stimulus and environmental conditions. Drosophila larvae are almost perfect in this regard, even before consideration of the fact that genetic tools will allow us to ‘dissect’ the underlying neural circuit in the behaving animal (see below). We thus propose to carry out an in depth behavioural analysis of a well-
established conditioning procedure in which larvae experience odours paired with appetitive or aversive tastants and are subsequently tested for their orientation in odour gradients under different conditions [8]. We will develop tracking systems that allow us to reconstruct entire individual trajectories and the precise body motion (peristaltic crawling, head bending and turning) of individual larvae, and use statistical learning methods to characterise the key changes to behavioural control that occur during learning (WP1 Objective 1.2 – To analyse the fine-scale dynamic behaviour of larvae during acquisition and expression of learning). These tracking capabilities and behavioural characterisations will further underpin the methods we can use to analyse neural circuits.
Our understanding of the behavioural changes during learning will be tested in an agent simulation that, by altering its moment by moment body motion, should reproduce the global trajectories and high level preference statistics observed in the larvae (WP4 Objective 4.2 – To test the model in closed loop behavioural simulations using identical experimental paradigms).
Associated Milestones (see summary table in section 1.3):
1 – Detailed trajectory statistics (month 7)
3 – Agent model replicates motion (month 13)
6 – Replicate change in motion during odour-taste conditioning (month 19)

This entry was posted in Objectives, Project. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *