Precisely neuromodulating deep brain regions could bring transformative advancements in both neuroscience and treatment. We demonstrate that non-invasive transcranial ultrasound stimulation (TUS) can selectively modulate deep brain activity and affect learning and decision making, comparable to deep brain stimulation (DBS). We tested whether TUS could causally influence neural and behavioural responses by targeting the nucleus accumbens (NAcc) using a reinforcement learning task. Twenty-six healthy adults completed a within-subject TUS-fMRI experiment with three conditions: TUS to the NAcc, dorsal anterior cingulate cortex (dACC), or Sham. After TUS, participants performed a probabilistic learning task during fMRI. TUS-NAcc altered BOLD responses to reward expectation in the NAcc and surrounding areas. It also affected reward-related behaviours, including win-stay strategy use, learning rate following rewards, learning curves, and repetition rates of rewarded choices. DBS-NAcc perturbed the same features, confirming target engagement. These findings establish TUS as a viable approach for non-invasive deep-brain neuromodulation.

Interval timing is an evolutionarily well-preserved function that presents similar behavioral signatures across different species. However, the neural basis of interval timing remains an open question. For instance, although dopamine has been implicated as a vital component of the internal clock, its precise role is debated due to equivocal findings from various methodologies and their interpretations. We tested this question by optogenetically exciting versus inhibiting tyrosine hydroxylase-positive (TH+) neurons of the substantia nigra pars compacta while male mice produced at least a 3-second-long interval by depressing a lever for reward. Excitation of TH+ neurons shifted their timing behavior to the right, while inhibition led to a shift to the left. Our drift-diffusion-timing model-based analysis of the behavioral data clearly showed that TH+ neuron excitation and inhibition heightened and lowered the timing threshold, respectively, without affecting the rate of temporal integration (i.e., clock speed). Our work attributes a clear mechanistic role (i.e., threshold setting) to nigrostriatal dopaminergic function as part of the internal clock. Despite the ubiquity of time experience, how the brain perceives time is unresolved. Dopamine is a key neuromodulator system involved in subjective time experience. For instance, the time sense is disrupted in conditions characterized by dopaminergic dysfunction (e.g., Parkinson's disease, schizophrenia). However, the mechanistic role of dopamine in the operation of the internal clock is debated. We resolve this debate by optogenetically upregulating and downregulating the nigrostriatal dopamine in mice and evaluating the behavioral outcomes under a computational framework that assumes that the brain times by accumulating brain signals up to a threshold. Our results showed that modulating the nigrostriatal dopamine system alters the level to which the brain integrates clock signals (temporal caution) without altering the clock speed.

Being able to switch from established choices to new alternatives when conditions change - behavioral flexibility - is essential for survival. Cholinergic signaling in the striatum contributes to such flexible behavior, yet the timing and spatial organization of acetylcholine release during contingency changes remain unclear, limiting conceptual understanding of its role in behavioral flexibility. Using a genetically encoded acetylcholine sensor and 2-photon imaging in the dorsal striatum of behaving mice, we visualized acetylcholine dynamics during acquisition and reversal learning in a virtual reality Y-maze. Rewarded outcomes evoked phasic decreases in acetylcholine, whereas unexpected non-reward following reversal triggered widespread increases that predicted lose-shift behavior. Targeted inhibition of cholinergic interneurons reduced this adaptive response. Spatial analysis revealed heterogeneous, temporally distinct signals forming functionally diverse microdomains. These findings suggest that widespread and focal acetylcholine release during unexpected outcomes promotes adaptive response shifts, offering a mechanistic framework for understanding disorders such as addiction and obsessive-compulsive rituals.

The basal ganglia (BG) are a collection of subcortical nuclei involved in motor control, sensorimotor integration, and procedural learning. They play a key role in the acquisition and adaptation of movements, a process driven by dopamine-dependent plasticity at cortico-striatal projections, which serve as BG input. However, BG output is not necessary for executing many well-learned movements. This raises a fundamental question: How can plasticity at BG input contribute to the acquisition and adaptation of movements which execution does not require BG output? Existing models of BG function often neglect the feedback dynamics within cortico-BG-thalamo-cortical circuitry and do not capture the interaction between the cortex and BG in movement generation and adaptation. In this work, we address the above question in a theoretical model of the BG-thalamo-cortical multiregional network, incorporating anatomical, physiological, and behavioral evidence. We examine how its dynamics influence the execution and reward-based adaptation of reaching movements. We demonstrate how the BG-thalamo-cortical network can shape cortical motor output through the combination of three mechanisms: i) the diverse dynamics emerging from its closed-loop architecture, ii) attractor dynamics driven by recurrent cortical connections, and iii) reinforcement learning via dopamine-dependent cortico-striatal plasticity. Our study highlights the role of the cortico-BG-thalamo-cortical feedback in efficient visuomotor adaptation. It also suggests a mechanism for early-stage acquisition of reaching movements through motor babbling. More generally, our model explains how the BG-cortical network refines motor output through its intricate closed-loop dynamics and dopamine-dependent plasticity at cortico-striatal synapses.

Neural activity encoding recent experiences is replayed during sleep and rest to promote consolidation of memories. However, precisely which features of experience influence replay prioritisation to optimise adaptive behaviour remains unclear. Here, we trained adult male rats on a novel maze-based reinforcement learning task designed to dissociate reward outcomes from reward-prediction errors. Four variations of a reinforcement learning model were fitted to the rats' behaviour over multiple days. Behaviour was best predicted by a model incorporating replay biased by reward-prediction error, compared to the same model with no replay, random replay or reward-biased replay. Neural population recordings from the hippocampus and ventral striatum of rats trained on the task evidenced preferential reactivation of reward-prediction and reward-prediction error signals during post-task rest. These insights disentangle the influences of salience on replay, suggesting that reinforcement learning is tuned by post-learning replay biased by reward-prediction error, not by reward per se. This work therefore provides a behavioural and theoretical toolkit with which to measure and interpret the neural mechanisms linking replay and reinforcement learning.

Axons of dopaminergic neurons express gamma-aminobutyric acid type-A receptors (GABARs) and nicotinic acetylcholine receptors (nAChRs), which are positioned to shape striatal dopamine release. We examine how interactions between GABARs and nAChRs influence dopaminergic axon excitability. Axonal patch-clamp recordings reveal that potentiation of GABARs by benzodiazepines suppress dopaminergic axon responses to cholinergic interneuron transmission. In imaging experiments, we use the first temporal derivative of axonal calcium signals to distinguish between direct stimulation of dopaminergic axons and nAChR-evoked activity. Inhibition of GABARs with gabazine selectively enhance nAChR-evoked axonal calcium signals but does not alter the strength or dynamics of acetylcholine release, suggesting that the enhancement is mediated primarily by GABARs on dopaminergic axons. Unexpectedly, we find that a widely used GABAR antagonist, picrotoxin, inhibits axonal nAChRs and should be used cautiously for striatal circuit analysis. Overall, we demonstrate that GABARs on dopaminergic axons regulate integration of nicotinic input to shape axonal excitability.

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes from preceding cues or actions. For example, reward outcomes may be linked to specific stimuli and actions, corresponding to stimulus- and action-based learning. But how does the brain arbitrate between such models? Here, we combined multiple computational approaches to quantify concurrent learning in male monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for a dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala in model arbitration. We demonstrated that the amygdala adjusts the initial balance between the two learning systems and is essential for updating arbitration according to the correct model, which in turn alters the interaction between arbitration and learning that governs the time course of learning and choice behavior. In contrast, VS lesions lead to an overall reduction in stimulus-value signals. This role of the amygdala reconciles existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

Flexible control of motor timing is crucial for behaviour. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviours, distinguishing their individual roles in this timer function remains challenging. Here, to address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings are consistent with a model in which the striatum is part of a network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.

All mammals exhibit flexible decision policies that depend, at least in part, on the cortico-basal ganglia-thalamic (CBGT) pathways. Yet understanding how the complex connectivity, dynamics, and plasticity of CBGT circuits translate into experience-dependent shifts of decision policies represents a longstanding challenge in neuroscience. Here we present the results of a computational approach to address this problem. Specifically, we simulated decisions during the early learning process driven by CBGT circuits under baseline, unrewarded conditions using a spiking neural network, and fit an evidence accumulation model to the resulting behavior. Using canonical correlation analysis, we then replicated the identification of three control ensembles (responsiveness, pliancy and choice) within CBGT circuits, with each of these subnetworks mapping to a specific configuration of the evidence accumulation process. We subsequently simulated learning in a simple two-choice task with one optimal (i.e., rewarded) target and found that, during early stages of learning, feedback-driven dopaminergic plasticity on cortico-striatal synapses effectively increases reward rate over time. The learning-related changes in the decision policy can be decomposed in terms of the contributions of each control ensemble, whose influence is driven by sequential reward prediction errors on individual trials. Our results provide a clear and simple mechanism for how dopaminergic plasticity shifts subnetworks within CBGT circuits so as to increase reward rate by strategically modulating how evidence is used to drive decisions.

Dopamine (DA) is essential for the production of vigorous actions, but how DA modifies the gain of motor commands remains unclear. Here we show that subsecond DA transients in the striatum of mice are neither required nor sufficient for specifying the vigor of ongoing forelimb movements. Our findings have important implications for our understanding of how DA contributes to motor control under physiological conditions and in Parkinson's disease.

Dopaminergic transmission within the ventral striatum is broadly implicated in risk/reward decision making and impulse control, and the rat gambling task (rGT) measures both behaviours concurrently. While the resulting indices of risky choice and impulsivity correlate at the population level, dopaminergic manipulations rarely impact both behaviours uniformly, with changes in choice more likely when dopaminergic transmission is altered during task acquisition. Although the task structure of the rGT remains constant, the relative importance of ventral striatal dopamine signals relevant for reward prediction versus impulse control may vary as learning progresses; the former should dominate while rats learn the probabilistic contingencies of the task, whereas suppression of premature responses becomes more valuable once a decision-making strategy is established and exploited. Striatal cholinergic interneurons (CINs) critically influence reinforcement learning by modulating dopamine release and gating periods of dopamine-facilitated neuroplasticity. We therefore hypothesised that ventral striatal CINs (vsCINs) could influence reward learning or impulse control during task acquisition or stable performance, respectively. Using chemogenetics in Sprague Dawley rats (Rattus norvegicus), we found support for this hypothesis: activation and inhibition of vsCINs once behaviour was stable increased and decreased motor impulsivity in both sexes but had no effect on choice patterns. In contrast, activating and inhibiting vsCINs during task acquisition did not alter motor impulsivity but instead decreased and increased risky choice, respectively. Notably, the former effect was only observed in males, and the latter in females. We conclude by proposing testable predictions regarding acetylcholine-dopamine interactions that may explain sex differences. Impairments in decision making and impulsivity are central to psychiatric conditions such as addiction, ADHD, and impulse control disorders. Understanding how these behaviours are regulated in the brain, and why they differ across individuals and sexes, is critical for developing targeted treatments. This study identifies ventral striatal cholinergic interneurons as important modulators of both impulsivity and risk-based decision making, with their influence depending on learning stage and biological sex. These results show how acetylcholine and dopamine systems interact to shape behaviour in flexible and individualized ways. By revealing circuit-level mechanisms that may underlie sex-specific vulnerabilities and stage-specific treatment outcomes, this work lays the groundwork for more personalized approaches to treating disorders involving poor impulse control and risky decision making.

The substantia nigra pars reticulata (SNr), a key basal ganglia output nucleus, is modulated by dopamine (DA) believed to be released locally from midbrain DA neurons. Although DA has been proposed to regulate γ-aminobutyric acid (GABA) release from medium spiny neuron (MSN) terminals via presynaptic D1 receptors, the precise mechanisms remain unclear. Using presynaptic optical recordings of synaptic vesicle fusion, calcium influx in D1-MSN synapses together with postsynaptic patch-clamp recordings from SNr neurons, we found that DA inhibits D1-MSN GABA release in a frequency-dependent manner. Unexpectedly, this effect was independent of DA receptors and instead required 5-HT1B receptor activation. Using two-photon serotonin biosensor imaging in slices and fiber photometry in vivo, we demonstrate that DA enhances extracellular serotonin in the SNr via inhibition of serotonin reuptake. Our results suggest that serotonin mediates DAergic control of basal ganglia output and contributes to the therapeutic actions of dopaminergic medications for Parkinson's disease and psychostimulant-related disorders.

Research indicates that midbrain dopaminergic neurons encode reward prediction error (RPE) signals involved in positive reinforcement learning. However, studies on dopamine's role in negative reinforcement learning (NRL) are scarce. Learning to escape aversive stimuli is vital for survival and may differ significantly from positive reinforcement in behavior and neural mechanisms. This study employs footshocks as aversive stimuli to investigate neural activity, synaptic transmission, and intrinsic excitability in a NRL paradigm using fiber photometry and ex vivo electrophysiology. Results show that inescapable footshocks initially increase activity in substantia nigra pars compacta (SNc) dopaminergic neurons, which later shifts to reflect shock termination as exposure increases. Electrophysiological observations reveal increased intrinsic excitability and excitatory synaptic transmission in SNc neurons, with decreased inhibitory transmission. After mice learn to escape the shock by nose-poking, dopaminergic activity shifts from shock termination to shock onset. Furthermore, inhibitory input increases, while excitatory input decreases after learning, with intrinsic excitability returning to baseline levels. This indicates that SNc dopaminergic neurons exhibit RPE-like signals in response to aversive stimuli, with their intrinsic excitability adjusting according to expectations of shock termination. These findings enhance our understanding of RPE encoding in negative reinforcement learning and may inform therapeutic strategies for disorders caused by environmental factors such as aversive stimuli.

Active ensembles of neurons form an engram during learning. However, engrams are not immutable, and their organization may change with time via systems consolidation. Here, we labeled engram ensembles in the prelimbic (PrL) cortex during contextual fear conditioning. We found that distinct engram subpopulations ("sub-engrams") contribute to memory recall at recent versus remote delays, with sub-engram contribution determined by their projection profile. At recent delays, sub-engrams projecting to the basal amygdala (BA) and lateral entorhinal cortex (LEC) are activated, and their activity is necessary and sufficient for memory retrieval. At remote delays, sub-engrams projecting to the nucleus reuniens (NRe) and nucleus accumbens (NAc) are additionally recruited, and their activity is necessary and sufficient for memory retrieval. Recruitment of NRe- and NAc-projecting sub-engrams to remote recall is an active process, depending on post-training activation of PrL parvalbumin-expressing interneurons. Post-training chemogenetic inhibition of PrL parvalbumin-expressing interneurons prevented sub-engram recruitment and impaired remote memory.

Previous studies revealed critical involvement of the striatum in adapting to the environment by actions that anticipate rewards from experiences as a policy. However, it remains unclear how current policy is evaluated to explore more advantageous alternatives. Here, we show that during policy-based sequential actions in a rat reversal task, the dorsomedial striatum plays an essential role in pathway-specific manner. Recording and optical manipulation of the indirect pathway showed that late-onset activity following unrewarded suboptimal action represents a lowered valuation of the current action policy and a heightened bias to try the suboptimal action. The early-onset activity complementarily mediated policy-based suppression of unrewarded action. These results demonstrate the indirect pathway's role in monitoring unreliability of current action policy and probing alternative one. This study extends conventional understanding of consequence-guided persistence with reward-oriented action policy and provides key insights regarding how the dorsomedial striatum enables proactive and flexible adaptation to environmental changes.