Reinforcement Learning and Dopamine Neurons Function - Essay Example

Reinforcement Learning and Dopamine Neurons Function

Reinforcement learning refers to an approach where the learner interacts with the reactive environment as means of knowing how to solve problems. The agents may evolve during the learning process to overcome the problems they face in their environment and adapt to survive (Schultz 1998, p. 1). The concept remains active for humans in most of the fields including computer science, engineering and military. The process also has rewards after completing an action that encourages the agent to do the same thing when they encounter a similar problem. In these occasions, the learner needs to interact continuously with environmental elements that pose various challenges and need solutions. Earlier studies primarily focused on the animal and the way to use the environment to alter their behavior through reinforced learning. In these cases, there is always a goal that the learning process aims to achieve which is unique.

The research in the subject of reinforced learning links it to biological systems that could explain the changes in the individuals. The neuromodulator seems to have a primary effect on the reinforcement learning. It controls the brains during the rewards and affects the synaptic plasticity. The theory supporting the reinforcement learning also uses the reward based learning process. It shows the linkage between the two concepts in affecting the learning of an agent in a reactive environment through changes in their brains. The learner may not be correct, but they receive feedbacks that can be rewards or punishments that encourage or discourage the actions (Benjamin 2008, p. 78). Such feedbacks often have an influence on the brain of the learner through changing the release of dopamine.

Theory of Reinforcement Learning

This learning process uses the behaviorism theory that tries to explain the actions of people based on the interaction with the environment. It is a theory applied in current psychology to explain the behavior of people. The theory implies that people behave while considering the consequences of their actions. Reinforcement theory finds significant use in the society such as training of animals, motivating employees or raising kids. It focuses on the environmental factors that shape the behaviors of people and animals. The approaches in this theory include negative and positive reinforcement or negative and positive punishment as possible ways of learning (Hanson, 2011 20). These approaches have an effect on the brain by affecting the amount of dopamine released to modify the action of the agent. Thus, the theory applies the law of effect that shows that actions producing a satisfying effect in a particular situation are common to occur again in similar cases. The learning process greatly depends on the prediction errors encoded by the dopamine neurons (Tobler, Dickinson & Schultz 2003, p.46). However, actions producing less satisfying effect are less likely to occur in a similar situation.

Rewards come in different forms that depend on the environment. The brain stores the rewards from the previous experience. When the agent faces a similar situation, the brain retrieves the information using neuronal mechanisms. The reward system assists in the creation of the prediction system. These predictions are crucial in the choices that people make after learning through reinforcement process (Dayan & Balleine 2002, p. 1). The application of the reinforcement learning aims at minimizing the punishment while maximizing the rewards.

Dopamine in the brain

In the mammals, production of dopamine takes place in the brain. The neurons that contain this neurotransmitter fall in three mesencephalic groups that include A8 cells found in the retrorubral field, A9 cells in the substantia nigra and A10 cell located in the ventral tegmental area (Alcaro, Huber & Panksepp 2007, p. 3). These neurons have large cell bodies and long complex axonal arbors. The axons have a terminal specialization that allows them to release neurotransmitter into the extracellular space for the broad anatomical action of dopamine. The arrangement is also similar to that in the birds and reptiles. In addition, other regions in the brain have these DA neurons including the periaqueductal gray, dorsal raphe and supramammillary region in the hypothalamus. The axons of the A8, A9 and A10 cells project primarily to the anterior part of the forebrain. These three types of cells are the primary producers of dopamine in animals. They also modulate the cognitive-executive reentrant circuit that increases their ability to influence the motivated behaviors.

A8 and A10 cells of VTA make a connection with the frontocortical and ventral striatum. The A9 connects putamen and caudate. However, the three neurons seem to exhibit intermixing. It is hard to separate their locations in the brain entirely.

The amount of dopamine remains low due to regulation mechanisms in the brain. Its production often takes place when there is some reward for an action. In the daily lives of animals, the neurotransmitter often makes the people and animals feel pleasure for their actions. These reactions play significant roles in the reinforcement learning process. The good feeling from its release functions as a motivational factor to encourage similar actions. Its increase is evident in people who take psychostimulant drugs like Adderall. Dopamine further plays a part in motor control and triggering the release of various hormones.

The dopamine neurons often send signals to the frontal cortex and basal ganglia. The neurons return the information to the frontal cortex through a thalamic relay that assists in the reinforcement learning (Glimcher 2011, p 15647). The connection forms a loop that allows continuous communication, which assists in the generation of the behavior output.

Connection of dopamine and reinforced learning

The mesolimbic dopamine system is the most important reward system in the brain. The dopamine neuron cells play the central role in the regulation of the amount of the transmitter released in the brain. It acts to detect rewarding stimulus from the environment. It is an old pathway while considering in the perspective of evolution due to its presence in most of the animals. The system works in normal conditions through a circuit that controls the agent’s response to natural factors such as food, social interactions and sex. Thus, it serves as a significant motivation determinant with the aim of gaining particular incentives. This system is crucial to the learning and survival of the organisms. Besides, the system instructs an individual to repeat their actions that they did in the previous similar situation to receive a pleasing result. It further guides the brain to pay attention to specific features to the rewarding experiences.

The mesolimbic pathway originates in several structures of the limbic system and ventral tegmental area. This pathway connects VTA found in the midbrain to the nucleus accumbens. Most of the antipsychotic drugs inhibit the reaction in this pathway as a way to reduce the intense emotions amongst patient with diseases such as schizophrenia. The data from studies show that the activation of dopamine neurons in the midbrain encodes reward prediction error useful in the learning process within the basal ganglia and the frontal cortex (Glimcher 2011, p 15647). The scientists believe that the subjects use the signals to estimate the magnitude of prediction error in the present and future situations.

The dopamine neurons show differing reactions depending on the outcome of the learning process. During the reinforcement learning, giving the rewards as stimuli activate the dopamine neurons. It leads to the high release of dopamine in the brain causing the individual to have a feeling of pleasure for their actions. The interaction of the mesolimbic system and the cognitive part of the brain causes storage of the information in the brain cells. The mode of activation may require the use of visual or audio stimuli associated with the reward during the learning process. The activation of these neurons can take place even in the absence of the reward and does not discriminate the stimuli as long as there is an association with reward. Thus, in the reinforcement learning, the use of rewarding stimuli leads to continuous repetition of the action to obtain a pleasurable outcome. The activation of the dopamine neurons continues as long as there is prediction error that shows learning is still in progress.

Activation of the dopamine neurons increases with discrepancies between potential rewards (Fiorillo, Tobler & Schultz 2003, p. 1901). However, the activation of the few neurons ceases after some time when there are no more prediction errors. The experiments show that prediction error remains crucial during the activation of the dopamine neurons. The activation of the neurons is essential in the modification of the behavior of the individual in the reinforcement learning (Fiorillo, Tobler & Schultz 2003, p. 1901). Thus, the dopamine functions through activation when there is prediction error to enable modification of the memory so that the person can fully eliminate the prediction error after they learn.

The evidence from studies also shows that changing the non-rewarding actions to rewarding ones also activates these dopamine neurons. This kind of situations gives rise to prediction errors to the individual. The activation of the dopamine neurons occurs as the agent tries to learn about the new expectations. In such situations, the dopamine neurons enable communications with other parts of the brain to allow memorizing of the new contradicting information. The dopamine secreting cells function using the neurotransmitter. The activation of these neurons continues until there are no more prediction errors. The cells lose their reward activation after exposure to similar reward over a long period (Hollerman & Schultz 1998, p.305). At this condition, the agents can confidently predict what will happen with no need for neuron activation for reinforcement learning.

According to the Pavlov’s studies, the repetition of the behavior occurs due to the preexisting anatomical connection (Glimcher 2011, p.15648). The neurons relay the information between the midbrain and other parts through dopamine that allows the creation of the memory about the experience. It allows the organism to respond in a manner that will enable them to get the reward in future.

The ability to achieve a stage where there are no prediction errors differs among the rewards. However, the rate at which the responsiveness of the neurons decreased corresponded to the learning period. If there is a fast reduction dopamine neurons activation, then the learning period is short (Hollerman & Schultz 1998, p.305). In cases where the learning took place slowly, dopamine neurons activation rate also slows. In all cases, there was no activation of the dopamine neurons after agents could accurately predict the future.

The dopamine neurons seem encodes the difference amid the expected and happening reward (Tobler, Dickinson & Schultz 2003, p. 10406). The dopamine neurons have a tendency to show depression when there is the withdrawal of rewards associated with stimulus at the initial periods (Tobler, Dickinson & Schultz 2003, p. 10406). However, continuous exposure to similar reward omission experience led to the loss of depression among the dopamine neurons. The findings show that the agent learns through the prediction error that the stimuli formerly associated with rewards do not lead to benefits anymore (Waelti, Dickinson & Schultz 2001, p. 43).

The model that often supports the function of the dopamine secreting cells in the reinforcement is actor-critic. It offers the subdivisions in the actions of the midbrain dopamine system. In one of the versions, the dopamine cells within substantia nigra pars compacta (SNc)and ventral tegmental area (VTA) reports similar prediction error but to different uses (Dayan & Balleine, 2002, p. 285). SNc dopamine cells subdue learning of actions in competitive cortico-striato-thalamo-cortical loops. The dopamine neurons in the VTA control the learning values found in the basolateral nucleus of the orbitofrontal cortex and amygdala (Dayan & Balleine 2002, p. 1). These pathways show the actions in the brain that lead to the behavior modifications of the individual.

Thus, the cells secreting dopamine play a significant role in the reinforcement learning. They serve in communication when there are prediction errors to modify the behaviors of the agents. The activation occurs when there is prediction error. However, the cells will reduce their activation as the agent learns. There is also depression when there is the omission of the rewards. The depression occurs due to negative prediction errors. It shows that the prediction errors have effects on the dopamine neurons that in turn control the reinforcement learning process.

In conclusion, the reinforcement learning uses the stimuli to modify the behavior of an individual. It uses the dopamine activation system to achieve the objectives. The dopamine neurons play a significant role in these learning processes. Their activation relays the information to other parts of the brain that influence the behavior of the organism. The anatomical structures of dopamine neurons allow them to support the dopaminergic pathway that is crucial in the reward system of learning.