How Do Attention and Memory Work in Deep Learning?

TL;DR

Attention lets a neural network focus its processing on information that matters while responding less strongly to irrelevant inputs. Memory can be understood as attention extended over time, because useful recall depends on selecting salient past information rather than accessing everything simultaneously. Network Jacobians reveal this implicit selectivity by measuring how sensitive outputs are to different parts of an input.

Transcript

okay everyone my name is Alex graves I'm a research scientist at beep mind which should no longer say google deepmind but only deep mine but I haven't updated my slides obviously and what I'm going to be talking about in this lecture is attention and memory and how we use these concepts in deep learning systems so just to get started I think it sho... Read More

Key Insights

• Attention is the ability to focus on one item while ignoring others, allocating limited mental or computational bandwidth selectively. It applies not only to sights and sounds, but also to pursuing one thought or recalling one event instead of considering every possible thought or memory simultaneously.
• Memory is attention extended over time, because useful human memory depends on selecting salient past information rather than merely storing and accessing everything. Computers can store enormous quantities of data, but storage capacity alone does not capture the selective character associated with human remembering.
• Neural networks are parametric nonlinear function approximators that learn mappings from input vectors to output vectors. An image can be represented as an input vector, while the corresponding output can represent a probability distribution over possible labels such as leopard, jaguar, cheetah, snow leopard, or cat.
• Implicit attention is present inside ordinary neural networks even when no explicit attention mechanism has been trained. A classifier can respond more strongly to discriminative image regions, such as patterned fur, ears, or eyes, because those features contribute differently to its final class prediction.
• The network Jacobian is the sensitivity of output values with respect to input values. Because neural networks are differentiable and trained through gradient descent and backpropagation, the same general machinery can identify which input regions most strongly condition a particular output prediction.
• State value is a prediction of the expected future reward associated with the current state. In the Atari driving example, its sensitivity covers more of the screen, including the current score and cars farther down the track that may provide points when overtaken.
• Action advantage is the predicted benefit of choosing a particular action relative to other available actions. It can operate on a different scale from state value, so the demonstrated architecture models value and advantage through two separate output heads before using them for decisions.
• Action sensitivity is strongest when available choices have meaningfully different consequences. During ordinary driving, going left or right may matter little, but a nearby car creates a critical distinction because steering toward it risks collision while steering away may permit overtaking and additional score.

Questions & Answers

Q: What is attention in deep learning?

Attention in this lecture means selective allocation of processing toward certain information while other information receives less emphasis. The idea is comparable to following one voice in a crowded room, focusing on one visual region, or pursuing one thought at a time. Neural networks can display this selectivity implicitly by becoming more sensitive to some input features than to others, even without a dedicated attention mechanism.

Q: How are attention and memory related?

Attention and memory are closely connected because remembering is not simply the storage or simultaneous retrieval of everything. Useful memory requires selecting salient events from the past and giving them current processing priority. The lecture therefore describes memory as a form of attention over time. This perspective distinguishes human-style remembering from a computer's capacity to store very large amounts of information in a small physical space.

Q: How can neural networks show implicit attention?

A neural network shows implicit attention when its outputs respond more strongly to particular parts of an input, despite receiving the complete input vector and lacking a separately trained attention component. For an animal image, the prediction might depend especially on fur patterns that distinguish classes, while ears and eyes help establish that the image contains an animal rather than an unrelated object such as a tree.

Q: What does a network Jacobian reveal?

A network Jacobian reveals how sensitive the network's outputs are to changes in its inputs. When a classifier predicts a label, this sensitivity indicates which input components most strongly conditioned that decision. Applied to an image or video frame, it can be visualized as a heatmap that highlights regions where changes would have a greater effect on the selected output than changes elsewhere.

Q: How is backpropagation used to visualize attention?

Backpropagation can calculate output sensitivity because neural networks are designed as differentiable function approximators. Instead of propagating the gradient of a training loss with respect to the outputs, the described procedure uses the output activations and propagates their influence backward toward the inputs. The resulting derivatives form the Jacobian and allow the input regions associated with a prediction to be displayed as a sensitivity map.

Q: Why separate state value and action advantage?

State value and action advantage can be modeled with two output heads because they represent different quantities and commonly operate on different scales. State value estimates the reward expected from the current situation, which may span a wide range. Action advantage measures the comparatively smaller difference produced by selecting one action instead of another. Splitting them lets the system model these distinct signals separately.

Q: What information does the state-value head use?

The state-value head uses information that helps estimate future reward from the current state. In the demonstrated Atari driving system, its sensitivity map covers much of the screen and repeatedly focuses on the score. It also looks farther down the track, where visible cars suggest opportunities to overtake and receive points. Future rewards are weighted with an exponential decay, emphasizing nearer outcomes over distant ones.

Q: When does the action-advantage head become sensitive?

The action-advantage head becomes strongly sensitive when one available action is meaningfully better than another. In the driving example, it activates only rarely because continuing straight is often sufficient and small left or right choices make little difference. When another car is close and offset to one side, steering direction becomes important because one choice may cause a collision while the other enables safe passage or overtaking.

Summary & Key Takeaways

• Attention is the capacity to focus limited processing on one source, thought, or event while ignoring alternatives. The cocktail party problem illustrates sensory attention, since a person can follow one voice among many. Memory is closely related because recalling effectively also requires selecting salient information across time instead of treating every stored event equally.

• Neural networks are differentiable, parametric, nonlinear function approximators that transform input vectors into output vectors. Although a conventional network appears to process an entire input, its learned function responds more strongly to some features than others. In image classification, discriminative details such as fur spots, ears, and eyes may influence predictions with different strengths.

• A network Jacobian measures output sensitivity with respect to input components and can expose implicit attention. In the Atari driving example, the state-value output uses broad information such as the score and cars farther ahead. The action-advantage output responds more selectively, becoming important when a nearby car makes one action better than another.