The Multi-Armed Bandit Problem and Its Solutions

The Multi-Armed Bandit Problem and Its Solutions

The multi-armed bandit problem is a classic problem that well demonstrates the exploration vs exploitation dilemma. Imagine you are in a casino facing multiple slot machines, and each is configured with an unknown probability of how likely you can get a reward at one play. The question is: What is the best strategy to achieve the highest long-term rewards?

We need exploration because information is valuable. In terms of the exploration strategies, we can do no exploration at all, focusing on the short-term returns. Or we occasionally explore at random. Or even further, we explore, and we are picky about which options to explore — actions with higher uncertainty are favored because they can provide higher information gain.

How to Improve Your Creative Thinking

The way memories are associated makes it possible to connect your ideas later. In fact, recent evidence shows that semantic memory structure—how we relate language-based concepts—is one of the biggest predictors of creativity. Creativity is the ability to produce an artifact or an idea that is both novel and useful given a particular social context.

Researchers generally believe that creativity is a two-part process. The first is to generate candidate ideas and make novel connections between them, and the second is to narrow down to the most useful one. As a memory moves from short-term to long-term storage, what’s represented in that memory is associated with existing memories (aka “schemas”), and your overall understanding of the world shifts slightly.

Understanding how our brains give rise to the creative experience helps us understand the process more deeply and fills in a critical component that is thus far missing—attention. When Delilah is coming up with all of these different uses (i.e., divergent thinking), a network of brain regions called the "default network" is activated. In brain imaging studies, this network shows the highest activation during spontaneous, self-generated thought, including mind wandering, which is specifically not related to a task. Governing this process is a "control network" that allows us to think flexibly, inhibit outside distractions, and pull from our memory.

Switching between the default and control networks is thought to be governed by a "salience network," which helps us to engage each one effectively. Higher creative thinking scores are correlated with greater coupling between these networks.

“Constraints breed creativity.” Memories are the materials needed for creative thinking. Consuming other people's ideas in conversation by reading or listening to them stimulates creativity and deepens existing associations. To widen our divergent thinking funnel, we could try and seek out new ideas that are maximally different from our own, but this typically won’t work. It’s actually better to make incremental steps outside your own filter bubble because new information must overlap somewhat with what you already know to be effectively associated and assimilated.

Taking notes, making highlights, and revisiting that material when it’s related to what you’re currently thinking about is a good way to maintain and strengthen connections. These connections can happen spontaneously, often during a period of mind wandering when our attention is less focused or during an engaging conversation when we’re focused but someone else’s ideas are helping spark our own that we can share.

The best state for doing creative thought work depends on the type of work, your attentional state, your mood, and how engaged you can be. In terms of your mood, it’s helpful to be in a positive affective mental state—e.g., happy and not anxious. Counterintuitively, the more we try and force ourselves to focus, the narrower our thinking may become. Mind wandering is what allows us to explore, retrieve, and experiment with connecting our memories in different ways.

Common Points and Actionable Advice

Both the multi-armed bandit problem and improving creative thinking involve the concept of exploration. In the bandit problem, exploration allows us to gather valuable information, while in creative thinking, exploration helps us make novel connections between ideas.

To apply these concepts to our own lives, here are three actionable pieces of advice:

1. Embrace exploration: Whether it's in decision-making or creative endeavors, don't be afraid to explore different options. By trying out new strategies or seeking out diverse ideas, we can gather valuable information and make innovative connections.

2. Foster associations: Just as memories and associations play a crucial role in creative thinking, actively seek out opportunities to connect ideas and concepts. Engage in conversations, read widely, and take notes to strengthen your mental connections and expand your creative thinking abilities.

3. Balance focus and mind wandering: While focus is important for task completion, allowing your mind to wander can lead to valuable insights and creative breakthroughs. Find a balance between focused attention and periods of mind wandering, as both are essential for exploring new ideas and making unique connections.

In conclusion, the multi-armed bandit problem and improving creative thinking share common elements of exploration and the importance of associations. By understanding these concepts and applying actionable advice, we can enhance our decision-making abilities and foster our creative thinking skills. So, whether you're facing a dilemma or seeking innovative solutions, remember the power of exploration and the value of making meaningful connections.