How Did Humans Get Smart? Prompt Engineering for Claude's Long Context Window

How Did Humans Get Smart? Prompt Engineering for Claude's Long Context Window

In observing other people's actions and registered thoughts, we learn from the environment we're inserted in. What you hear, what you read, the things you go through...they all shape your future decisions, as well as what you choose to do with the information you receive. Humans are smart, but they can get smarter if they wish.

The bottom line is: each person responds differently to situations depending on how their brain is wired. That's why we all have different opinions and beliefs. Humans need different upbringings and experiences to keep the knowledge wheels turning. Besides, humans need to keep looking up to other humans.

Again, human intelligence has never improved on its own, and it never will. Through the ages, humans got smarter not only by observing other humans but by refuting them, agreeing with them, and reaching their own conclusions based on a number of factors. Those include their upbringing and the situations they've encountered throughout their lives.

One interesting aspect to consider when discussing human intelligence is the concept of prompt engineering for Claude's long context window. While performance on the beginning and middle of the document is substantially improved by the use of a scratchpad and examples, performance on the end can be degraded. This could be because the addition of the examples in the prompt increases the distance between the very end of the document (where the relevant information is) and when Claude needs to answer it.

To achieve the best performance on both short and long context lengths, it is recommended to use many examples and the scratchpad. Pulling relevant quotes into the scratchpad is helpful in all head-to-head comparisons, even though it comes at a small cost to latency. Interestingly, generic examples on general or external knowledge do not seem to help performance, indicating the importance of specific and relevant information.

Furthermore, when considering the performance of Claude Instant, there seems to be a monotonic inverse relationship between performance and the distance of the relevant passage to the question and the end of the prompt. This suggests that the closer the relevant information is to the question, the better the performance. On the other hand, Claude 2 performance on 95K sees a small dip in the middle, indicating that the specific context length can affect performance differently depending on the model.

In conclusion, human intelligence is a product of observation, learning, and experiences. By observing others, refuting or agreeing with their thoughts, and reaching our own conclusions, we continue to expand our intelligence. Similarly, in the field of AI, prompt engineering plays a crucial role in optimizing performance. Using a scratchpad, incorporating relevant quotes, and prioritizing specific examples can enhance the accuracy and efficiency of AI models like Claude. So, whether it's human intelligence or AI intelligence, continuous learning and thoughtful engineering are key to getting smarter.

Actionable advice:

1. Embrace diversity: Just as humans need different upbringings and experiences to expand their intelligence, embrace diverse perspectives and learn from people with different opinions and beliefs.
2. Prioritize relevance: When designing prompts or questions, ensure that the relevant information is in close proximity to improve performance and accuracy.
3. Continuously refine and optimize: Like prompt engineering for AI models, continuously refine your own learning and decision-making processes. Incorporate new information, challenge existing beliefs, and seek ways to optimize your own intelligence.