The length of tasks (measured by how long they take human professionals) that generalist frontier model agents can complete autonomously with 50% reliability has been doubling approximately every 7 months for the last 6 years.

AI performance has increased rapidly on many benchmarks across a variety of domains. However, translating this increase in performance into predictions of the real world usefulness of AI can be challenging.

We find that measuring the length of tasks that models can complete is a helpful lens for understanding current AI capabilities. \ [1] 1 This makes sense: AI agents often seem to struggle with stringing together longer sequences of actions more than they lack skills or knowledge needed to solve single steps.

the time taken by human experts is strongly predictive of model success on a given task: current models have almost 100% success rate on tasks taking humans less than 4 minutes, but succeed <10% of the time on tasks taking more than around 4 hours.

the length of tasks models can complete is well predicted by an exponential trend, with a doubling time of around 7 months.

If the measured trend from the past 6 years continues for 2-4 more years, generalist autonomous agents will be capable of performing a wide range of week-long tasks.