Source note · Independent commentary
When AI helps until the task changes
It shows that a tool's competence is uneven in ways that are hard to see from inside a task, which makes knowing where the tool fails a form of experience in its own right.
Research reviewed
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
- Authors
- Fabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, Karim Lakhani
- Institution
- Harvard Business School
- Publication
- HBS Working Paper 24-013; forthcoming in Organization Science
- Date
- 2023-09
- Type
- Working paper
A preregistered field experiment with several hundred management consultants. On tasks inside the model's capability, consultants using GPT-4 finished more tasks, faster, and at higher rated quality. On a task designed to sit outside that capability, those using the tool were more likely to get the answer wrong.
Read the original sourceEvidence
What the source supports
- AI capability is uneven across tasks that look similar in difficulty. The authors call this the jagged frontier.
- For tasks inside the frontier, consultants with the tool performed measurably better on speed and rated quality.
- For a task outside the frontier, tool use was associated with worse accuracy.
Boundary
What the source does not support
- That the frontier is fixed. The authors describe it as moving and hard to see from the inside.
- That professionals can reliably tell in advance which side of the frontier a task sits on.
- Any general claim about consulting careers, hiring, or the value of experience over time.
Interpretation
What Sougata is doing with it
Sougata uses this for one specific point: a tool's competence is uneven in ways that are hard to see from inside a task, so knowing where it fails is worth something. This is context about AI and work. It is not evidence that the Second Door Plan helps anyone, and the paper makes no claim about that method.
This is an independent Voyagers Beyond source note. Voyagers Beyond is not affiliated with or endorsed by the authors, publishers, universities, organizations, or institutions referenced here unless explicitly stated otherwise. The summary and analysis are written independently; nothing from the work is reproduced.
Why it matters here