Together AI· Inference platform

Plan, divide, and conquer: How weak models excel at long context tasks

TL;DR The Intuition: Don't ask one genius to read a library in an hour. Ask ten interns to read one book each. Our research, "When Does Divide and Conquer Work for Long Context LLM?" (ICLR 2026), introduces a framework to study this. We found that smaller models using a strategic "Divide & Conquer" design can match or beat GPT-4o single-shot on long context tasks. [ paper, ICLR 2026 ] [ code ] Modern LLMs increasingly support massive context windows like 128K, 200K, even 1M+ tokens. This theoretically unlocks powerful use cases like analyzing entire codebases or summarizing full books in a single prompt. However, the promise of "just throw everything into one prompt" frequently fails in practice. As context length grows, performance degrades in unexpected ways. “Divide and Conquer” turns out to be an attractive solution to long context tasks, as shown in the figure below. “Divide and Conquer” Framework. The Planner rewrites the job description. The Worker each processes a subset of the long document, while the Manager aggregates the information and provides the final answer. …

Announced

26 Mar 2026 00:00 UTC