Tree of Thoughts: Deliberate Problem Solving with Large Language Models
FreeDeliberate reasoning via tree search over thoughts
About Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Tree of Thoughts (ToT) is a framework that enhances large language model (LLM) reasoning by enabling deliberate exploration over multiple intermediate reasoning paths. Unlike standard chain-of-thought prompting, ToT treats reasoning as a search over a tree of thought units, where the LLM generates and evaluates candidate thoughts at each step, using breadth-first or depth-first search to arrive at better solutions. The approach is particularly effective for tasks requiring planning, exploration, and strategic lookahead, such as mathematical problem-solving, creative writing, and puzzle solving. The paper introduces this method and demonstrates its superiority over traditional prompting techniques on several benchmarks.
Key Features
Pros & Cons
- Significantly improves accuracy on complex reasoning tasks compared to chain-of-thought
- Enables systematic exploration and backtracking for better solutions
- Provides interpretability through explicit thought paths
- Flexible – can be adapted with different search strategies and evaluation heuristics
- Higher computational cost due to multiple LLM calls per problem
- Requires careful design of thought decomposition and scoring functions
- Performance depends on the quality of the LLM’s self-evaluation
- May not scale well to very deep search trees without pruning