Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts provide excellent ways of thinking about reasoning.
The authors present topologies of reasoning as ways of thinking about reasoning using LLMs, or 'thoughts' that are called nodes and edges are dependencies between the thoughts are edges. If one thought is reachable from a task statement, that is a solution node, and the route is the solution topology.
They share thorough discussions on the following methods.
- Basic Input-Output (IO)
- Chain-of-Thought (CoT)
- Multiple CoTs (CoT-SC)
- Tree of Thoughts (ToT)
- Graph of Thoughts (GoT)
They consider common concepts such as:
- Multistep reasoning
- Zero-Shot Reasoning
- Planning and & Task Decomposition
- Task Preprocessing
- Iterative Refinement
- Tool Utilizatoin
They also summarize the general flow of a prompting interaction.
- The user sends their prompt
- Preprocessing
- Adding to into a prompting context
- Input the content to the LLM
- LLM Generation
- Post-processing (Checking NSFW)
- Returning information into the context, and either
- Iterating before returning to the user
- Reply to the user
They then share some important concepts related to topology.
They finally discuss Research opportunities:
- Exploring New Topology Classes
- Explicit Representation in Single-prompt Settings
- Automatically Deriving Tree and Graph Topologies
- Advancing Single-Prompt Schemes
- Investigating New Schedule Approaches
- Investigating Novel Graph Classes
- Integrating Graph Algorithms and Paradigms
- Diversifying Modalities in Prompting (multimodal)
- Enhancing Retrieval in Prompting
- Parallel Design in Prompting
- Integrating Structure-Enhanced Prompting with Graph Neural Networks
- Integrating Structure-Enhanced Prompting with Complex Architectures
- Hardware acceleration