Read-agent: A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts

Jupyter notebook Developments

The authors reveal a manner of reading long documents and summarizing it using Gist memory to deal with Long Contexts.

Problem

Context length of long inputs limits the ability for model to perform effectively and efficienntly.

Solution

With inspiration in how people interactively read long documents, the authors implement a simple prompting-based system that

1. Decides what content should be stored togeter in a memory episode
2. Compresses those memories into short episodic memories called gist memories and
3. Takes actions to look up sections in the original text if memory needs to be refreshed

Results The simple method improves reading comperhension tasks at the same time as enabling context windows that are 3-20x bigger.

Paper