A store for previously computed key-value pairs that speeds up text generation in transformers.
Performance retention over long documents and conversations