reformer enwik8

Name: reformer enwik8
Author: Google

by Google

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released March 202266K context≈ 49,152 words

A specialist trained on raw byte-level text data (enwik8, a Wikipedia corpus), this model demonstrates the Reformer architecture's core trick: replacing standard attention with locality-sensitive hashing to handle very long sequences cheaply. It trades general-purpose knowledge for architectural efficiency, making it more of a research artifact than a conversational tool. Expect it to reflect its narrow training data rather than broad world knowledge.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Coding

Moderate

Instruction Following

reformer enwik8

by Google

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released March 202266K context≈ 49,152 words

A specialist trained on raw byte-level text data (enwik8, a Wikipedia corpus), this model demonstrates the Reformer architecture's core trick: replacing standard attention with locality-sensitive hashing to handle very long sequences cheaply. It trades general-purpose knowledge for architectural efficiency, making it more of a research artifact than a conversational tool. Expect it to reflect its narrow training data rather than broad world knowledge.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Coding

Moderate

Instruction Following

Glossary

ArchitectureThe underlying structural design of a neural network that defines how data flows through layers and components.ArtifactA released model, dataset, or tool that is publicly documented and available.AttentionA mechanism that lets the model focus on relevant parts of the input when generating each output token.CorpusA collection of documents or text used as the knowledge base for retrieval in RAG systems.General-PurposeDesigned to handle a wide variety of different tasks rather than being specialized for one specific domain.Reformer ArchitectureA transformer-based model design that uses locality-sensitive hashing and reversible layers to efficiently process long sequences with reduced memory requirements.Research ArtifactA packaged, deployable component of research (code, data, configuration) that can be shared and reused.Training DataThe examples and information used to teach a model how to perform a task, in this case human-written and AI-generated grammatical corrections.World KnowledgeA model's learned understanding of facts, concepts, and relationships about the real world, typically acquired during training on diverse text data.

Capabilities

Capabilities

Use Case Fit

Glossary