BERT Tiny L 2 H 128 A 2

BERT

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 20191K context≈ 384 words

A stripped-down, no-frills embedding specialist — just 2 layers, 128 hidden dimensions, and 2 attention heads. It trades depth and representational richness for extreme speed and a tiny memory footprint. Expect fast, lightweight embeddings suitable for constrained environments, but don't expect the nuanced representations that larger BERT variants produce.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Multilingual

Moderate

Instruction Following

Use Case Fit

Fit scores are AI-generated based on model capabilities, intended use, and technical specifications. Learn more

BERT Tiny L 2 H 128 A 2

BERT

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 20191K context≈ 384 words

A stripped-down, no-frills embedding specialist — just 2 layers, 128 hidden dimensions, and 2 attention heads. It trades depth and representational richness for extreme speed and a tiny memory footprint. Expect fast, lightweight embeddings suitable for constrained environments, but don't expect the nuanced representations that larger BERT variants produce.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Multilingual

Moderate

Instruction Following

Use Case Fit

Fit scores are AI-generated based on model capabilities, intended use, and technical specifications. Learn more

Glossary

AttentionA mechanism that lets the model focus on relevant parts of the input when generating each output token.BERTA foundational neural network architecture designed to understand the meaning of words in context by learning from large amounts of text.EmbeddingA dense numerical vector that represents a word, sentence, or concept in a high-dimensional space.EmbeddingsNumerical representations of text that capture semantic meaning, allowing the model to measure similarity between different words or phrases.Memory FootprintThe amount of RAM or storage space a model requires to run, which is critical for deployment on resource-constrained devices.