ERNIE 4.5 21B A3B MLX 4bit

ERNIE

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released July 2025131K context≈ 98,304 words21B params

A mid-sized Chinese language model from Baidu's ERNIE family, quantized to 4-bit precision for efficient local deployment via MLX. It activates only 3 billion of its 21 billion parameters at a time, keeping inference fast while retaining a broad knowledge base. The trade-off is that quantization and sparse activation may reduce precision on complex reasoning tasks compared to full-precision variants.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Tool Use

Moderate

Coding

ERNIE 4.5 21B A3B MLX 4bit

ERNIE

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released July 2025131K context≈ 98,304 words21B params

A mid-sized Chinese language model from Baidu's ERNIE family, quantized to 4-bit precision for efficient local deployment via MLX. It activates only 3 billion of its 21 billion parameters at a time, keeping inference fast while retaining a broad knowledge base. The trade-off is that quantization and sparse activation may reduce precision on complex reasoning tasks compared to full-precision variants.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Tool Use

Moderate

Coding

Glossary

4-bit PrecisionA quantization level where model weights are stored using only 4 bits per value, significantly reducing model size at the cost of some accuracy.Bit PrecisionThe number of bits used to represent each number in a model; lower bit precision (like 3-bit) means smaller file size but potentially less accurate calculations.Complex ReasoningThe ability to work through multi-step problems, analyze nuanced information, and draw logical conclusions.Full-PrecisionA model using standard 32-bit floating-point numbers to represent weights, providing maximum accuracy but requiring more memory.InferenceThe process of running a trained model to generate predictions or outputs from new inputs.Knowledge BaseA structured or unstructured collection of documents and facts that a system retrieves from to answer queries.Language ModelAn AI model trained to predict and generate text by learning patterns from large amounts of written data.Local DeploymentRunning a model directly on your own computer or server instead of sending requests to a remote service.MLXA machine learning framework optimized for running models efficiently on Apple Silicon chips.ParametersThe learned numerical values in a model — more parameters generally means more capacity but higher compute cost.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizationReducing a model's numerical precision (e.g., from 16-bit to 4-bit) to shrink memory usage and speed up inference.QuantizedA technique that reduces a model's size and memory usage by storing weights with lower precision (fewer bits), trading some accuracy for efficiency.ReasoningThe model's ability to work through multi-step logical problems and provide justified answers rather than just pattern-matching.Reasoning TasksProblems that require a model to think through multiple steps logically to arrive at an answer, rather than just pattern-matching.Sparse ActivationA technique where only a subset of a model's parameters are used for each input, reducing computational cost while maintaining performance.

Capabilities

Capabilities

Use Case Fit

Glossary