ERNIE 4.5 21B A3B MLX 6bit

ERNIE

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released July 2025131K context≈ 98,304 words21B params

A mid-sized Chinese language model from Baidu's ERNIE family, quantized to 6-bit precision and packaged by lmstudio-community for local MLX inference. With 21 billion total parameters but only 3 billion active at a time via sparse activation, it balances capability with efficiency. The MLX format targets Apple Silicon hardware, making it practical for on-device use without requiring a dedicated GPU.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Instruction Following

Moderate

Reasoning & Logic

ERNIE 4.5 21B A3B MLX 6bit

ERNIE

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released July 2025131K context≈ 98,304 words21B params

A mid-sized Chinese language model from Baidu's ERNIE family, quantized to 6-bit precision and packaged by lmstudio-community for local MLX inference. With 21 billion total parameters but only 3 billion active at a time via sparse activation, it balances capability with efficiency. The MLX format targets Apple Silicon hardware, making it practical for on-device use without requiring a dedicated GPU.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Instruction Following

Moderate

Reasoning & Logic

Glossary

6-bit PrecisionA quantization method that represents model weights using only 6 bits per value, significantly reducing memory requirements compared to standard 32-bit floating-point storage.Apple SiliconApple's custom-designed processors (like M1, M2, M3) optimized for running machine learning models on Mac computers.Bit PrecisionThe number of bits used to represent each number in a model; lower bit precision (like 3-bit) means smaller file size but potentially less accurate calculations.InferenceThe process of running a trained model to generate predictions or outputs from new inputs.Language ModelAn AI model trained to predict and generate text by learning patterns from large amounts of written data.MLXA machine learning framework optimized for running models efficiently on Apple Silicon chips.MLX FormatA model format designed specifically for efficient inference on Apple Silicon devices, optimized for the MLX machine learning framework.On-DeviceA model designed to run directly on a user's device (phone, laptop, etc.) rather than requiring a remote server.ParametersThe learned numerical values in a model — more parameters generally means more capacity but higher compute cost.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizedA technique that reduces a model's size and memory usage by storing weights with lower precision (fewer bits), trading some accuracy for efficiency.Sparse ActivationA technique where only a subset of a model's parameters are used for each input, reducing computational cost while maintaining performance.

Capabilities

Capabilities

Use Case Fit

Glossary