Qwen3 Reranker 4B seq cls vllm W4A16

Qwen 3

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 202541K context≈ 30,720 words

A focused specialist that takes in pairs of texts and outputs relevance scores rather than generated text. It operates as a reranker — given a query and a set of candidate passages, it judges how well each passage matches the query. The W4A16 quantization means weights are stored at 4-bit precision while activations run at 16-bit, reducing memory footprint compared to a full-precision version.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Creative Writing

Moderate

Instruction Following

Qwen3 Reranker 4B seq cls vllm W4A16

Qwen 3

Open WeightModel weights are publicly available — can be downloaded and self-hosted

Released August 202541K context≈ 30,720 words

A focused specialist that takes in pairs of texts and outputs relevance scores rather than generated text. It operates as a reranker — given a query and a set of candidate passages, it judges how well each passage matches the query. The W4A16 quantization means weights are stored at 4-bit precision while activations run at 16-bit, reducing memory footprint compared to a full-precision version.

Capabilities

Capability scores are AI-generated based on model documentation, benchmarks, and technical specifications. Learn more

Creative Writing

Moderate

Instruction Following

Glossary

4-bit PrecisionA quantization level where model weights are stored using only 4 bits per value, significantly reducing model size at the cost of some accuracy.Bit PrecisionThe number of bits used to represent each number in a model; lower bit precision (like 3-bit) means smaller file size but potentially less accurate calculations.Full-PrecisionA model using standard 32-bit floating-point numbers to represent weights, providing maximum accuracy but requiring more memory.Memory FootprintThe amount of RAM or storage space a model requires to run, which is critical for deployment on resource-constrained devices.PrecisionThe level of numerical detail a model uses to represent its internal values; higher precision means more accurate calculations but requires more memory.QuantizationReducing a model's numerical precision (e.g., from 16-bit to 4-bit) to shrink memory usage and speed up inference.RerankerA model that takes an initial set of search results and reorders them by relevance, typically used to refine results from a faster but less accurate retrieval system.W4A16A quantization format where model weights are stored in 4-bit precision while calculations use 16-bit precision, balancing efficiency with accuracy.W4A16 QuantizationA specific quantization scheme where weights are stored in 4-bit precision while activations remain in 16-bit precision, balancing memory savings with accuracy.WeightsThe numerical parameters inside a neural network that determine how it processes input and generates output.

Capabilities

Capabilities

Use Case Fit

Glossary