Quantization - Python SDK

Quantization method reference

The Python SDK and docs are currently in beta. Report issues on GitHub.

Values

Name	Value
INT4	int4
INT8	int8
FP4	fp4
FP6	fp6
FP8	fp8
FP16	fp16
BF16	bf16
FP32	fp32
UNKNOWN	unknown