Supported head sizes are: [64, 80, 96, 112, 128, 256].
错误信息:
Head size 160 is not supported by PagedAttention. Supported head sizes are: [64, 80, 96, 112, 128, 256].
解决方法:
vim /root/miniconda3/envs/myvllm/lib/python3.10/site-packages/vllm/attention/ops/paged_attn.py
---
def get_supported_head_sizes() -> List[int]:
return [64, 80, 96, 112, 128, 160, 256]
---