> ## Content Index > Fetch the complete content index at: https://lucent.blog/llms.txt > Use this file to discover other available public pages before exploring further. # 华为910B 通过 vllm 部署 Qwen3.8 系列模型 - URL: https://lucent.blog/hua-wei-910b-tong-guo-vllm-bu-shu-qwen3-8-xi-lie-mo-xing/ - Published: 2026-09-02T05:05:01.000Z - Updated: 2026-09-02T05:05:00.000Z - Author: Lucent # **一、环境说明** 我这里的环境大概是这样： - 机器：华为 Ascend 910B - 系统：openEuler / CentOS 类 Linux - 驱动和 CANN：已经安装好 - 模型：Qwen3.5-27B - 部署方式：Docker Compose - 推理框架：vllm-ascend 先确认宿主机能看到 NPU： ``` npu-smi info ``` 如果能看到设备列表，说明驱动基本没问题。 例如： ``` +------------------------------------------------------------------------------------------------+ | npu-smi 25.2.1 Version: 25.2.1 | +---------------------------+---------------+----------------------------------------------------+ | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)| | Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) | +===========================+===============+====================================================+ | 0 910B2 | OK | 97.2 41 0 / 0 | | 0 | 0000:C1:00.0 | 0 0 / 0 64231/ 65536 | +===========================+===============+====================================================+ | 1 910B2 | OK | 93.3 43 0 / 0 | | 0 | 0000:01:00.0 | 0 0 / 0 64231/ 65536 | +===========================+===============+====================================================+ | 2 910B2 | OK | 93.5 39 0 / 0 | | 0 | 0000:C2:00.0 | 0 0 / 0 64231/ 65536 | +===========================+===============+====================================================+ | 3 910B2 | OK | 102.5 43 0 / 0 | | 0 | 0000:02:00.0 | 0 0 / 0 64231/ 65536 | +===========================+===============+====================================================+ | 4 910B2 | OK | 97.8 41 0 / 0 | | 0 | 0000:81:00.0 | 0 0 / 0 3390 / 65536 | +===========================+===============+====================================================+ | 5 910B2 | OK | 97.5 43 0 / 0 | | 0 | 0000:41:00.0 | 0 0 / 0 3390 / 65536 | +===========================+===============+====================================================+ | 6 910B2 | OK | 88.6 40 0 / 0 | | 0 | 0000:82:00.0 | 0 0 / 0 3390 / 65536 | +===========================+===============+====================================================+ | 7 910B2 | OK | 97.5 43 0 / 0 | | 0 | 0000:42:00.0 | 0 0 / 0 3390 / 65536 | +===========================+===============+====================================================+ +---------------------------+---------------+----------------------------------------------------+ | NPU Chip | Process id | Process name | Process memory(MB) | +===========================+===============+====================================================+ | 0 0 | 1316011 | VLLMWorker_TP | 60891 | +===========================+===============+====================================================+ | 1 0 | 1316012 | VLLMWorker_TP | 60891 | +===========================+===============+====================================================+ | 2 0 | 1316013 | VLLMWorker_TP | 60891 | +===========================+===============+====================================================+ | 3 0 | 1316014 | VLLMWorker_TP | 60891 | +===========================+===============+====================================================+ | No running processes found in NPU 4 | +===========================+===============+====================================================+ | No running processes found in NPU 5 | +===========================+===============+====================================================+ | No running processes found in NPU 6 | +===========================+===============+====================================================+ | No running processes found in NPU 7 | +===========================+===============+====================================================+ ``` # **二、准备模型缓存** 我不想让容器每次启动都重新下载模型，所以先在宿主机把模型缓存好。 我用的是 ModelScope： ``` pip install modelscope ``` 下载模型： ``` modelscope download --model Qwen/Qwen3.8-27B ``` 默认会下载到： ``` ~/.cache/modelscope/hub/models/Qwen/Qwen3.8-27B ``` 如果你的磁盘空间比较紧张，也可以提前设置： ``` export MODELSCOPE_CACHE=/mnt/disk1/modelscope ``` 这样模型就会下载到你指定的目录。 # **三、docker-compose 配置** 我最后用的是下面这份配置。 Ascend 文档明确说 **Qwen3.8-27B 是 vLLM-Ascend 0.23.0 首次支持**,所以使用最新版镜像 **quay.io/ascend/vllm-ascend:v0.23.0** 我使用 4 卡来运行，实测 2 卡也可以，但是速度较慢 ```yaml services: qwen38-27b: image: quay.io/ascend/services: qwen38-27b: image: quay.io/ascend/vllm-ascend:v0.23.0 container_name: qwen38-27b network_mode: host shm_size: "1g" restart: unless-stopped environment: VLLM_USE_MODELSCOPE: "true" # 限制使用 0~3 号 NPU ASCEND_RT_VISIBLE_DEVICES: "0,1,2,3" # 减少 NPU 显存碎片 PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True" # HCCL 通信缓存 HCCL_BUFFSIZE: "512" OMP_PROC_BIND: "false" OMP_NUM_THREADS: "1" TASK_QUEUE_ENABLE: "1" devices: - /dev/davinci0:/dev/davinci0 - /dev/davinci1:/dev/davinci1 - /dev/davinci2:/dev/davinci2 - /dev/davinci3:/dev/davinci3 - /dev/davinci_manager:/dev/davinci_manager - /dev/devmm_svm:/dev/devmm_svm - /dev/hisi_hdc:/dev/hisi_hdc volumes: # ModelScope 模型 - /mnt/disk1/modelscope/models:/root/.cache/modelscope/hub/models # Ascend Driver - /usr/local/dcmi:/usr/local/dcmi - /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool - /usr/local/bin/npu-smi:/usr/local/bin/npu-smi - /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ - /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info - /etc/ascend_install.info:/etc/ascend_install.info command: - vllm - serve - /root/.cache/modelscope/hub/models/Qwen/Qwen3___8-27B - --host - 0.0.0.0 - --port - "8000" - --served-model-name - Qwen - --tensor-parallel-size - "4" - --max-model-len - "262144" - --max-num-batched-tokens - "8192" - --gpu-memory-utilization - "0.90" - --trust-remote-code # Prefix KV Cache - --enable-prefix-caching # Qwen Thinking / Reasoning - --reasoning-parser - qwen3 # Function Calling - --tool-call-parser - qwen3_coder - --enable-auto-tool-choice # Ascend 异步调度 - --async-scheduling # Ascend Graph - --compilation-config - '{"cudagraph_mode":"FULL_DECODE_ONLY"}' container_name: qwen38-27b network_mode: host shm_size: "1g" restart: unless-stopped environment: VLLM_USE_MODELSCOPE: "true" # 限制使用 0~3 号 NPU ASCEND_RT_VISIBLE_DEVICES: "0,1,2,3" # 减少 NPU 显存碎片 PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True" # HCCL 通信缓存 HCCL_BUFFSIZE: "512" OMP_PROC_BIND: "false" OMP_NUM_THREADS: "1" TASK_QUEUE_ENABLE: "1" devices: - /dev/davinci0:/dev/davinci0 - /dev/davinci1:/dev/davinci1 - /dev/davinci2:/dev/davinci2 - /dev/davinci3:/dev/davinci3 - /dev/davinci_manager:/dev/davinci_manager - /dev/devmm_svm:/dev/devmm_svm - /dev/hisi_hdc:/dev/hisi_hdc volumes: # ModelScope 模型 - /mnt/disk1/modelscope/models:/root/.cache/modelscope/hub/models # Ascend Driver - /usr/local/dcmi:/usr/local/dcmi - /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool - /usr/local/bin/npu-smi:/usr/local/bin/npu-smi - /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ - /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info - /etc/ascend_install.info:/etc/ascend_install.info command: - vllm - serve - /root/.cache/modelscope/hub/models/Qwen/Qwen3___8-27B - --host - 0.0.0.0 - --port - "8000" - --served-model-name - Qwen - --tensor-parallel-size - "4" - --max-model-len - "262144" - --max-num-batched-tokens - "8192" - --gpu-memory-utilization - "0.90" - --trust-remote-code # Prefix KV Cache - --enable-prefix-caching # Qwen Thinking / Reasoning - --reasoning-parser - qwen3 # Function Calling - --tool-call-parser - qwen3_coder - --enable-auto-tool-choice # Ascend 异步调度 - --async-scheduling # Ascend Graph - --compilation-config - '{"cudagraph_mode":"FULL_DECODE_ONLY"}' ``` 启动： ``` docker-compose up -d ``` 查看日志： ``` docker-compose logs -f ``` 如果看到： ``` INFO:     Application startup complete. ``` 说明服务已经起来了。 # **四、测试一下接口** 服务起来以后，可以直接用 curl 测。 最简单的聊天： ``` curl http://127.0.0.1:8000/v1/chat/completions \  -H "Content-Type: application/json" \  -d '{    "model": "Qwen",    "messages": [     {        "role": "user",        "content": "你好，介绍一下你自己"     }   ] }' ``` # **五、测试视觉能力** Qwen3.5 是统一多模态模型，原生支持视觉(图片、视频输入)。 ``` curl http://127.0.0.1:8000/v1/chat/completions \  -H "Content-Type: application/json" \  -d '{    "model": "Qwen",    "messages": [     {        "role": "user",        "content": [         {            "type": "text",            "text": "图里有什么？"         },         {            "type": "image_url",            "image_url": {              "url": "https://your-image-url"           }         }       ]     }   ] }' ``` # **六、测试工具调用** ``` curl http://127.0.0.1:8000/v1/chat/completions \  -H "Content-Type: application/json" \  -d '{    "model": "Qwen",    "messages": [     {        "role": "user",        "content": "帮我查一下北京今天的天气"     }   ],    "tools": [     {        "type": "function",        "function": {          "name": "get_weather",          "description": "获取天气",          "parameters": {            "type": "object",            "properties": {              "city": {                "type": "string"             }           },            "required": ["city"]         }       }     }   ],    "tool_choice": "auto" }' ``` 如果返回里出现： ``` "tool_calls": [ {    "function": {      "name": "get_weather",      "arguments": "{\"city\":\"北京\"}"   } } ] ``` 说明工具调用能力已经正常了。