项目地址
下载模型
cd ~/Downloads
git clone --depth=1 https://www.modelscope.cn/ZhipuAI/chatglm2-6b.git创建测试项目
cd ~/code
mkdir ai/chatglm2-6b -p
cd ai/chatglm2-6b
mkdir ZhipuAI
ln -s ~/Downloads/chatglm2-6b ZhipuAI/
创建虚拟环境安装依赖
python3 -m venv venv
source venv/bin/activate
pip install transformers modelscope cpm_kernels -U
pip install torch
pip install accelerate
pip install sentencepiece完整的venv环境
(venv) yeqiang@yeqiang-MS-7B23:~/code/ai/chatglm2-6b$ pip list
Package Version
------------------------ ------------
accelerate 0.26.1
addict 2.4.0
aiohttp 3.9.1
aiosignal 1.3.1
aliyun-python-sdk-core 2.14.0
aliyun-python-sdk-kms 2.16.2
async-timeout 4.0.3
attrs 23.2.0
certifi 2023.11.17
cffi 1.16.0
charset-normalizer 3.3.2
cpm-kernels 1.0.11
crcmod 1.7
cryptography 41.0.7
datasets 2.16.1
dill 0.3.7
einops 0.7.0
filelock 3.13.1
frozenlist 1.4.1
fsspec 2023.10.0
gast 0.5.4
huggingface-hub 0.20.2
idna 3.6
importlib-metadata 7.0.1
Jinja2 3.1.3
jmespath 0.10.0
MarkupSafe 2.1.3
modelscope 1.11.0
mpmath 1.3.0
multidict 6.0.4
multiprocess 0.70.15
networkx 3.2.1
numpy 1.26.3
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
oss2 2.18.4
packaging 23.2
pandas 2.1.4
pillow 10.2.0
pip 22.0.2
platformdirs 4.1.0
psutil 5.9.7
pyarrow 14.0.2
pyarrow-hotfix 0.6
pycparser 2.21
pycryptodome 3.20.0
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 6.0.1
regex 2023.12.25
requests 2.31.0
safetensors 0.4.1
scipy 1.11.4
sentencepiece 0.1.99
setuptools 59.6.0
simplejson 3.19.2
six 1.16.0
sortedcontainers 2.4.0
sympy 1.12
tokenizers 0.15.0
tomli 2.0.1
torch 2.1.2
torchflow 0.0.3
tqdm 4.66.1
transformers 4.36.2
triton 2.1.0
typing_extensions 4.9.0
tzdata 2023.4
urllib3 2.1.0
xxhash 3.4.1
yapf 0.40.2
yarl 1.9.4
zipp 3.17.0
main.py
# 备注：最新模型版本要求modelscope >= 1.9.0
# pip install modelscope -U
from modelscope.utils.constant import Tasks
from modelscope import Model
from modelscope.pipelines import pipeline
model = Model.from_pretrained('ZhipuAI/chatglm2-6b', device_map='auto', revision='v1.0.12')
pipe = pipeline(task=Tasks.chat, model=model)
inputs = {'text':'你好', 'history': []}
result = pipe(inputs)
inputs = {'text':'介绍下清华大学', 'history': result['history']}
result = pipe(inputs)
执行
python进程cpu使用率100.3%持续了很长时间，没有启用多线程？
