本文记录清华大学开放使用的ChatGLM2-6B的完整搭建过程，在本地机器32G内存，无显卡，可以缓慢运行。要求python3.7+环境，本文用的是python 3.9 + redhat 7.9。如果你没有大内存机器，就别测试了。如果没有8G显存的nvidia显卡，就别太指望性能。它支持多个显卡。
目录导航
搭建结果演示
文件下载
所有的内容都在这里下载
https://huggingface.co/THUDM/chatglm2-6b/tree/main
清华大学有个镜像，里面几个大的bin文件，可以从镜像下。 但其它的文件还是要从上面那里下载。
https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/?p=%2Fchatglm2-6b&mode=list
下载后的内容，放到一个目录下，比如
[root@vm249 ChatGLM]# ll chatglm2-6b
total 12195696
-rw-r--r--. 1 root root 1231 Jul 19 00:12 config.json
-rw-r--r--. 1 root root 2246 Jul 19 00:11 configuration_chatglm.py
-rw-r--r--. 1 root root 50717 Jul 19 00:11 modeling_chatglm.py
-rw-r--r--. 1 root root 1827780615 Jul 19 00:10 pytorch_model-00001-of-00007.bin
-rw-r--r--. 1 root root 1968299005 Jul 19 00:02 pytorch_model-00002-of-00007.bin
-rw-r--r--. 1 root root 1927414561 Jul 19 00:06 pytorch_model-00003-of-00007.bin
-rw-r--r--. 1 root root 1815225523 Jul 19 00:08 pytorch_model-00004-of-00007.bin
-rw-r--r--. 1 root root 1968299069 Jul 19 00:00 pytorch_model-00005-of-00007.bin
-rw-r--r--. 1 root root 1927414561 Jul 19 00:04 pytorch_model-00006-of-00007.bin
-rw-r--r--. 1 root root 1052808067 Jul 19 00:11 pytorch_model-00007-of-00007.bin
-rw-r--r--. 1 root root 20438 Jul 19 00:24 pytorch_model.bin.index.json
-rw-r--r--. 1 root root 14692 Jul 19 00:11 quantization.py
-rw-r--r--. 1 root root 10002 Jul 19 00:11 tokenization_chatglm.py
-rw-r--r--. 1 root root 244 Jul 19 00:11 tokenizer_config.json
-rw-r--r--. 1 root root 1018370 Jul 19 00:11 tokenizer.model
[root@vm249 ChatGLM]#
修改配置文件config.json，将里面的路径改成正确的。如果不写绝对路径，那就只保留最后一个目录。
"_name_or_path": "/opt/ChatGLM/chatglm2-6b",Python3.9 环境安装
如果有redhat 8，建议直接使用。 我本例用的redhat7 , 自带的是3.6，所以要自己升级到python3.9。
安装依赖包
yum install libffi-dev
yum install bzip2-devel
安装python3.9
过程不描述了。 安装后一些命令默认是带版本好的，比如python3.9 pip3.9。
./configure
make
make install升级urllib3
自带的1.10.2卸载了，升级到1.23。
[root@vm249 chatglm2-6b-int4]# rpm -qa|grep urllib3
python-urllib3-1.10.2-7.el7.noarch
[root@vm249 chatglm2-6b-int4]# pip3 uninstall urllib3
Found existing installation: urllib3 2.0.3
Uninstalling urllib3-2.0.3:
Would remove:
/usr/local/lib/python3.9/site-packages/urllib3-2.0.3.dist-info/*
/usr/local/lib/python3.9/site-packages/urllib3/*
Proceed (y/n)? y
Successfully uninstalled urllib3-2.0.3
[root@vm249 chatglm2-6b-int4]# pip install urllib3==1.23
bash: pip: command not found...
[root@vm249 chatglm2-6b-int4]# pip3.9 install urllib3==1.23
Collecting urllib3==1.23
Downloading urllib3-1.23-py2.py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 1.3 MB/s
Installing collected packages: urllib3
Successfully installed urllib3-1.23
WARNING: You are using pip version 20.2.3; however, version 23.2 is available.
You should consider upgrading via the '/usr/local/bin/python3.9 -m pip install --upgrade pip' command.
[root@vm249 chatglm2-6b-int4]#ChatGLM2-6B安装依赖
会自动下载并安装很多，特别是nvidia的显卡驱动。
pip3.9 install -r requirements.txt
[root@vm249 ChatGLM]# pip3.9 install -r requirements.txt
Collecting protobuf
Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
|████████████████████████████████| 304 kB 789 kB/s
Collecting transformers==4.30.2
Downloading transformers-4.30.2-py3-none-any.whl (7.2 MB)
|████████████████████████████████| 7.2 MB 2.6 MB/s
Collecting cpm_kernels
Downloading cpm_kernels-1.0.11-py3-none-any.whl (416 kB)
|████████████████████████████████| 416 kB 787 kB/s
Collecting torch>=2.0
Downloading torch-2.0.1-cp39-cp39-manylinux1_x86_64.whl (619.9 MB)
|████████████████████████████████| 619.9 MB 29 kB/s
Collecting gradio
Downloading gradio-3.37.0-py3-none-any.whl (19.8 MB)
|████████████████████████████████| 19.8 MB 2.6 MB/s
Collecting mdtex2html
Downloading mdtex2html-1.2.0-py3-none-any.whl (13 kB)
Collecting sentencepiece
Downloading sentencepiece-0.1.99-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 4.1 MB/s
Collecting accelerate
Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)
|████████████████████████████████| 244 kB 2.9 MB/s
Collecting sse-starlette
Downloading sse_starlette-1.6.1-py3-none-any.whl (9.6 kB)
Collecting streamlit>=1.24.0
Downloading streamlit-1.24.1-py2.py3-none-any.whl (8.9 MB)
|████████████████████████████████| 8.9 MB 2.6 MB/s
Collecting regex!=2019.12.17
Downloading regex-2023.6.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB)
|████████████████████████████████| 769 kB 3.3 MB/s
Collecting requests
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 852 kB/s
Collecting filelock
Downloading filelock-3.12.2-py3-none-any.whl (10 kB)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
|████████████████████████████████| 7.8 MB 3.4 MB/s
Collecting pyyaml>=5.1
Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB)
|████████████████████████████████| 738 kB 3.3 MB/s
Collecting safetensors>=0.3.1
Downloading safetensors-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 3.4 MB/s
Collecting packaging>=20.0
Downloading packaging-23.1-py3-none-any.whl (48 kB)
|████████████████████████████████| 48 kB 3.1 MB/s
Collecting numpy>=1.17
Downloading numpy-1.25.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.7 MB)
|████████████████████████████████| 17.7 MB 3.4 MB/s
Collecting huggingface-hub<1.0,>=0.14.1
Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
|████████████████████████████████| 268 kB 3.2 MB/s
Collecting tqdm>=4.27
Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 3.4 MB/s
Collecting sympy
Downloading sympy-1.12-py3-none-any.whl (5.7 MB)
|████████████████████████████████| 5.7 MB 3.4 MB/s
Collecting nvidia-cusparse-cu11==11.7.4.91; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)
|████████████████████████████████| 173.2 MB 65 kB/s
Collecting jinja2
Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 3.2 MB/s
Collecting nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
|████████████████████████████████| 21.0 MB 869 kB/s
Collecting nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
|████████████████████████████████| 849 kB 3.5 MB/s
Collecting nvidia-curand-cu11==10.2.10.91; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)
|████████████████████████████████| 54.6 MB 3.4 MB/s
Collecting nvidia-cuda-cupti-cu11==11.7.101; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)
|████████████████████████████████| 11.8 MB 2.6 MB/s
Collecting networkx
Downloading networkx-3.1-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 137 kB/s
Collecting nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
|████████████████████████████████| 557.1 MB 22 kB/s
Collecting nvidia-nccl-cu11==2.14.3; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)
|████████████████████████████████| 177.1 MB 3.4 MB/s
Collecting triton==2.0.0; platform_system == "Linux" and platform_machine == "x86_64"
Downloading triton-2.0.0-1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)
|████████████████████████████████| 63.3 MB 2.3 MB/s
Collecting typing-extensions
Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Collecting nvidia-cusolver-cu11==11.4.0.1; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)
|████████████████████████████████| 102.6 MB 2.6 MB/s
Collecting nvidia-nvtx-cu11==11.7.91; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)
|████████████████████████████████| 98 kB 2.5 MB/s
Collecting nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
|████████████████████████████████| 317.1 MB 37 kB/s
Collecting nvidia-cufft-cu11==10.9.0.58; platform_system == "Linux" and platform_machine == "x86_64"
Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)
|████████████████████████████████| 168.4 MB 136 kB/s
Collecting orjson~=3.0
Downloading orjson-3.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
|████████████████████████████████| 138 kB 2.9 MB/s
Collecting pillow<11.0,>=8.0
Downloading Pillow-10.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
|████████████████████████████████| 3.3 MB 3.4 MB/s
Collecting aiofiles<24.0,>=22.0
Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)
Collecting semantic-version~=2.0
Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting pandas<3.0,>=1.0
Downloading pandas-2.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.4 MB)
|████████████████████████████████| 12.4 MB 2.6 MB/s
Collecting websockets<12.0,>=10.0
Downloading websockets-11.0.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
|████████████████████████████████| 129 kB 3.3 MB/s
Collecting aiohttp~=3.0
Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
|████████████████████████████████| 1.0 MB 3.6 MB/s
Collecting pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4
Downloading pydantic-2.0.3-py3-none-any.whl (364 kB)
|████████████████████████████████| 364 kB 2.9 MB/s
Collecting python-multipart
Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)
|████████████████████████████████| 45 kB 5.6 MB/s
Collecting httpx
Downloading httpx-0.24.1-py3-none-any.whl (75 kB)
|████████████████████████████████| 75 kB 4.2 MB/s
Collecting markdown-it-py[linkify]>=2.0.0
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
|████████████████████████████████| 87 kB 2.6 MB/s
Collecting mdit-py-plugins<=0.3.3
Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)
|████████████████████████████████| 50 kB 6.0 MB/s
Collecting pydub
Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting matplotlib~=3.0
Downloading matplotlib-3.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
|████████████████████████████████| 11.6 MB 2.6 MB/s
Collecting altair<6.0,>=4.2.0
Downloading altair-5.0.1-py3-none-any.whl (471 kB)
|████████████████████████████████| 471 kB 3.3 MB/s
Collecting markupsafe~=2.0
Downloading MarkupSafe-2.1.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting fastapi
Downloading fastapi-0.100.0-py3-none-any.whl (65 kB)
|████████████████████████████████| 65 kB 2.9 MB/s
Collecting gradio-client>=0.2.10
Downloading gradio_client-0.2.10-py3-none-any.whl (288 kB)
|████████████████████████████████| 288 kB 4.4 MB/s
Collecting uvicorn>=0.14.0
Downloading uvicorn-0.23.1-py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 4.1 MB/s
Collecting ffmpy
Downloading ffmpy-0.3.1.tar.gz (5.5 kB)
Collecting latex2mathml
Downloading latex2mathml-3.76.0-py3-none-any.whl (73 kB)
|████████████████████████████████| 73 kB 2.7 MB/s
Collecting markdown
Downloading Markdown-3.4.3-py3-none-any.whl (93 kB)
|████████████████████████████████| 93 kB 2.6 MB/s
Collecting psutil
Downloading psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
|████████████████████████████████| 282 kB 3.3 MB/s
Collecting starlette
Downloading starlette-0.30.0-py3-none-any.whl (68 kB)
|████████████████████████████████| 68 kB 4.2 MB/s
Collecting toml<2
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting blinker<2,>=1.0.0
Downloading blinker-1.6.2-py3-none-any.whl (13 kB)
Collecting pympler<2,>=0.9
Downloading Pympler-1.0.1-py3-none-any.whl (164 kB)
|████████████████████████████████| 164 kB 4.0 MB/s
Collecting gitpython!=3.1.19,<4,>=3
Downloading GitPython-3.1.32-py3-none-any.whl (188 kB)
|████████████████████████████████| 188 kB 4.7 MB/s
Collecting rich<14,>=10.11.0
Downloading rich-13.4.2-py3-none-any.whl (239 kB)
|████████████████████████████████| 239 kB 3.3 MB/s
Collecting pydeck<1,>=0.1.dev5
Downloading pydeck-0.8.1b0-py2.py3-none-any.whl (4.8 MB)
|████████████████████████████████| 4.8 MB 1.5 MB/s
Collecting tornado<7,>=6.0.3
Downloading tornado-6.3.2-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (426 kB)
|████████████████████████████████| 426 kB 3.4 MB/s
Collecting python-dateutil<3,>=2
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
|████████████████████████████████| 247 kB 4.5 MB/s
Collecting cachetools<6,>=4.0
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting watchdog; platform_system != "Darwin"
Downloading watchdog-3.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)
|████████████████████████████████| 82 kB 156 kB/s
Collecting validators<1,>=0.2
Downloading validators-0.20.0.tar.gz (30 kB)
Collecting tenacity<9,>=8.0.0
Downloading tenacity-8.2.2-py3-none-any.whl (24 kB)
Collecting pyarrow>=4.0
Downloading pyarrow-12.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.0 MB)
|████████████████████████████████| 39.0 MB 2.6 MB/s
Collecting importlib-metadata<7,>=1.4
Downloading importlib_metadata-6.8.0-py3-none-any.whl (22 kB)
Collecting tzlocal<5,>=1.1
Downloading tzlocal-4.3.1-py3-none-any.whl (20 kB)
Collecting click<9,>=7.0
Downloading click-8.1.6-py3-none-any.whl (97 kB)
|████████████████████████████████| 97 kB 3.3 MB/s
Collecting idna<4,>=2.5
Downloading idna-3.4-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 198 kB/s
Collecting urllib3<3,>=1.21.1
Downloading urllib3-2.0.3-py3-none-any.whl (123 kB)
|████████████████████████████████| 123 kB 4.5 MB/s
Collecting charset-normalizer<4,>=2
Downloading charset_normalizer-3.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (202 kB)
|████████████████████████████████| 202 kB 3.4 MB/s
Collecting certifi>=2017.4.17
Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
|████████████████████████████████| 156 kB 3.3 MB/s
Collecting fsspec
Downloading fsspec-2023.6.0-py3-none-any.whl (163 kB)
|████████████████████████████████| 163 kB 3.4 MB/s
Collecting mpmath>=0.19
Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
|████████████████████████████████| 536 kB 4.2 MB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from nvidia-cusparse-cu11==11.7.4.91; platform_system == "Linux" and platform_machine == "x86_64"->torch>=2.0->-r requirements.txt (line 4)) (49.2.1)
Collecting wheel
Downloading wheel-0.40.0-py3-none-any.whl (64 kB)
|████████████████████████████████| 64 kB 3.5 MB/s
Collecting lit
Downloading lit-16.0.6.tar.gz (153 kB)
|████████████████████████████████| 153 kB 4.3 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Collecting cmake
Downloading cmake-3.26.4-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (24.0 MB)
|████████████████████████████████| 24.0 MB 2.5 MB/s
Collecting tzdata>=2022.1
Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
|████████████████████████████████| 341 kB 3.0 MB/s
Collecting pytz>=2020.1
Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
|████████████████████████████████| 502 kB 4.4 MB/s
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.4.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (228 kB)
|████████████████████████████████| 228 kB 3.6 MB/s
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
|████████████████████████████████| 114 kB 4.6 MB/s
Collecting yarl<2.0,>=1.0
Downloading yarl-1.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (269 kB)
|████████████████████████████████| 269 kB 4.4 MB/s
Collecting attrs>=17.3.0
Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 4.8 MB/s
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting pydantic-core==2.3.0
Downloading pydantic_core-2.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.9 MB)
|████████████████████████████████| 1.9 MB 3.9 MB/s
Collecting annotated-types>=0.4.0
Downloading annotated_types-0.5.0-py3-none-any.whl (11 kB)
Collecting httpcore<0.18.0,>=0.15.0
Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)
|████████████████████████████████| 74 kB 3.4 MB/s
Collecting sniffio
Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting mdurl~=0.1
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting linkify-it-py<3,>=1; extra == "linkify"
Downloading linkify_it_py-2.0.2-py3-none-any.whl (19 kB)
Collecting importlib-resources>=3.2.0; python_version < "3.10"
Downloading importlib_resources-6.0.0-py3-none-any.whl (31 kB)
Collecting pyparsing<3.1,>=2.3.1
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 3.9 MB/s
Collecting cycler>=0.10
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
|████████████████████████████████| 1.6 MB 4.0 MB/s
Collecting contourpy>=1.0.1
Downloading contourpy-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
|████████████████████████████████| 300 kB 3.4 MB/s
Collecting fonttools>=4.22.0
Downloading fonttools-4.41.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.4 MB)
|████████████████████████████████| 4.4 MB 3.5 MB/s
Collecting toolz
Downloading toolz-0.12.0-py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 2.8 MB/s
Collecting jsonschema>=3.0
Downloading jsonschema-4.18.4-py3-none-any.whl (80 kB)
|████████████████████████████████| 80 kB 5.2 MB/s
Collecting h11>=0.8
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 7.7 MB/s
Collecting anyio<5,>=3.4.0
Downloading anyio-3.7.1-py3-none-any.whl (80 kB)
|████████████████████████████████| 80 kB 5.2 MB/s
Collecting gitdb<5,>=4.0.1
Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 1.9 MB/s
Collecting pygments<3.0.0,>=2.13.0
Downloading Pygments-2.15.1-py3-none-any.whl (1.1 MB)
|████████████████████████████████| 1.1 MB 4.5 MB/s
Collecting six>=1.5
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting decorator>=3.4.0
Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting zipp>=0.5
Downloading zipp-3.16.2-py3-none-any.whl (7.2 kB)
Collecting pytz-deprecation-shim
Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)
Collecting uc-micro-py
Downloading uc_micro_py-1.0.2-py3-none-any.whl (6.2 kB)
Collecting jsonschema-specifications>=2023.03.6
Downloading jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Collecting referencing>=0.28.4
Downloading referencing-0.30.0-py3-none-any.whl (25 kB)
Collecting rpds-py>=0.7.1
Downloading rpds_py-0.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
|████████████████████████████████| 1.2 MB 3.9 MB/s
Collecting exceptiongroup; python_version < "3.11"
Downloading exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Collecting smmap<6,>=3.0.1
Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Using legacy 'setup.py install' for ffmpy, since package 'wheel' is not installed.
Using legacy 'setup.py install' for validators, since package 'wheel' is not installed.
Building wheels for collected packages: lit
Building wheel for lit (PEP 517) ... done
Created wheel for lit: filename=lit-16.0.6-py3-none-any.whl size=93582 sha256=99bebe8b690a70de2742271e05289e9d109e63b5dfc98170d951c992e83fe477
Stored in directory: /root/.cache/pip/wheels/a5/36/d6/cac2e6fb891889b33a548f2fddb8b4b7726399aaa2ed32b188
Successfully built lit
Installing collected packages: protobuf, regex, idna, urllib3, charset-normalizer, certifi, requests, filelock, tokenizers, pyyaml, safetensors, packaging, numpy, tqdm, typing-extensions, fsspec, huggingface-hub, transformers, cpm-kernels, mpmath, sympy, wheel, nvidia-cusparse-cu11, markupsafe, jinja2, nvidia-cuda-nvrtc-cu11, nvidia-cuda-runtime-cu11, nvidia-curand-cu11, nvidia-cuda-cupti-cu11, networkx, nvidia-cublas-cu11, nvidia-cudnn-cu11, nvidia-nccl-cu11, lit, cmake, triton, nvidia-cusolver-cu11, nvidia-nvtx-cu11, nvidia-cufft-cu11, torch, orjson, pillow, aiofiles, semantic-version, tzdata, six, python-dateutil, pytz, pandas, websockets, frozenlist, aiosignal, multidict, yarl, attrs, async-timeout, aiohttp, pydantic-core, annotated-types, pydantic, python-multipart, exceptiongroup, sniffio, anyio, h11, httpcore, httpx, mdurl, uc-micro-py, linkify-it-py, markdown-it-py, mdit-py-plugins, pydub, zipp, importlib-resources, pyparsing, cycler, kiwisolver, contourpy, fonttools, matplotlib, toolz, rpds-py, referencing, jsonschema-specifications, jsonschema, altair, starlette, fastapi, gradio-client, click, uvicorn, ffmpy, gradio, latex2mathml, importlib-metadata, markdown, mdtex2html, sentencepiece, psutil, accelerate, sse-starlette, toml, blinker, pympler, smmap, gitdb, gitpython, pygments, rich, pydeck, tornado, cachetools, watchdog, decorator, validators, tenacity, pyarrow, pytz-deprecation-shim, tzlocal, streamlit
Running setup.py install for ffmpy ... done
Running setup.py install for validators ... done
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
mdit-py-plugins 0.3.3 requires markdown-it-py<3.0.0,>=1.0.0, but you'll have markdown-it-py 3.0.0 which is incompatible.
fastapi 0.100.0 requires starlette<0.28.0,>=0.27.0, but you'll have starlette 0.30.0 which is incompatible.
streamlit 1.24.1 requires pillow<10,>=6.2.0, but you'll have pillow 10.0.0 which is incompatible.
Successfully installed accelerate-0.21.0 aiofiles-23.1.0 aiohttp-3.8.4 aiosignal-1.3.1 altair-5.0.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.2 attrs-23.1.0 blinker-1.6.2 cachetools-5.3.1 certifi-2023.5.7 charset-normalizer-3.2.0 click-8.1.6 cmake-3.26.4 contourpy-1.1.0 cpm-kernels-1.0.11 cycler-0.11.0 decorator-5.1.1 exceptiongroup-1.1.2 fastapi-0.100.0 ffmpy-0.3.1 filelock-3.12.2 fonttools-4.41.0 frozenlist-1.4.0 fsspec-2023.6.0 gitdb-4.0.10 gitpython-3.1.32 gradio-3.37.0 gradio-client-0.2.10 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 huggingface-hub-0.16.4 idna-3.4 importlib-metadata-6.8.0 importlib-resources-6.0.0 jinja2-3.1.2 jsonschema-4.18.4 jsonschema-specifications-2023.7.1 kiwisolver-1.4.4 latex2mathml-3.76.0 linkify-it-py-2.0.2 lit-16.0.6 markdown-3.4.3 markdown-it-py-3.0.0 markupsafe-2.1.3 matplotlib-3.7.2 mdit-py-plugins-0.3.3 mdtex2html-1.2.0 mdurl-0.1.2 mpmath-1.3.0 multidict-6.0.4 networkx-3.1 numpy-1.25.1 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 orjson-3.9.2 packaging-23.1 pandas-2.0.3 pillow-10.0.0 protobuf-4.23.4 psutil-5.9.5 pyarrow-12.0.1 pydantic-2.0.3 pydantic-core-2.3.0 pydeck-0.8.1b0 pydub-0.25.1 pygments-2.15.1 pympler-1.0.1 pyparsing-3.0.9 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3 pytz-deprecation-shim-0.1.0.post0 pyyaml-6.0.1 referencing-0.30.0 regex-2023.6.3 requests-2.31.0 rich-13.4.2 rpds-py-0.9.2 safetensors-0.3.1 semantic-version-2.10.0 sentencepiece-0.1.99 six-1.16.0 smmap-5.0.0 sniffio-1.3.0 sse-starlette-1.6.1 starlette-0.30.0 streamlit-1.24.1 sympy-1.12 tenacity-8.2.2 tokenizers-0.13.3 toml-0.10.2 toolz-0.12.0 torch-2.0.1 tornado-6.3.2 tqdm-4.65.0 transformers-4.30.2 triton-2.0.0 typing-extensions-4.7.1 tzdata-2023.3 tzlocal-4.3.1 uc-micro-py-1.0.2 urllib3-2.0.3 uvicorn-0.23.1 validators-0.20.0 watchdog-3.0.0 websockets-11.0.3 wheel-0.40.0 yarl-1.9.2 zipp-3.16.2
WARNING: You are using pip version 20.2.3; however, version 23.2 is available.
You should consider upgrading via the '/usr/local/bin/python3.9 -m pip install --upgrade pip' command.
You have new mail in /var/spool/mail/root
[root@vm249 ChatGLM]#OPENSSL 1.1.1从1.0升级
要求版本是1.1.1+， 而redhat 7是1.0的，所以要升级。
升级后，有2个Link要做一下，否则运行openssl会报so找不到。
openssl-1.1.1u.tar.gz
./config
make
make install
[root@vm249 openssl-1.1.1u]# ln -s /usr/local/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1
[root@vm249 openssl-1.1.1u]# ln -s /usr/local/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1
[root@vm249 openssl-1.1.1u]#运行ChatGLM2-6B
参考 https://github.com/THUDM/ChatGLM2-6B
注意，如下的第三行代码，改成了float(),因为没有显卡，用CPU模拟。
在文件夹同级运行，比如/opt/ChatGLM/
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("chatglm2-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("chatglm2-6b",trust_remote_code=True).float()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)图形界面
在https://github.com/THUDM/ChatGLM2-6B 有web_demo2.py。 放到下载文件夹的同级即可。比如/opt/ChatGLM/
启动服务，然后用提示的连接访问，分本地和外部访问两种。
[root@vm249 ChatGLM]# streamlit run web_demo2.py
You can now view your Streamlit app in your browser.
Local URL: http://localhost:8501
Network URL: http://172.16.3.249:8501
总结
直接用ChatGLM2-6B完整版，不要用那个int4的，我本地启动报错。 忽略错误后，回答问题乱七八糟。