前言

在阿里云docker虚拟环境下安装了一次pytorch,踩了一堆坑,记录一下。
这次想在阿里的国产arm环境下搭建一个python3.8 + pytorch1.10.2环境。由于torch都更新2.0了,旧版本的arm环境又得费劲装了

环境说明

服务器环境

阿里云

・实例: 1核 2G ecs.c8y ecs-7・I/O 优化实例: I/O 优化实例・系统盘: 增强型SSD云盘 /dev/xvda 40GB 模块属性 PL0・操作系统: Alibaba Cloud Linux 3.2104 LTS 64位 ARM版 Linux 64位

目标环境

python==3.8.16
torch==1.10.2

详细版本

系统和CPU环境如下所示

cat /proc/versionLinux version 5.10.134-13.1.al8.aarch64 (mockbuild@r75e11320.cm10.tbsite.net) (gcc (GCC) 10.2.1 20200825 (Alibaba 10.2.1-3 2.32), GNU ld version 2.35-12.2.al8) #1 SMP Mon Feb 6 07:07:12 UTC 2023uname -aLinux 810a9ecee45c 5.10.134-13.1.al8.aarch64 #1 SMP Mon Feb 6 07:07:12 UTC 2023 aarch64 GNU/Linux

查看Linux内核版本的命令可参考如何查看Linux系统版本?

一、拉取docker镜像

这里选取docker官方提供的python3.8环境镜像,地址

镜像拉取指令

docker pull python:3.8.16-buster# 这个镜像需要手动添加yum。使用下面语句更新apt-get,安装yumapt-get -y updateapt-get install yum

docker使用方式可参考docker简明使用指南

二、选取对应版本离线包.whl

在线安装

这里因为已经无法用在线安装的方式选取对应版本,所以采用离线方式安装,较新版本可以先查询在线安装版本:

pip install torch==

产生错误:无可用版本

pip install torch==Looking in indexes: http://mirrors.aliyun.com/pypi/simple/ERROR: Could not find a version that satisfies the requirement torch== (from versions: none)ERROR: No matching distribution found for torch==

pytorch 官方并未提供 arm 平台的安装包
于是安装官方页面提供的命令是无论如何也无法安装成功的
只要cpu是arm芯片,无论是windows linux maxos都不行
解决方法有两个
1、下载源代码然后进行手动编译(太过复杂和耗时)
2、用社区其他人编译好的版本,这个就比较简单了

下载离线安装包

这里直接拿来主义,根据系统版本下载对应的离线版本安装包.whl
清华源里torch下载地址
这里从清华源中找到安装包torch-1.10.2-cp38-cp38-manylinux2014_aarch64.whl
其中cp38代表python 3.8,aarch64即arm64

三、安装离线包

这里尝试直接安装离线包

pip install XXX.whl#如当前准备安装的 torch1.10.2 的linux安装包pip install torch-1.10.2-cp38-cp38-manylinux2014_aarch64.whl

产生错误:is not a supported wheel on this platform

whl安装包不被该平台支持(其实不是)

pip install torch-1.10.2-cp38-cp38-manylinux2014_aarch64.whl Looking in indexes: http://mirrors.aliyun.com/pypi/simple/ERROR: torch-1.10.2-cp38-cp38-manylinux2014_aarch64.whl is not a supported wheel on this platform.

但其实我们发现这里linux只有这个版本的pytorch安装包,没得选

解决方案:修改安装包名称

其实只是安装包的后缀不对,系统判断识别得是whl文件后缀(好傻)
首先查询

pip debug --verbose

(语句适用于pip 20.0等新版本的pip,之前pip版本需要参考这个)
输入这个命令后,会显示大量信息:

$ pip debug --verboseWARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.pip version: pip 23.0.1 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)sys.version: 3.8.16 (default, Mar 29 2023, 23:56:55) [GCC 12.2.1 20220924]sys.executable: /usr/local/bin/pythonsys.getdefaultencoding: utf-8sys.getfilesystemencoding: utf-8locale.getpreferredencoding: UTF-8sys.platform: linuxsys.implementation:name: cpython'cert' config value: globalREQUESTS_CA_BUNDLE: NoneCURL_CA_BUNDLE: Nonepip._vendor.certifi.where(): /usr/local/lib/python3.8/site-packages/pip/_vendor/certifi/cacert.pempip._vendor.DEBUNDLED: Falsevendored library versions:CacheControl==0.12.11colorama==0.4.6distlib==0.3.6distro==1.8.0msgpack==1.0.4packaging==21.3platformdirs==2.6.2pyparsing==3.0.9pyproject-hooks==1.0.0requests==2.28.2certifi==2022.12.07chardet==5.1.0idna==3.4urllib3==1.26.14rich==12.6.0 (Unable to locate actual module version, using vendor.txt specified version)pygments==2.13.0typing_extensions==4.4.0 (Unable to locate actual module version, using vendor.txt specified version)resolvelib==0.8.1setuptools==44.0.0 (Unable to locate actual module version, using vendor.txt specified version)six==1.16.0tenacity==8.1.0 (Unable to locate actual module version, using vendor.txt specified version)tomli==2.0.1webencodings==0.5.1 (Unable to locate actual module version, using vendor.txt specified version)Compatible tags: 87cp38-cp38-musllinux_1_2_aarch64cp38-cp38-musllinux_1_1_aarch64cp38-cp38-musllinux_1_0_aarch64cp38-cp38-linux_aarch64cp38-abi3-musllinux_1_2_aarch64cp38-abi3-musllinux_1_1_aarch64cp38-abi3-musllinux_1_0_aarch64cp38-abi3-linux_aarch64cp38-none-musllinux_1_2_aarch64cp38-none-musllinux_1_1_aarch64cp38-none-musllinux_1_0_aarch64cp38-none-linux_aarch64cp37-abi3-musllinux_1_2_aarch64cp37-abi3-musllinux_1_1_aarch64cp37-abi3-musllinux_1_0_aarch64cp37-abi3-linux_aarch64cp36-abi3-musllinux_1_2_aarch64cp36-abi3-musllinux_1_1_aarch64cp36-abi3-musllinux_1_0_aarch64cp36-abi3-linux_aarch64cp35-abi3-musllinux_1_2_aarch64cp35-abi3-musllinux_1_1_aarch64cp35-abi3-musllinux_1_0_aarch64cp35-abi3-linux_aarch64cp34-abi3-musllinux_1_2_aarch64cp34-abi3-musllinux_1_1_aarch64cp34-abi3-musllinux_1_0_aarch64cp34-abi3-linux_aarch64cp33-abi3-musllinux_1_2_aarch64cp33-abi3-musllinux_1_1_aarch64cp33-abi3-musllinux_1_0_aarch64cp33-abi3-linux_aarch64cp32-abi3-musllinux_1_2_aarch64cp32-abi3-musllinux_1_1_aarch64cp32-abi3-musllinux_1_0_aarch64cp32-abi3-linux_aarch64py38-none-musllinux_1_2_aarch64py38-none-musllinux_1_1_aarch64py38-none-musllinux_1_0_aarch64py38-none-linux_aarch64py3-none-musllinux_1_2_aarch64py3-none-musllinux_1_1_aarch64py3-none-musllinux_1_0_aarch64py3-none-linux_aarch64py37-none-musllinux_1_2_aarch64py37-none-musllinux_1_1_aarch64py37-none-musllinux_1_0_aarch64py37-none-linux_aarch64py36-none-musllinux_1_2_aarch64py36-none-musllinux_1_1_aarch64py36-none-musllinux_1_0_aarch64py36-none-linux_aarch64py35-none-musllinux_1_2_aarch64py35-none-musllinux_1_1_aarch64py35-none-musllinux_1_0_aarch64py35-none-linux_aarch64py34-none-musllinux_1_2_aarch64py34-none-musllinux_1_1_aarch64py34-none-musllinux_1_0_aarch64py34-none-linux_aarch64py33-none-musllinux_1_2_aarch64py33-none-musllinux_1_1_aarch64py33-none-musllinux_1_0_aarch64py33-none-linux_aarch64py32-none-musllinux_1_2_aarch64py32-none-musllinux_1_1_aarch64py32-none-musllinux_1_0_aarch64py32-none-linux_aarch64py31-none-musllinux_1_2_aarch64py31-none-musllinux_1_1_aarch64py31-none-musllinux_1_0_aarch64py31-none-linux_aarch64py30-none-musllinux_1_2_aarch64py30-none-musllinux_1_1_aarch64py30-none-musllinux_1_0_aarch64py30-none-linux_aarch64cp38-none-anypy38-none-anypy3-none-anypy37-none-anypy36-none-anypy35-none-anypy34-none-anypy33-none-anypy32-none-anypy31-none-anypy30-none-any

这里我们看到Compatible tags字样,这些就是当前Python版本可以适配的标签,也就是可以匹配下面这些文件名。
发现没有我们现有安装包后缀,尝试修改安装包后缀。
看起来这个后缀比较合适 cp38-abi3-linux_aarch64。

cp torch-1.10.2-cp38-cp38-manylinux2014_aarch64.whl torch-1.10.2-cp38-abi3-linux_aarch64.whl

再次尝试安装

pip install torch-1.10.2-cp38-abi3-linux_aarch64.whl

显示如下信息

pip install torch-1.10.2-cp38-abi3-linux_aarch64.whlLooking in indexes: http://mirrors.aliyun.com/pypi/simple/Processing ./torch-1.10.2-cp38-abi3-linux_aarch64.whlCollecting typing-extensionsDownloading http://mirrors.aliyun.com/pypi/packages/31/25/5abcd82372d3d4a3932e1fa8c3dbf9efac10cc7c0d16e78467460571b404/typing_extensions-4.5.0-py3-none-any.whl (27 kB)Installing collected packages: typing-extensions, torchSuccessfully installed torch-1.10.2 typing-extensions-4.5.0

安装成功!撒花

结语

虽然是arm环境下pytorch安装方式,实际上可以适用于大部分python库的安装。

参考资料

[1]如何查看Linux系统版本?
[2]问题推进贴:arm64下安装pytorch,torchvision,torchaudio以及它们之间的版本对应关系
[3]问题终结贴:【2021新教程】解决is not a supported wheel on this platform-解决pip has no attribute pep425tags-解决网上旧教程不适用的问题

其余参考:
[4]Torch 、torchvision 、Python 版本对应关系以及安装 GPU 或 CPU 版本的 pytorch
[5]arm下安装pytorch
[6]在Arm/Linux下安装python的pytorch库