text-generation-webui加载codellama报错DLL load failed while importing flash_attn_2_cuda: 找不到指定的模块。

使用text-generation-webui加载codellama，报错：

Traceback (most recent call last):File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\utils\import_utils.py", line 1353, in _get_modulereturn importlib.import_module("." + module_name, self.__name__)File "D:\Anaconda\Anaconda\envs\codellama\lib\importlib_init_.py", line 126, in import_modulereturn _bootstrap._gcd_import(name[level:], package, level)File "", line 1050, in _gcd_importFile "", line 1027, in _find_and_loadFile "", line 1006, in _find_and_load_unlockedFile "", line 688, in _load_unlockedFile "", line 883, in exec_moduleFile "", line 241, in _call_with_frames_removedFile "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\llama\modeling_llama.py", line 48, infrom flash_attn import flash_attn_func, flash_attn_varlen_funcFile "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\flash_attn_init_.py", line 3, infrom flash_attn.flash_attn_interface import (File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\flash_attn\flash_attn_interface.py", line 8, inimport flash_attn_2_cuda as flash_attn_cudaImportError: DLL load failed while importing flash_attn_2_cuda: 找不到指定的模块。The above exception was the direct cause of the following exception:Traceback (most recent call last):File "E:\模型\text-generation-webui\text-generation-webui\modules\ui_model_menu.py", line 209, in load_model_wrappershared.model, shared.tokenizer = load_model(shared.model_name, loader)File "E:\模型\text-generation-webui\text-generation-webui\modules\models.py", line 85, in load_modeloutput = load_func_map[loader](model_name)File "E:\模型\text-generation-webui\text-generation-webui\modules\models.py", line 155, in huggingface_loadermodel = LoaderClass.from_pretrained(path_to_model, **params)File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\auto\auto_factory.py", line 565, in from_pretrainedmodel_class = _get_model_class(config, cls._model_mapping)File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\auto\auto_factory.py", line 387, in _get_model_classsupported_models = model_mapping[type(config)]File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\auto\auto_factory.py", line 740, in getitemreturn self._load_attr_from_module(model_type, model_name)File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\auto\auto_factory.py", line 754, in _load_attr_from_modulereturn getattribute_from_module(self._modules[module_name], attr)File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\models\auto\auto_factory.py", line 698, in getattribute_from_moduleif hasattr(module, attr):File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\utils\import_utils.py", line 1343, in getattrmodule = self._get_module(self._class_to_module[name])File "C:\Users\Ma\AppData\Roaming\Python\Python310\site-packages\transformers\utils\import_utils.py", line 1355, in _get_moduleraise RuntimeError(RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):DLL load failed while importing flash_attn_2_cuda: 找不到指定的模块。

一开始排查是以为transformers的版本不对，先确定了transformers的版本，transformers的版本应该大于4.35.0

把transformers升级为4.35.0后仍然报错

接着排查cuda和torch的版本

最后发现是cuda版本与torch版本不匹配

>>> print(torch.version.cuda) # 检查CUDA版本>>> 11.8

控制台运行nvcc –version：

输出：

nvcc: NVIDIA (R) Cuda compiler driverCopyright (c) 2005-2023 NVIDIA CorporationBuilt on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023Cuda compilation tools, release 12.1, V12.1.66Build cuda_12.1.r12.1/compiler.32415258_0

最后解决：

先卸载原本的torch：

pip uninstall torch torchvision torchaudio

然后安装12.1的：

pip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu121/torch_stable.html

最后加载成功codellama