在将DAG部署到Composer气流环境时,我遇到了一个非常奇怪的错误。
我已经使用以下导入创建了一个自定义操作符:
from exchangelib import DELEGATE, Account, Credentials, Message, FileAttachment, HTMLBody
自定义操作符在名为airflow_mail_operator.py的文件中定义,DAG在名为test_main.py的文件中定义。
我得到的错误来自气流get服务器,并且状态是ModuleNotFoundError:没有名为'exchangelib'的模块,即使Composer环境已经显示了这个库。
即使在DAG文件中,如果我添加导入语句,错误也将停止显示,这对我来说真的很奇怪!
遵循完整的堆栈跟踪:
Traceback (most recent call last): File "/usr/local/lib/airflow/airflow/models/dagbag.py", line 243, in process_file m = imp.load_source(mod_name, filepath) File "/opt/python3.6/lib/python3.6/imp.py", line 172, in load_source module = _load(spec) File "<frozen importlib._bootstrap>", line 684, in _load File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/airflow/gcs/dags/dg_dags/test_main.py", line 5, in <module> from dg_dags.airflow_mail_operator import CustomEmailOperator File "/home/airflow/gcs/dags/dg_dags/airflow_mail_operator.py", line 5, in <module> from exchangelib import DELEGATE, Account, Credentials, Message, FileAttachment, HTMLBody ModuleNotFoundError: No module named 'exchangelib'
有没有人遇到过类似的错误(也许已经解决了)?
我的环境是:
composer-1.10.0-airflow-1.10.6
python3
提前感谢!
发布于 2022-04-06 08:23:27
对我来说,解决方案是通过更新底层的k8s集群来强制对Composer集群进行更新!
https://stackoverflow.com/questions/63300146
复制相似问题