我正在尝试使用Gitlab将我的代码(使用fb先知)部署为一个EC2实例。
我的 requirements.txt 文件:
requirements.txt
-i https://pypi.org/simple cmdstanpy==0.4 convertdate==2.2.1 cycler==0.10.0 cython==0.29.21; python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3' ephem==3.7.7.1 fbprophet==0.6 holidays==0.10.3 joblib==0.16.0; python_version >= '3.6' kiwisolver==1.2.0; python_version >= '3.6' korean-lunar-calendar==0.2.1 lunarcalendar==0.0.9; python_version >= '2.7' and python_version < '4' matplotlib==3.3.0; python_version >= '3.6' numpy==1.19.1 pandas==1.0.5 pillow==7.2.0; python_version >= '3.5' psycopg2-binary==2.8.5 pymeeus==0.3.7 pyparsing==2.4.7; python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3' pystan==2.19.1.1 python-dateutil==2.8.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' pytz==2020.1 scikit-learn==0.23.1 scipy==1.5.2; python_version >= '3.6' setuptools-git==1.2 six==1.15.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' sqlalchemy==1.3.18 threadpoolctl==2.1.0; python_version >= '3.5'
当 .gitlab-ci.yml 文件是:
.gitlab-ci.yml
image: ubuntu:18.04 variables: AWS_DEFAULT_REGION: eu-central-1 GIT_SUBMODULE_STRATEGY: recursive S3_TEST_BUCKET: $BUCKET_UNPACK stages: - deploy packageAndDeploy: stage: deploy script: - apt-get -y update - apt-get -y install gcc g++ build-essential - apt-get -y install python3-pip python3.7 zip python3.7-dev - python3.7 -m pip install --upgrade pip - python3.7 -V - pip3.7 install -r requirements.txt
它成功地运行
$ pip3.7 install -r requirements.txt Collecting cmdstanpy==0.4 Downloading cmdstanpy-0.4.0-py3-none-any.whl (22 kB) Collecting convertdate==2.2.1 Downloading convertdate-2.2.1-py2.py3-none-any.whl (43 kB) Collecting cycler==0.10.0 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Collecting cython==0.29.21 Downloading Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB) Collecting ephem==3.7.7.1 Downloading ephem-3.7.7.1-cp37-cp37m-manylinux2010_x86_64.whl (1.2 MB) Collecting fbprophet==0.6 Downloading fbprophet-0.6.tar.gz (54 kB) Collecting holidays==0.10.3 Downloading holidays-0.10.3.tar.gz (114 kB) Collecting joblib==0.16.0 Downloading joblib-0.16.0-py3-none-any.whl (300 kB) Collecting kiwisolver==1.2.0 Downloading kiwisolver-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (88 kB) Collecting korean-lunar-calendar==0.2.1 Downloading korean_lunar_calendar-0.2.1-py3-none-any.whl (8.0 kB) Collecting lunarcalendar==0.0.9 Downloading LunarCalendar-0.0.9-py2.py3-none-any.whl (18 kB) Collecting matplotlib==3.3.0 Downloading matplotlib-3.3.0-1-cp37-cp37m-manylinux1_x86_64.whl (11.5 MB) Collecting numpy==1.19.1 Downloading numpy-1.19.1-cp37-cp37m-manylinux2010_x86_64.whl (14.5 MB) Collecting pandas==1.0.5 Downloading pandas-1.0.5-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB) Collecting pillow==7.2.0 Downloading Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB) Collecting psycopg2-binary==2.8.5 Downloading psycopg2_binary-2.8.5-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB) Collecting pymeeus==0.3.7 Downloading PyMeeus-0.3.7.tar.gz (732 kB) Collecting pyparsing==2.4.7 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting pystan==2.19.1.1 Downloading pystan-2.19.1.1-cp37-cp37m-manylinux1_x86_64.whl (67.3 MB) Collecting python-dateutil==2.8.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) Collecting pytz==2020.1 Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB) Collecting scikit-learn==0.23.1 Downloading scikit_learn-0.23.1-cp37-cp37m-manylinux1_x86_64.whl (6.8 MB) Collecting scipy==1.5.2 Downloading scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB) Collecting setuptools-git==1.2 Downloading setuptools_git-1.2-py2.py3-none-any.whl (10 kB) Collecting six==1.15.0 Downloading six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting sqlalchemy==1.3.18 Downloading SQLAlchemy-1.3.18-cp37-cp37m-manylinux2010_x86_64.whl (1.3 MB) Collecting threadpoolctl==2.1.0 Downloading threadpoolctl-2.1.0-py3-none-any.whl (12 kB) Building wheels for collected packages: fbprophet, holidays, pymeeus Building wheel for fbprophet (setup.py): started Building wheel for fbprophet (setup.py): finished with status 'error' ERROR: Command errored out with exit status 1: command: /usr/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-upo5plzy/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-upo5plzy/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-c5ch2465 cwd: /tmp/pip-install-upo5plzy/fbprophet/ Complete output (40 lines): running bdist_wheel running build running build_py creating build creating build/lib creating build/lib/fbprophet creating build/lib/fbprophet/stan_model Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-upo5plzy/fbprophet/setup.py", line 148, in <module> File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 129, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 204, in run self.run_command('build') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-install-upo5plzy/fbprophet/setup.py", line 48, in run build_models(target_dir) File "/tmp/pip-install-upo5plzy/fbprophet/setup.py", line 36, in build_models from fbprophet.models import StanBackendEnum File "/tmp/pip-install-upo5plzy/fbprophet/fbprophet/__init__.py", line 8, in <module> from fbprophet.forecaster import Prophet File "/tmp/pip-install-upo5plzy/fbprophet/fbprophet/forecaster.py", line 14, in <module> import numpy as np ModuleNotFoundError: No module named 'numpy' ---------------------------------------- ERROR: Failed building wheel for fbprophet Running setup.py clean for fbprophet Building wheel for holidays (setup.py): started Building wheel for holidays (setup.py): finished with status 'done' Created wheel for holidays: filename=holidays-0.10.3-py3-none-any.whl size=122264 sha256=d9a0eb199d8421b97d7a17d173acfa998d9feeac4da85ae5eac7452e2a52e54e Stored in directory: /root/.cache/pip/wheels/d0/67/01/18308fa32fabcd0af569b9f52934b2118c9b30bcf7b4b378f8 Building wheel for pymeeus (setup.py): started Building wheel for pymeeus (setup.py): finished with status 'done' Created wheel for pymeeus: filename=PyMeeus-0.3.7-py3-none-any.whl size=691116 sha256=ca5c9777c09575d57de0d69f0f3a2e1328c84d67bd05a1c1f11ad308811ea5be Stored in directory: /root/.cache/pip/wheels/80/32/5f/2a67880d4ce584b9cf99146f9945e46942dfb010a9382c6ff5 Successfully built holidays pymeeus Failed to build fbprophet Installing collected packages: numpy, pytz, six, python-dateutil, pandas, cmdstanpy, pymeeus, convertdate, cycler, cython, ephem, lunarcalendar, korean-lunar-calendar, holidays, pyparsing, pillow, kiwisolver, matplotlib, pystan, setuptools-git, fbprophet, joblib, psycopg2-binary, threadpoolctl, scipy, scikit-learn, sqlalchemy Attempting uninstall: six Found existing installation: six 1.11.0 Uninstalling six-1.11.0: Successfully uninstalled six-1.11.0 Running setup.py install for fbprophet: started Running setup.py install for fbprophet: still running... Running setup.py install for fbprophet: finished with status 'done' Successfully installed cmdstanpy-0.4.0 convertdate-2.2.1 cycler-0.10.0 cython-0.29.21 ephem-3.7.7.1 fbprophet-0.6 holidays-0.10.3 joblib-0.16.0 kiwisolver-1.2.0 korean-lunar-calendar-0.2.1 lunarcalendar-0.0.9 matplotlib-3.3.0 numpy-1.19.1 pandas-1.0.5 pillow-7.2.0 psycopg2-binary-2.8.5 pymeeus-0.3.7 pyparsing-2.4.7 pystan-2.19.1.1 python-dateutil-2.8.1 pytz-2020.1 scikit-learn-0.23.1 scipy-1.5.2 setuptools-git-1.2 six-1.15.0 sqlalchemy-1.3.18 threadpoolctl-2.1.0 Job succeeded
但是,当我将CI文件更改为(用于在AWS上部署)时:
image: ubuntu:18.04 variables: AWS_DEFAULT_REGION: eu-central-1 GIT_SUBMODULE_STRATEGY: recursive S3_TEST_BUCKET: $BUCKET_UNPACK stages: - deploy packageAndDeploy: stage: deploy script: - apt-get -y update - apt-get -y install gcc g++ build-essential - apt-get -y install python3-pip python3.7 zip python3.7-dev - python3.7 -m pip install --upgrade pip - python3.7 -V - pip3.7 install virtualenv - mv iso_forest_ad.py ~ - mv requirements.txt ~ # Setup virtual environment - mkdir ~/forEC2 #- mv package ~/forEC2 - cd ~/forEC2 - virtualenv -p python3.7 venv - source venv/bin/activate - pip3.7 install -r ~/requirements.txt -t ~/forEC2/venv/lib/python3.7/site-packages/ # Package environment and dependencies - cd ~/forEC2/venv/lib/python3.7/site-packages/ - zip -r9 ~/forEC2/archive.zip . - cd ~ - zip -g ~/forEC2/archive.zip iso_forest_ad.py # Install AWS CLI - pip install awscli --upgrade - export PATH=$PATH:~/.local/bin # Configure AWS connection - aws configure set aws_access_key_id $AWS_TEST_ACCESS_KEY_ID - aws configure set aws_secret_access_key $AWS_TEST_SECRET_ACCESS_KEY - aws configure set default.region $AWS_DEFAULT_REGION - aws sts get-caller-identity --output text --query 'Account' - aws s3 cp ~/forEC2/archive.zip $BUCKET_UNPACK/xyz.zip
它引发以下错误:
$ pip3.7 install -r ~/requirements.txt -t ~/forEC2/venv/lib/python3.7/site-packages/ Collecting cmdstanpy==0.4 Downloading cmdstanpy-0.4.0-py3-none-any.whl (22 kB) Collecting convertdate==2.2.1 Downloading convertdate-2.2.1-py2.py3-none-any.whl (43 kB) Collecting cycler==0.10.0 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Collecting cython==0.29.21 Downloading Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB) Collecting ephem==3.7.7.1 Downloading ephem-3.7.7.1-cp37-cp37m-manylinux2010_x86_64.whl (1.2 MB) Collecting fbprophet==0.6 Downloading fbprophet-0.6.tar.gz (54 kB) Collecting holidays==0.10.3 Downloading holidays-0.10.3.tar.gz (114 kB) Collecting joblib==0.16.0 Downloading joblib-0.16.0-py3-none-any.whl (300 kB) Collecting kiwisolver==1.2.0 Downloading kiwisolver-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (88 kB) Collecting korean-lunar-calendar==0.2.1 Downloading korean_lunar_calendar-0.2.1-py3-none-any.whl (8.0 kB) Collecting lunarcalendar==0.0.9 Downloading LunarCalendar-0.0.9-py2.py3-none-any.whl (18 kB) Collecting matplotlib==3.3.0 Downloading matplotlib-3.3.0-1-cp37-cp37m-manylinux1_x86_64.whl (11.5 MB) Collecting numpy==1.19.1 Downloading numpy-1.19.1-cp37-cp37m-manylinux2010_x86_64.whl (14.5 MB) Collecting pandas==1.0.5 Downloading pandas-1.0.5-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB) Collecting pillow==7.2.0 Downloading Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB) Collecting psycopg2-binary==2.8.5 Downloading psycopg2_binary-2.8.5-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB) Collecting pymeeus==0.3.7 Downloading PyMeeus-0.3.7.tar.gz (732 kB) Collecting pyparsing==2.4.7 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting pystan==2.19.1.1 Downloading pystan-2.19.1.1-cp37-cp37m-manylinux1_x86_64.whl (67.3 MB) Collecting python-dateutil==2.8.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) Collecting pytz==2020.1 Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB) Collecting scikit-learn==0.23.1 Downloading scikit_learn-0.23.1-cp37-cp37m-manylinux1_x86_64.whl (6.8 MB) Collecting scipy==1.5.2 Downloading scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB) Collecting setuptools-git==1.2 Downloading setuptools_git-1.2-py2.py3-none-any.whl (10 kB) Collecting six==1.15.0 Downloading six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting sqlalchemy==1.3.18 Downloading SQLAlchemy-1.3.18-cp37-cp37m-manylinux2010_x86_64.whl (1.3 MB) Collecting threadpoolctl==2.1.0 Downloading threadpoolctl-2.1.0-py3-none-any.whl (12 kB) Building wheels for collected packages: fbprophet, holidays, pymeeus Building wheel for fbprophet (setup.py): started Building wheel for fbprophet (setup.py): finished with status 'error' ERROR: Command errored out with exit status 1: command: /root/forEC2/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-eq1vplr2/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-eq1vplr2/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-mfybmls1 cwd: /tmp/pip-install-eq1vplr2/fbprophet/ Complete output (40 lines): running bdist_wheel running build running build_py creating build creating build/lib creating build/lib/fbprophet creating build/lib/fbprophet/stan_model Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 148, in <module> File "/root/forEC2/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 165, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/root/forEC2/venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 223, in run self.run_command('build') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 48, in run build_models(target_dir) File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 36, in build_models from fbprophet.models import StanBackendEnum File "/tmp/pip-install-eq1vplr2/fbprophet/fbprophet/__init__.py", line 8, in <module> from fbprophet.forecaster import Prophet File "/tmp/pip-install-eq1vplr2/fbprophet/fbprophet/forecaster.py", line 14, in <module> import numpy as np ModuleNotFoundError: No module named 'numpy' ---------------------------------------- ERROR: Failed building wheel for fbprophet Running setup.py clean for fbprophet Building wheel for holidays (setup.py): started Building wheel for holidays (setup.py): finished with status 'done' Created wheel for holidays: filename=holidays-0.10.3-py3-none-any.whl size=115855 sha256=7a285975b4ee67336ed9f72ba86938bc50e0525828c6c4a0ee23f89d2a4138ca Stored in directory: /root/.cache/pip/wheels/d0/67/01/18308fa32fabcd0af569b9f52934b2118c9b30bcf7b4b378f8 Building wheel for pymeeus (setup.py): started Building wheel for pymeeus (setup.py): finished with status 'done' Created wheel for pymeeus: filename=PyMeeus-0.3.7-py3-none-any.whl size=702876 sha256=cc24a563f61d172dca5e377cff86172e162429e1330621cb6534ca1f7d964bb9 Stored in directory: /root/.cache/pip/wheels/80/32/5f/2a67880d4ce584b9cf99146f9945e46942dfb010a9382c6ff5 Successfully built holidays pymeeus Failed to build fbprophet Installing collected packages: numpy, pytz, six, python-dateutil, pandas, cmdstanpy, pymeeus, convertdate, cycler, cython, ephem, pystan, pillow, pyparsing, kiwisolver, matplotlib, lunarcalendar, korean-lunar-calendar, holidays, setuptools-git, fbprophet, joblib, psycopg2-binary, scipy, threadpoolctl, scikit-learn, sqlalchemy Running setup.py install for fbprophet: started Running setup.py install for fbprophet: finished with status 'error' ERROR: Command errored out with exit status 1: command: /root/forEC2/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-eq1vplr2/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-eq1vplr2/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-oymqs6p3/install-record.txt --single-version-externally-managed --home /tmp/pip-target-2bx5n0o4 --compile --install-headers /root/forEC2/venv/include/site/python3.7/fbprophet cwd: /tmp/pip-install-eq1vplr2/fbprophet/ Complete output (42 lines): running install running build running build_py creating build creating build/lib creating build/lib/fbprophet creating build/lib/fbprophet/stan_model Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 148, in <module> File "/root/forEC2/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 165, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/root/forEC2/venv/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run return orig.install.run(self) File "/usr/lib/python3.7/distutils/command/install.py", line 589, in run self.run_command('build') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 48, in run build_models(target_dir) File "/tmp/pip-install-eq1vplr2/fbprophet/setup.py", line 36, in build_models from fbprophet.models import StanBackendEnum File "/tmp/pip-install-eq1vplr2/fbprophet/fbprophet/__init__.py", line 8, in <module> from fbprophet.forecaster import Prophet File "/tmp/pip-install-eq1vplr2/fbprophet/fbprophet/forecaster.py", line 14, in <module>