如何在conda环境中使用Jupyter笔记本?

91 人关注

通常,人们在终端运行 jupyter notebook jupyter-notebook ipython notebook 来 启动本地的Jupyter笔记本网络服务器(并在浏览器中打开URL)。当使用 锥形瓶 锥形瓶 environments , what is the best way to run a Jupyter notebook which allows to import Python modules installed in the 锥形瓶 environment?

正如它看起来的那样。 this is not quite 前, 许多 users 类似的 troubles .

Most common error message seems to be: after installing a package XYZ in a 锥形瓶 environment my-env one can run import XYZ in a python console started in my-env , but running the same code in the Jupyter笔记本将导致ImportError .

This question has been asked 许多 times, but there is no good place to answer it, most Q&A's 和 Github tickets are quite messy so let's start a new Q&A here .

1 个评论
我已经被这个问题困扰了好几年了!谢谢你解决这个问题--正如你所说的--这个问题的git问题跟踪是个烂摊子。
python
jupyter-notebook
environment-variables
jupyter
conda
lumbric
lumbric
发布于 2019-09-24
3 个回答
lumbric
lumbric
发布于 2021-12-30
已采纳
0 人赞同

免责声明 :ATM只在Ubuntu和Windows中进行了测试(见该答案的评论)。

Jupyter在一个单独的进程中运行用户的代码,称为 kernel 内核可以是一个不同的Python安装(在不同的conda环境或virtualenv中,或者是Python 2而不是Python 3),甚至是一个不同语言的解释器(例如Julia或R)。内核的配置是通过指定解释器、名称和其他一些参数(见 Jupyter文件 ),配置可以存储在整个系统、活动环境(或virtualenv)或每个用户。如果使用 nb_conda_kernels ,除了静态配置的内核外,在Jupyter笔记本中还会为每个安装了 ipykernel 的conda环境提供一个单独的内核。

简而言之,有三种选择,如何使用conda环境和Jupyter。

Option 1: Run Jupyter server和kernel inside the conda environment

Do something like:

conda create -n my-conda-env         # creates new virtual env
conda activate my-conda-env          # activate environment in terminal
conda install jupyter                # install jupyter + notebook
jupyter notebook                     # start server + kernel inside my-conda-env

Jupyter将被完全安装在conda环境中。不同版本的Jupyter可以被用于 不同的conda环境,但这个选项可能有点矫枉过正。只需在环境中包括 包括环境中的内核就足够了,它是包裹着Python的组件,可以运行代码。 Jupyter笔记本的其他部分可以被看作是编辑器或浏览器,没有必要为每个环境单独安装并包括 没有必要为每个环境单独安装,并将其包含在每个env.yml文件中。因此,下面两个选项中的一个 中的一个可能更好,但这是最简单的一个,绝对没问题。

Option 2: Create special kernel for the conda environment

Do something like:

conda create -n my-conda-env                               # creates new virtual env
conda activate my-conda-env                                # activate environment in terminal
conda install ipykernel                                    # install Python kernel in new conda env
ipython kernel install --user --name=my-conda-env-kernel   # configure Jupyter to use Python kernel

然后从系统安装或不同的conda环境运行jupyter。

conda deactivate          # this step can be omitted by using a different terminal window than before
conda install jupyter     # optional, might be installed already in system e.g. by 'apt install jupyter' on debian-based systems
jupyter notebook          # run jupyter from system

内核的名字和conda环境的名字是相互独立的,但使用一个类似的名字可能会有意义。

只有Python内核会在conda环境中运行,来自系统的Jupyter或不同的conda环境会被使用--它不会被安装在conda环境中。通过调用ipython kernel install,jupyter被配置为使用conda环境作为内核,参见Jupyter文件IPython文档以了解更多信息。在大多数Linux安装中,这个配置是*.json中的~/.local/share/jupyter/kernels/my-conda-env-kernel/kernel.json文件。

"argv": [ "/opt/miniconda3/envs/my-conda-env/bin/python", "-m", "ipykernel_launcher", "-f", "{connection_file}" "display_name": "my-conda-env-kernel", "language": "python"

Option 3: Use nb_conda_kernels to use a kernel in the conda environment

When the package nb_conda_kernels安装后,一个单独的内核会自动提供给每个 包含conda包ipykernel或不同内核(R,Julia,...)的conda环境自动提供一个单独的内核。

conda activate my-conda-env    # this is the environment for your project and code
conda install ipykernel
conda deactivate
conda activate base            # could be also some other environment
conda install nb_conda_kernels
jupyter notebook

You should be able to choose the Kernel Python [conda env:my-conda-env]. Note that nb_conda_kernels seems to be available only via conda和not via pip or other package managers like apt.

Troubleshooting

使用Linux/Mac,命令行上的which会告诉你使用的是哪种jupyter。 使用选项1(从conda环境中运行Jupyter),它应该是一个可执行文件 的可执行文件。

$ which jupyter
/opt/miniconda3/envs/my-conda-env/bin/jupyter
$ which jupyter-notebook   # this might be different than 'which jupyter'! (see below)
/opt/miniconda3/envs/my-conda-env/bin/jupyter-notebook

在笔记本里面,你应该看到Python使用来自conda环境的Python路径。

[1] !which python
/opt/miniconda3/envs/my-conda-env/bin/python
[2] import sys; sys.executable
'/opt/miniconda3/envs/my-conda-env/bin/python'
['/home/my_user',
 '/opt/miniconda3/envs/my-conda-env/lib/python37.zip',
 '/opt/miniconda3/envs/my-conda-env/lib/python3.7',
 '/opt/miniconda3/envs/my-conda-env/lib/python3.7/lib-dynload',
 '/opt/miniconda3/envs/my-conda-env/lib/python3.7/site-packages',
 '/opt/miniconda3/envs/my-conda-env/lib/python3.7/site-packages/IPython/extensions',
 '/home/my_user/.ipython']

Jupyter提供的命令jupyter-troubleshoot或在Jupyter笔记本中。

!jupyter-troubleshoot

This will print a lot of helpful information about including the outputs mentioned above as well as installed libraries和others. When asking for help regarding Jupyter installations questions, it might be good idea to provide this information in bug reports or questions.

要列出所有配置的Jupyter内核,请运行。

jupyter kernelspec list

Common errors和traps

Jupyter notebook not installed in conda environment

注意:症状不是这里描述的问题所特有的。

Symptoms:在Jupyter笔记本中,对于安装在conda环境中的模块(但未在系统中安装),会出现导入错误。 但在Python终端导入时却没有错误。

解释。 You tried to run jupyter notebook from inside your conda environment (option 1, see above), there is no configuration for a kernel for this conda environment (this would be option 2)和nb_conda_kernels is not installed (option 3), but jupyter notebook is not (fully) installed in the conda environment, even if which jupyter might make you believe it was.

在GNU/Linux中,你可以输入which jupyter来检查Jupyter的哪个可执行文件被运行。

这意味着使用了系统的Jupyter,可能是因为没有安装Jupyter。

(my-conda-env) $ which jupyter-notebook
/usr/bin/jupyter

如果路径指向你conda环境中的一个文件,Jupyter将从Jupyter内部运行。

(my-conda-env) $ which jupyter-notebook
/opt/miniconda3/envs/my-conda-env/bin/jupyter-notebook

请注意,在安装conda软件包ipykernel时,会提供一个可执行文件jupyter,但没有可执行文件jupyter-notebook。 没有可执行文件jupyter-notebook。这意味着,which jupyter将返回一个通往conda 环境的路径,但jupyter notebook将启动系统的jupyter-nootebook(参见here):

 $ conda create -n my-conda-env
 $ conda activate my-conda-env
 $ conda install ipykernel
 $ which jupyter            # this looks good, but is misleading!
 /opt/miniconda3/envs/my-conda-env/bin/jupyter
 $ which jupyter-notebook   # jupyter simply runs jupyter-notebook from system...
 /usr/bin/jupyter-notebook

发生这种情况是因为jupyter notebook搜索了jupyter-notebook,找到了 /usr/bin/jupyter-notebook和 开始一个新的Python进程。替换代码34】中的shebang是#!/usr/bin/python3。 和不是一个动态的 #!/usr/bin/env python. Therefore Python manages to break out of the conda environment. I guess jupyter could call python /usr/bin/jupyter-notebook instead to overrule the shebang, but mixing system's bin files和the environment's python path can't work well anyway.

Solution:在conda环境中安装jupyter笔记本。

 conda activate my-conda-env
 conda install jupyter
 jupyter notebook

Wrong kernel configuration: Kernel is configured to use system Python

注意:症状不是这里描述的问题所特有的。

Symptoms:在Jupyter笔记本中,对于安装在conda环境中的模块(但未在系统中安装),会出现导入错误。 但在Python终端导入时却没有错误。

解释一下。通常情况下,系统提供了一个名为python3的内核(显示名称为 "Python 3")。 配置为使用/usr/bin/python3,见例如/usr/share/jupyter/kernels/python3/kernel.json。 这通常被conda环境中的内核所覆盖,它指向的环境是 python二进制/opt/miniconda3/envs/my-conda-env/bin/python。两者都是由软件包生成的 ipykernel(见herehere).

A user kernel specification in ~/.local/share/jupyter/kernels/python3/kernel.json might override the system-wide和environment kernel. If the environment kernel is missing or the user kernel points to a python installation outside the environment option 1 (installation of jupyter in the environment) will fail.

For occurrences和discussions of this problem和variants see here, here, 和also here, herehere.

Solution:使用jupyter kernelspec list来列出活动内核的位置。

$ conda activate my-conda-env
$ jupyter kernelspec list
Available kernels:
  python3 /opt/miniconda3/envs/my-conda-env/share/jupyter/kernels/python3

If the kernel in the environment is missing, you can try creating it manually using ipython kernel install --sys-prefix in the activated environment, but it is probably better to check your installation, because conda install ipykernel should have created the environment (maybe try re-crate the environment和re-install all packages?).

如果一个用户内核规范阻碍了环境内核规范,你可以 删除它,或者使用相对的 python 路径,它将使用 $PATH 来确定使用哪个 python。 所以像这样的东西,应该是完全没问题的。

$ cat ~/.local/share/jupyter/kernels/python3/kernel.json
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 "display_name": "Python 3",
 "language": "python"

Correct conda environment not activated

Symptoms: ImportError for modules installed in the conda environment (but not installed system wide) in Jupyter notebooks和Python terminals

解释一下。每个终端都有一组环境变量,当终端关闭时,这些变量就会丢失。 关闭时,这些环境变量就会丢失。为了使用conda环境,需要设置某些环境变量。 这是通过使用conda activate my-conda-env激活它来完成的。如果你试图在conda环境中运行Jupyter 笔记本(选项1),但在运行前没有激活conda环境 但在运行前没有激活conda环境,它可能会运行系统中的jupyter。

Solution:在运行Jupyter之前激活conda环境。

 conda activate my-conda-env
 jupyter notebook

Broken kernel configuration

Symptoms:奇怪的事情发生了。也许与上述症状相似,例如ImportError

解释一下。 If you attempted to use option 2, i.e. running Jupyter from system和the Jupyter kernel inside the conda environment by using an explicit configuration for the kernel, but it does not behave as you expect, the configuration might be corrupted in 某一方式.

Solution: Check configuration in ~/.local/share/jupyter/kernels/my-kernel-name/kernel.json 和fix mistakes manually or remove the entire directory和re-create it using the command provided above for option 2. If you can't find the kernel configuration there run jupyter kernelspec list.

Python 2 vs 3

Symptoms: ImportError due to wrong Python version of the Jupyter kernel or other problems with Python 2/3

解释一下。 The kernel configuration can have all sorts of confusing和misleading effects. For example the default Python 3 kernel configuration will allow me to launch a Jupyter notebook running on Python 2:

conda create -n my-conda-env
conda activate my-conda-env
conda install python=2
conda install jupyter
jupyter notebook

默认的Python 3内核。

$ cat ~/.local/share/jupyter/kernels/python3/kernel.json
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 "display_name": "Python 3",
 "language": "python"

在用Python 3内核创建一个新的Jupyter笔记本后,即使Jupyter显示为 "Python 3",来自conda环境的Python 2也会被使用。 环境将被使用,即使 Jupyter 显示的是 "Python 3"。

Solution: Don't use Python 2 ;-)

merv
选项3有一个错误。 nb_conda 之所以能工作,是因为它将 nb_conda_kernels 作为一个依赖项来安装。替换代码1】是一个环境中的Jupyter实例自动识别任何安装了 ipykernel 的环境的唯一要求( see docs ). nb_conda 包是一个Jupyter扩展,它为Jupyter添加了一个 "Conda "标签,提供了一个类似Anaconda Navigator的GUI来管理环境。
@merv 呃,是的!非常感谢你指出这一点,我已经修正了这个错误。
The code in Option 1: conda install juypter should be conda install jupyter .
我确认这在windows上也能正常工作。你似乎失去了 !pip install xxx 命令,但这是一个很好的交易。
选项1在Manjaro上有效--谢谢;不错的简洁解决方案
Minura Punchihewa
Minura Punchihewa
发布于 2021-12-30
0 人赞同

下面的命令也可以作为一个单行线来创建你的Conda环境,运行最新版本的Python和最新版本的Jupyter Notebooks。

conda create -n <env-name> python jupyter

如果你想安装特定版本的Python或Jupyter,你可以做到。

conda create -n <env-name> python=<version> jupyter=<version>
conda create -n <env-name> python=3.10.4 jupyter=1.0.0

如果有其他软件包,你想在这个环境中使用你的笔记本,你可以做以下工作。

conda create -n <env-name> python jupyter <another-package> <another-package> ...
conda create -n <env-name> python jupyter scikit-learn

注意,与之前类似,这些命令将安装最新版本的 Python 和相关软件包。如果你想要特定的版本,你可以使用=<version>的语法。

此外,一旦环境被创建,你仍然可以使用pip installconda install来安装你需要的任何软件包。

在你创建了你的环境后(使用上面给出的任何方法),你可以简单地运行以下命令来激活你的环境并运行Jupyter笔记本。

conda activate <env-name>
jupyter notebook 
    
这基本上是我答案中的选项1,但使用了 conda install 的快捷方式。可以这样做,如果你想少输入一个命令。我不建议使用pip,如果软件包可以通过conda获得。
Siddharth Shakya
Siddharth Shakya
发布于 2021-12-30
0 人赞同

Following worked for me :

  • Activate the environment that you want to use : conda activate <env_name>

  •