Python Modules & Environments

Using Conda

Conda is a package manager application that quickly installs, runs, and updates packages and their dependencies. It can query and search the package index and current installation, and install and update packages into existing conda environments. Conda is only aware of a subset of Python packages, and is not meant as a replacement to pip.

A virtual environment is a named, isolated, working copy of Python that that maintains its own files, directories, and paths so that you can work with specific versions of libraries or Python itself without affecting other Python projects. Conda is also an environment manager application. For example, you may have one environment with NumPy 1.7 and its dependencies, and another environment with NumPy 1.6 for legacy testing. If you change one environment, your other environments are not affected. You can easily activate or deactivate (switch between) these environments.

For example, to create an Python 3.6 environment with the Pandas version of 0.19 and its dependencies, open the Anaconda shell from the RCE applications menu, then issue these commands:

conda create --name mypandas019 python=3.6 pandas=0.19
source activate mypandas019

You can find full documentation for using Conda at https://conda.io/docs/using/envs.html.

We also have additional documentation for using Anaconda Python in the RCE.

If you install a module using conda, you can make it available to a limited set of conda environments if you so choose. If you install a module with pip, it automatically becomes available in all conda environments [for that version of Python].

Using pip

As an RCE user, you have the ability to install Python modules locally to your home directory and use them in your projects.

  1. Determine the required python version: You need to determine which version of Python you'd like to develop with. Currently, Python 2.7 is available via Anaconda 2, and Python 3.6 is available via Anaconda 3.
  2. Load the appropriate Anaconda environment: Open an Anaconda Shell via the RCE Powered Applications menu. Also see Working With Anaconda Python.
  3. Search for the Python module: Each version of Python installed on the RCE maintains its own module path. Determine whether a Python module is installed for a specific version of Python by using pip, to list packages installed for a desired version.
    pip list | grep $MODULE
    Example: pip list | grep simplejson
  4. Install your module: If your module is installed for the Python version you need, you're done. If the module is not installed install the module locally to your home directory.
    pip install $MODULE --user
    Example: pip install simplejson --user
  5. Can't find your module? If you're unable to locate your module using pip, maybe you're searching for the wrong module name. If you've decided you needed to install a module because, for example, import simplejson, did not work from a Python interactive console, you may have the wrong name. Often Python class names differ from Python module names. Try using the pip search feature.
  6. Still not found? Try searching the PyPI repository, the official Python module repository. Use google. Very rarely, some modules require that you manually compile Python packages using setup.py.
  7. Need help? Open a ticket by sending an email to support@help.hmdc.harvard.edu.