Table of Contents

Introduction

When it comes to working with Python, especially in the domain of application development, there are certain tools that you will see being mentioned often in various places or open source code. Despite being extremely commonly used, unfortunately sometimes it is hard to get a hold of a good manual to walk you through each step, which is absolutely vital when it comes to getting familiar with such important and needed tools.

In this the cloud provider article, we aim to fill you in on not only the basics, but also the logic behind popular Python tools and items as we dive into using them in real life scenarios. We will begin with downloading and installing some common libraries, setting and working with virtual environments (using virtualenv), and managing packages for development and production of your own applications.

> This article is aimed at beginners as well as those seeking to obtain more in-depth knowledge. If you would like to see and learn more, please feel free to make a suggestion in the comments section below.

Python on CentOS

Please remember that if you are using a CentOS/RHEL system, you should abstain from working with the default Python interpreter that is shipped with the operating system. Instead, you should opt for installing Python yourself.

If you have not done this already and would like to learn how, please see our tutorial: How To Set Up Python 2.7.6 and 3.3.3 on CentOS 6.4 before we begin.

Likewise, in order to install pip and virtualenv on CentOS with a custom Python installation, you can follow the instructions on that article.

Python and Packages

pip illustration for: Python and Packages

Although Python applications can be made of a single file, usually they consist of a series of functions, objects (classes), handy tools and of course, variables spread across multiple file(s), placed inside *modules*. These modules together make up what is referred as a _package_.

The traditional way of installing a package involves first spotting it and then downloading. It sounds soft and simple because it actually *is* like many things in Python – *but it is not perfect*.

When the files are ready and unpacked, using the _distutils_ module, you can install it by calling setup.py:

> Installation example of a package:

cd a_package

python setup.py install

> distutils (distribution utilities) is a toolset used for the packaging and distributing of code. It is shipped by default with Python (i.e. included in the *standard library*).

In spite of the simplicity of the procedure explained above, it is no use if the challenge abstracted from installing exists elsewhere in the process: finding and managing them. This is where package management via tools comes in – bringing along several benefits such as:

  • Uninstalling (e.g. pip uninstall package_x),
  • Versioning (e.g. pip install django==x),
  • And automatic dependency management (as packages can depend on others).

Package Management

—

Packages in Python can be tools, libraries, frameworks and applications. Given the popularity and the beauty of the language, there are tens of thousands of packages available which you can make use of for *your own projects*.

Package Management Tools

—

The two most common Python package managers are _pip_ and _easy_install_. Both of them aim to help users with the tasks of:

  • Downloading, Installing and Uninstalling
  • Building
  • Managing Python packages and much more

Both of them might appear to do the same thing from the outside and their joint dependence on the common library _setuptools_ increases this notion.

However, in this case, it is what's hidden from the eye that makes the difference — and a lot of it as well.

pip vs easy_install

—

The first tool created for the task was easy_install. Although it was a relief and a pleasure to use compared to doing everything manually, over time it has proven to be problematic in certain aspects. That created the grounds for development of pip, another package manager.

pip (as defined by the project itself) is a replacement for easy_install, which brings many benefits over its predecessor, including, but not limited to:

  • Downloading everything before installing
  • Providing useful feedback during the process
  • Keeping history of actions being taken
  • Providing useful error messages following Python tradition
  • Complements _virtualenv_ and works with it *very* nicely

> To learn more about *pip*, consider reading its introduction located at PyPI package index by clicking pypi.python.org.

A Thorough *pip* How-To

—

In this section, we will talk about getting the necessary dependencies for *pip*, installing its latest built followed by a walk-through of the core functionality offered such as installing, uninstalling, freezing and managing requirements.

**When would I use *pip*?**

As promised at the introduction, we aim to give you examples of real life scenarios.

Imagine that you are undertaking the development of a small application. You have set yourself a roadmap, and everything is going well. Then you discover a library (or a module) that can be of great help to you if you included in your application. You can download it the traditional way as we have explained. However, once you have not just one but 3, 4 or even 20 packages you need to deal with, this process becomes cumbersome. Include managing them (e.g. updating, uninstalling, replacing, using a different version), you can see the problems you will need to deal with, which are made redundant using *pip*, the package manager.

Installing *pip*

—

In order to install *pip*, we first need to take care of its dependencies. Do not worry though, it is very easy.

setuptools

As explained above, one of the dependencies of *pip* is the _setuptools_ library. It builds on the (standard) functionality of Python's *distribution utilities* toolset called _distutils_. Given that distils is provided by default, all we need left is *setuptools*.

We are going to securely download the setup files for setuptools using curl. cURL is a system library which allows data transfer over various protocols (i.e. a common language for data exchange between applications, such as HTTP). It will verify the *SSL* certificates from the source and pass the data to the *Python interpreter*.

These setup files, which Python interpreter is going to execute, automate the installation process as they set up the latest stable version on our system.

> Execute the following command:

curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | python –

This installation gives us the ability to use pip globally across the system. However, this is not the preferred way to install any other package. What is recommended is to always use self-contained Python environments, virtualenv. We will talk about it in the next section.

Note: You might need to explicitly gain *super user* privileges in order to continue with the download. In that case, consider using:

curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | sudo python –

After having its single dependency installed, we can now continue with downloading and setting up *pip*.

We will be again using curl to have the setup file securely downloaded and installed.

> Execute the following command:

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python –

> **Default installation folder for *pip* is located at:**

/usr/local/bin

In order to use it without stating the full path, it must be appended to PATH.

> **Updating *PATH*:**

export PATH="/usr/local/bin:$PATH"

After completing this step, we are ready to work with *pip*.

Using *pip*

—

Using *pip* is really fun and can be considered headache free. If you have dealt with extremely unnecessary problems in the past and did not even understand why, *pip* will ensure that they are kept to a minimum for you hereon.

Installing packages using *pip*

—

pip can do many things but it would not be a mistake to state that the most often used function of it is installing packages. There are several ways it can handle this job for you.

> Finding and installing packages:

pip install requests

> Finding and installing a specific version:

pip install requests==2.0.0

> Installing from a URL or a VCS repository:

pip install https://github.com/pypa/virtualenv/tarball/1.9.X

> **Installing inside a *virtualenv*:**

pip install -E venv requests

Uninstalling packages with *pip*:

—

The second most common function of *pip* is probably uninstalling packages.

> Uninstalling a package:

pip uninstall requests

Upgrading packages with *pip*

—

If you are thinking of uninstalling to install a newer version of an application, you can try *upgrading*.

> Upgrading a package:

pip install –upgrade requests

Searching for packages with *pip*

—

Before deciding to remove or upgrade a package, you might feel the need to first *search* for one.

> Searching for a package:

pip search django

Creating a list of installed packages with *pip*

—

One of the most truly exceptional and life saving abilities of pip is being able to create – with ease – lists ("freeze") of packages installed. This is also often called requirements. Depending on your Python environment (whether it be a virtual or a global one), *pip* will create a file listing all the packages installed with one single command.

> Creating a fresh list ("freeze"):

Note: This command will output a file in the current working directory.

pip freeze > package_list.txt

> Creating a list ("freeze") on top of a template:

Note: This command will output a file in the current working directory.

pip freeze package_list.txt > package_list_new.txt

Installing all the packages from a list with *pip

—

When you are working on an application – preferably inside a virtual environment – you will have all of its dependencies (_required packages_) installed there. After having extracted a list of them using freeze, you can get them installed again with install.

> Installing all packages from a list ("freeze"):

pip install -r package_list_new.txt

A Thorough *virtualenv* How-To

—

Let's begin with defining what exactly virtualenv is and the situation where it comes in handy.

virtualenv:

In the world of Python, an _environment_ is a folder (directory) which contains everything that a Python project (application) needs in order to run in an organised, isolated fashion. When it is initiated, it automatically comes with its own Python interpreter – a copy of the one used to create it – alongside its very own *pip*.

There are a number of problems that virtualenv solves:

  • Creating a fresh, isolated environment for a Python project
  • Being able to download packages without requiring admin/sudo privileges
  • Easily packaging an application
  • Creating a list of dependencies which belongs to a single project created with *pip*
  • Easily recovering the dependencies using a requirements file created with *pip*
  • Giving way to portability across systems

Using *virtualenv* is the recommended way for working with Python projects, regardless of how many you might be busy with. It is very easy to use and an excellent tool to have at your disposable. It truly does wonders when coupled with *pip*.

We will start with installing virtualenv the system.

Installing virtualenv

—

In order to install *virtualenv*, we are going to call in *pip* for help. We will install it as a globally available package for the Python interpreter to run.

There are two ways to obtain the application. The version you will be able to get depends on which one you choose.

The simplest method is using *pip* to search, download and install. This might not provide you the latest stable version.

> **Downloading *virtualenv* using *pip*:**

sudo pip install virtualenv

> **Downloading the latest available one using *curl*:**

The latest release of *virtualenv* is 1.11.X.

sudo pip install https://github.com/pypa/virtualenv/tarball/1.1.X

Using *virtualenv*

—

Using this tool consists of getting it to create a folder, containing the Python interpreter and a copy of *pip*. Afterwards, in order to work with it, we need to either specify the location of that interpreter or activate it.

All the applications you install using the interpreter inside the virtual environment will be places within that location.

When you use *pip* to create a list of them, only the ones inside the folder will be compiled into a file.

> Remember: When you are done working with one environment, before switching to another – or working with the globally installed one – you need to make sure to deactivate it.

Creating / Initiating a *virtual environment* (virtualenv)

—

> Creating an environment using the same interpreter used to run it:

virtualenv my_app

> Creating an environment with a custom Python interpreter:

virtualenv –python=/opt/python-3.3/bin/python my_app

Activating a *virtual environment*

source my_app/bin/activate

Working with a *virtual environment* without activating

—

For various reasons, you might choose not to activate the environment before using it. This brings more flexibility to commands you run, however, you need to make sure to target the correct interpreter each and every time.

my_app/bin/python python_script.py

Using the *pip* installation inside the environment without activation

my_app/bin/pip install requests

Deactivating a *virtual environment*:

deactivate

href="https://twitter.com/ostezer">O.S. Tezer</div>