Installing latest Python 3.x and PIP on any Debian-based system

I’m still running Debian Jessie on one of my machines and it bugs me that the latest Python 3 I can install is 3.4, released over 4(!!!) years ago,
with the latest now being 3.7. Also, pip keeps breaking every time for some reason, so I decided to fix it properly - once and for all!

Remove old version

1. sudo apt purge python3 python3-pip → will remove Python 3 and PIP from the system.
2. sudo apt-get autoremove → will remove all traces of unnecessary dependencies (apt may not always work).

Installing dependencies

As noted in pyenv wiki:

$ sudo apt install build-essential llvm libncurses5-dev libncursesw5-dev wget curl zlib1g-dev libbz2-dev liblzma-dev xz-utils libffi-dev tk-dev libreadline-dev libsqlite3-dev libssl-dev

• build-essential → dpkg-dev g++ gcc libc-dev and make.
• llvm → Compiling optimizer.
• libncurses5-dev and libncursesw5-dev → An implementation of GNU ncurses.
• wget and curl → Should already be installed. Used for transfering files over the network.
• zlib1g-dev, libbz2-dev, liblzma-dev and xz-utils → Compressors (similar to zip).
• libffi-dev, tk-dev, libreadline-dev, libsqlite3-dev
• libssl-dev → OpenSSL library. See bottom note on possible errors to make sure the right version is installed (>=1.0.2).

Building from source

1. Obtain the latest version from Python.org ftp folder.
   We’re looking for a .tgz file with the name Python-3.x.x.tgz. For example: Python-3.7.0.tgz.
   Copy the link and download it with wget https://www.python.org/ftp/python/3.x.x/Python-3.x.x.tgz.
2. tar xvf Python-3.x.x.tgz → Untar it
3. cd Python-3.x.x → Enter the folder
4. ./configure --enable-optimizations --with-ensurepip=install → Run configuration with optimizations and install pip flags (those can be removed if unneeded).
5. make (Can be sped-up with -j4 flag, while 4=no. of cpu cores on the machine)
6. sudo make install (The official docs recommend using altinstall but since we’re reinstalling everything from scratch, it should be fine).

Creating aliases

By default, Python will install as python3.x and PIP as pip3.x. However, it’s also very important to create an alias to python (and maybe even python3) since a lot of programs and scripts depend on it.

1. whereis python → First, we look for our current available executable python binaries. Please note that this command will return a lot more than what we actually need. Look for the latest version stored in a bin folder and maybe also for Python 2.x, if installed. In my case I have /usr/bin/python2.7 for Python 2.7 and /usr/local/bin/python3.7 for Python 3.7.
2. In the following example we setup alias (alternative) that will reside in /usr/bin/python, will be called python with priority 50 and 30 for versions 3.7 and 2.7.

   $ sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.7 50
   $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 30
   

3. And the same can be done for pip:

   $ sudo update-alternatives --install /usr/local/bin/pip pip /usr/local/bin/pip3.7 50
   $ sudo update-alternatives --install /usr/local/bin/pip pip /usr/local/bin/pip2.7 30
   

   Note: Make sure to check first where PIP is located using which pip (notice that the target is different)

4. Check that both are working:

   $ python --version
   Python 3.7.0
   $ pip --version
   pip 10.0.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
   

Note: Aliases can be changed later on via update-alternatives --config python

Possible errors

libssl-dev < 1.0.2

After about an hour the make process finished and I noticed an error:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

Trying to install anything via pip resulted in the following error:

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not find a version that satisfies the requirement PACKAGE_NAME (from versions: )
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host=’pypi.org’, port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)) - skipping

Digging further, I found out that the latest version of libssl-dev available for jessie official channel was 1.0.1 (via sudo apt-cache policy libssl-dev).
In order to fix that I had to do the following:

1. sudo echo "deb http://ftp.debian.org/debian jessie-backports jessie main contrib non-free" >> /etc/apt/sources.list → Add the jessie-backports to apt sources list. (More on backports)
   Note: Ubuntu Trusty (14.04) users can use deb http://security.ubuntu.com/ubuntu xenial-security main instead.
2. sudo apt update → To acquire date from the new source.
3. sudo apt install -t jessie-backports libssl-dev=1.0.2l-1~bpo8+1 → Reinstall libssl-dev, this time explicitly defining the source (jessie-backports) and version (1.0.2l-1~bpo8+1). If this version is no longer supported, it’s possible to check it with sudo apt-cache policy libssl-dev or simply passing 1.0.2* as the version.
4. Recomile Python again :(
   Note: The easiest way would be to remove the untarred Python-3.x.x folder, untar it again and restart the process from there.

Old gcc version

The latest version of gcc (the C compiler) on Ubuntu Trusty (14.04) is 4.8.2 and this is the error I got while running make:

Current thread 0x00002aba1f9b8400 (most recent call first):
Aborted
generate-posix-vars failed
make[1]: *** [pybuilddir.txt] Error 1
make[1]: Leaving directory `/home/ubuntu/workspace/Python-3.7.0’
make: *** [profile-opt] Error 2

In order to fix that we need at least version 4.9.2 (from my experience), but if we’re upgrading, we might as well upgrade to the latest version, right?
Fortunately, there’s a ppa that contains a backlog of all gcc versions from 4 to 8.

$ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
$ sudo apt update
$ sudo apt install gcc-8

So we add the new ppa, refresh the database and install the latest gcc version 8. (I recommend checking the ppa link first to see if there’s an updated version)
Next, it’s important to make sure that gcc is tied to the new version:

$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 50

Note: On some Ubuntu systems apt-add-repository may not be present and can be installed via sudo apt install software-properties-common.
Alternatively (or if the above command is broken for some reason), it’s possible to manually add the ppa:
sudo echo "deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu trusty main" >> /etc/apt/sources.list (Mind the trusty part).
Then, add the signing key (otherwise apt update will give an error):

$ sudo gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/securing.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv 60C317803A41BA51845E371A1E9377A2BA9EF27F

The signing key can be found under “Technical details about this PPA” on the ppa page.

Bonus: Upgrading PIP

PIP versions are not always in par with the latest Python version, so an annoying notice may appear whenever installing packages:

You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.

So let’s do it ;) sudo pip install --upgrade pip