PyPy is an alternate implementation of the Python interpreter. The regular Python interpreter (nicknamed “CPython”) is written in the C language.  The PyPy interpreter is written in a dialect of Python (“RPython”) and includes a number of interesting features, including a just-in-time compiler that makes it faster than the CPython interpreter when running Python code.

Sections

 

Features

  • Supports Python versions 2.7, 3.9, and 3.10
  • On average 4.8x faster than the regular CPython interpreter for Python code
  • Reduced memory usage compared with CPython
  • Compatibility with most existing Python libraries.
  • Support for stackless, a Python threading system that allows for hundreds (or more) of threads to be effectively used in a program.

 

Install PyPy

The PyPy website has pre-compiled binaries that can be downloaded and installed into your SCC project storage. However, The easiest way to try out PyPy is to create a conda environment for it. Installing libraries (such as numpy, pandas, etc.) through conda will then make sure that versions of libraries compiled specifically for PyPy are installed when available. The highest Python version supported in conda environments is 3.9.  As an example, here is how to create a conda environment on the SCC using PyPy for Python 3.9:

  1. Load a miniconda module: 
    [rcs@scc1 ~] module load miniconda/23.5.2
  2. Create a new conda environment named my-pypy-env, specifying the default channel to make sure only compatible PyPy packages are found: 
    [rcs@scc1 ~] conda create -c conda-forge -n my-pypy-env pypy python=3.9
  3. Activate the new conda environment:
    4. [rcs@scc1 ~] conda activate my-pypy-env
  4. Configure this environment to use strict channel priority so that conda-forge is always checked first for packages. This is a one-time setting: 
    (my-pypy-env) [rcs@scc1 ~] conda config --env --set channel_priority strict
  5. Install PyPy compatible libraries: 
    (my-pypy-env) [rcs@scc1 ~] conda install -c conda-forge numpy scipy pandas jupyterlab scikit-learn
  6. Check the version of the Python interpreter: 
    (my-pypy-env) [rcs@scc1 ~] python -V
Python 3.9.18 | packaged by conda-forge | (c5262994, Oct 26 2023, 08:28:20)
[PyPy 7.3.13 with GCC 12.3.0]
  7. Run your Python code with PyPy: 
    (my-pypy-env) [rcs@scc1 ~] python your_code.py

 

When to use PyPy

PyPy runs Python code faster. If you are running a Python program with a lot of pure Python code, using Python classes, objects, lists, and so on it will likely run significantly faster when executed under PyPy. If your program extensively uses libraries with externally compiled code like pandas, numpy, or scikit-learn, it may run faster. The speedup will come from faster execution of Python code, while the calls to the external libraries will not be accelerated.

There are some libraries that use compiled code that are not currently compatible with PyPy. The compatibility issue is based on the way the libraries communicate with the Python interpreter. Some popular libraries that are not compatible with PyPy include PyTorch, Tensorflow, and Numba.

If your Python program does not use libraries that are incompatible with PyPy, then it is worth testing out PyPy to see if it provides a performance benefit. The test would consist of creating a conda environment as described above, installing any required libraries, and trying to run your Python program.
 

Back to top