About the Scientific Python Lecture notes

About the Scientific Python Lecture notes#

Release: 2025.2rc0.dev0

The lectures are archived on Zenodo:

http://dx.doi.org/10.5281/zenodo.594102

Authors#

Editors#

  • K. Jarrod Millman

  • Stéfan van der Walt

  • Gaël Varoquaux

  • Emmanuelle Gouillart

  • Olav Vahtras

  • Pierre de Buyl

  • Peter Rush

  • Matthew Brett

Chapter authors#

Listed by alphabetical order.

  • Christopher Burns

  • Adrian Chauve

  • Robert Cimrman

  • Christophe Combelles

  • André Espaze

  • Emmanuelle Gouillart

  • Mike Müller

  • Fabian Pedregosa

  • Didrik Pinte

  • Nicolas Rougier

  • Gaël Varoquaux

  • Pauli Virtanen

  • Zbigniew Jędrzejewski-Szmek

  • Valentin Haenel (editor from 2011 to 2015)

Additional Contributions#

Listed by alphabetical order

  • Osayd Abdu

  • arunpersaud

  • Ross Barnowski

  • Sebastian Berg

  • Lilian Besson

  • Matthieu Boileau

  • Joris Van den Bossche

  • Michael Boyle

  • Matthew Brett

  • BSGalvan

  • Lars Buitinck

  • Pierre de Buyl

  • Ozan Çağlayan

  • Lawrence Chan

  • Adrien Chauve

  • Robert Cimrman

  • Christophe Combelles

  • David Cournapeau

  • Dave

  • dogacan dugmeci

  • Török Edwin

  • egens

  • Andre Espaze

  • André Espaze

  • Loïc Estève

  • Corey Farwell

  • Tim Gates

  • Stuart Geiger

  • Olivier Georg

  • Daniel Gerigk

  • Robert Gieseke

  • Philip Gillißen

  • Ralf Gommers

  • Emmanuelle Gouillart

  • Julia Gustavsen

  • Omar Gutiérrez

  • Matt Haberland

  • Valentin Haenel

  • Pierre Haessig

  • Bruno Hanzen

  • Michael Hartmann

  • Jonathan Helmus

  • Andreas Hilboll

  • Himanshu

  • Julian Hofer

  • Tim Hoffmann

  • B. Hohl

  • Tarek Hoteit

  • Gert-Ludwig Ingold

  • Zbigniew Jędrzejewski-Szmek

  • Thouis (Ray) Jones

  • jorgeprietoarranz

  • josephsalmon

  • Greg Kiar

  • kikocorreoso

  • Vince Knight

  • LFP6

  • Manuel López-Ibáñez

  • Marco Mangan

  • Nicola Masarone

  • John McLaughlin

  • mhemantha

  • michelemaroni89

  • K. Jarrod Millman

  • Mohammad

  • Zachary Moon

  • Mike Mueller

  • negm

  • John B Nelson

  • nicoguaro

  • Sergio Oller

  • Theofilos Papapanagiotou

  • patniharshit

  • Fabian Pedregosa

  • Philippe Pepiot

  • Tiago M. D. Pereira

  • Nicolas Pettiaux

  • Didrik Pinte

  • Evgeny Pogrebnyak

  • reverland

  • Maximilien Riehl

  • Kristian Rother

  • Nicolas P. Rougier

  • Pamphile Roy

  • Rutzmoser

  • Sander

  • João Felipe Santos

  • Mark Setchell

  • Helen Sherwood-Taylor

  • Shoeboxam

  • Simon

  • solarjoe

  • ssmiller

  • Scott Staniewicz

  • strpeter

  • surfer190

  • Bartosz Telenczuk

  • tommyod

  • Wes Turner

  • Akihiro Uchida

  • Utkarsh Upadhyay

  • Olav Vahtras

  • Stéfan van der Walt

  • Gaël Varoquaux

  • Nelle Varoquaux

  • Olivier Verdier

  • VirgileFritsch

  • Pauli Virtanen

  • Yosh Wakeham

  • yasutomo57jp

What’s new#

Release 2024.1 (April 2024)#

  • Python 3.10, 3.11, 3.12

  • Renamed Scientific Python Lectures

  • Removed old content

  • Major updates to support recent packages

  • Updates to the SciPy and scikit-image chapters

Release 2022.1 (August 2022)#

  • Replace scikit-learn housing example with California data (Marco Mangan)

  • Fix links and typos (Zachary Moon, Tim Gates, Marco Mangan, Gert-Ludwig Ingold)

  • Fix fftpack figure (Osayd Abdu)

  • Update software version (Pierre de Buyl)

Release 2020.2 (September 2020)#

  • Replace image i/o from scipy.misc by imageio (Pierre de Buyl)

  • Update information on dict ordering (Bharath Saiguhan)

  • Suppress warnings for mandelbrot example (Pierre de Buyl)

  • Update NumPy introduction and advanced usage for changes to NumPy: wording, bytes representation, floating point argument to np.zeros (Ross Barnowski)

  • Fix links to NumPy documentation to use numpy.org (Ross Barnowski)

  • Update note on transposed arrays (Ross Barnowski with Eric Wieser)

  • Use generated figure file for lidar data processing (Lawrence Chan)

  • Update link from PyMC2 to PyMC3 (B. Hohl)

  • Fix transparent popup menu to have a background (Pierre de Buyl)

Release 2020.1 (March 2020)#

  • Fix outdated URLs (Gert-Ludwig Ingold)

  • Update packages (Pierre de Buyl)

  • Remove Python 2 continuous integration (Olav Vahtras - EuroSciPy 2019 sprint)

  • Fix chessboard size (Mark Setchell)

  • Add objectives and design choices (Gert-Ludwig Ingold and Pierre de Buyl)

  • Make the numpy advanced iterator example more elaborate (Sebastian Berg)

  • Use empty list instead of empty tuple to deactivate ticks (Tim Hoffmann)

  • Fix typos (Sander van Rijn, cydave, Michel Corne) and off by 2 errors (Andreas Hilboll)

  • Improve readability of Polynomials example code (Michel Corne)

  • Replace suggestions for debugging environments (Gert-Ludwig Ingold)

  • Add section on Python 2 vs Python 3 (Pierre de Buyl)

Release 2019.1 (May 2019)#

  • Update matplotlib compatibility to version 2.2 (Mike Mueller, Joris Van den Bossche, Pierre de Buyl)

  • Make C-API example cos_module_np Python 2/3 compatible (Michael Boyle)

  • Fix typos and outdated URLs (Dogacan Dugmeci, Matthieu Boileau, Stuart Geiger, Omar Gutiérrez, Himanshu, Julian Hofer, Joseph Salmon, Manuel López-Ibáñez, Nicola Masarone, michelemaroni89, Evgeny Pogrebnyak, tommyod)

Release 2018.1 (September 2018)#

  • Fix wordings, typos, colours (Pierre de Buyl, Greg Kiar, Olav Vahtras Kristian Rother)

  • Fix interpolation example code (Scott Staniewicz)

  • Fix CSS for high density displays (Gaël Varoquaux)

  • Generate indexing figures with PyX (Gert Ingold)

  • Warn clearly against the use of Python 2 (Bruno Hanzen)

  • Update external links (Bruno Hanzen)

  • Update versions of dependencies: sphinx-gallery, pandas, statsmodels (Gaël Varoquaux)

Release 2017.1 (October 2017)#

  • Update optimization chapter (Michael Hartmann, Gaël Varoquaux)

  • Update SymPy chapter (Vince Knight)

  • Update advanced NumPy (Bartosz Teleńczuk)

  • Update scikit-learn chapter (Gaël Varoquaux)

  • Update SciPy chapter (Gaël Varoquaux)

  • Make ‘>>>’ in the prompts unselectable (Pierre de Buyl)

  • Use common package requirements for pip and conda and improve the build instructions (Gert-Ludwig Ingold, Vince Knight, Pierre de Buyl)

  • Set up Circle CI (Loïc Estève)

  • Improved support for Python 3 integer divisions and calls to print (Loïc Estève, Gert-Ludwig Ingold, Pierre de Buyl, Gaël Varoquaux)

  • Change test runner to pytest (Pierre de Buyl)

  • Replace the plot directive by sphinx-gallery (Gert-Ludwig Ingold)

Release 2016.1 (September 2016)#

  • Rework of intro chapter (Gaël Varoquaux)

  • Integrate sphinx-gallery: examples are now Jupyter notebooks (Gaël Varoquaux, Gert-Ludwig Ingold, Óscar Nájera)

  • Better Python 3 tests and support (Gert-Ludwig Ingold)

  • Adapt examples to Matplotlib 1.5 (Gaël Varoquaux)

  • Modernize numpy chapter (Bartosz Telenczuk)

Release 2015.3 (November 2015)#

  • Collapsed sidebar can now pop up for mid-sized display (Gaël Varoquaux)

  • Replaced pictures of Lena by raccoon face (Thouis Jones)

Release 2015.2 (October 2015)#

  • Authors on cover ordered as in bibtex entry (Nicolas Rougier)

  • Better rendering on mobile (Gaël Varoquaux)

  • Fix restructured text markup errors (Olav Vahtras)

Release 2015.1 (September 2015)#

  • New chapter on statistics with Python (Gaël Varoquaux)

  • Better layout in PDF (Gaël Varoquaux)

  • New HTML layout, simplified formatting, mobile-friendly and sidebar (Gaël Varoquaux, Nelle Varoquaux)

  • Logos on the HTML front page and on the PDF cover (Nicolas Rougier)

  • Python 3 compatible code (Gaël Varoquaux, Olav Vahtras)

  • Code put up to date for more recent versions of project (Pierre de Buyl, Emmanuelle Gouillart, Gert-Ludwig Ingold, Nicolas Pettiaux, Olav Vahtras, Gaël Varoquaux, Nelle Varoquaux)

  • Matplotlib updated with removal of deprecated pylab interface (Nicolas Rougier)

Release 2013.2 (21 August 2013)#

  • NumPy chapter simplified (Valentin Haenel)

  • New layout for the HTML rendering (Gaël Varoquaux)

Release 2013.1 (10 Feb 2013)#

  • Improvements to the advanced image manipulation chapter (Emmanuelle Gouillart)

  • Upgrade of the introductory language chapter (Valentin Haenel)

  • Upgrade of the introductory numpy chapter (Valentin Haenel)

  • New advanced chapter on interfacing with C (Valentin Haenel)

  • Minor fixes and improvements in various places (Robert Gieseke, Ozan Çağlayan, Sergio Oller, kikocorreo, Valentin Haenel)

Release 2012.3 (26 Nov 2012)#

This release integrates the changes written for the Euroscipy conference:

  • Matplotlib chapter completely redone (Nicolas Rougier, Gaël Varoquaux)

  • New advanced chapter on mathematical optimization (Gaël Varoquaux)

  • Mayavi chapter redone (Gaël Varoquaux)

  • Front page layout slightly improved: folding TOC (Gaël Varoquaux)

Release 2012.2 (22 Jun 2012)#

Minor release with a few clean ups (Gael Varoquaux).

Release 2012.1 (20 Jun 2012)#

This is a minor release with many clean ups. In particular, clean up of the layout (Gael Varoquaux), shortening of the numpy chapters and deduplications across the intro and advanced chapters (Gael Varoquaux) and doctesting of all the code (Gael Varoquaux).

Release 2012.0 (22 Apr 2012)#

This is a minor release with a few clean ups. In particular, clean up the scikit-learn chapter (Lars Buitinck), more informative section titles (Gael Varoquaux), and misc fixes (Valentin Haenel, Virgile Fritsch).

Release 2011.1 (16 Oct 2011)#

This release is a reworked version of the Euroscipy 2011 tutorial. Layout has been cleaned and optimized (Valentin Haenel and many others), the Traits chapter has been merged in (Didrik Pinte)

Release 2011 (1 Sept 2011)#

This release is used for the Euroscipy 2011 tutorial. The numpy introductory chapter has been rewamped (Pauli Virtanen). The outline of the introductory chapters has been simplified (Gaël Varoquaux). Advanced chapters have been added: advanced Python constructs (Zbigniew Jędrzejewski-Szmek), debugging code (Gaël Varoquaux), optimizing code (Gaël Varoquaux), image processing (Emmanuelle Gouillart), scikit-learn (Fabian Pedregosa).

License#

All code and material is licensed under a

Creative Commons Attribution 4.0 International License (CC-by)

https://creativecommons.org/licenses/by/4.0/

See the AUTHORS file for a list of contributors.

Contributing#

The Scientific Python Lectures are a community-based effort and require constant maintenance and improvements. New contributions such as wording improvements or inclusion of new topics are welcome.

To propose bugfixes or straightforward improvements to the lectures, see the contribution guide below.

For new topics, read the objectives first and open an issue on the GitHub project to discuss it with the editors.

Objectives and design choices for the lectures#

Contributors should keep the following objectives and design choices of the Scientific Python Lectures in mind.

Objectives:

  • Provide a self-contained introduction to Python and its primary computational packages, the ”Scientific Python stack“.

  • Provide tutorials for a selection of widely-used and stable computational libraries. Currently, we cover Pandas, Statmodels, some of Seaborn, Scikit-image, Scikit-learn, and Sympy.

  • We would like to apply automated testing to the code examples as much as possible.

Design choices:

  • Each chapter should provide a useful basis for a 1‒2 h tutorial.

  • The code should be readable.

  • An idiomatic style should be followed, e.g. import numpy as np, preference for array operations, PEP8 coding conventions.

Contributing guide#

The directory guide contains instructions on how to contribute:

Contribution guide

Building instructions#

To generate the html output for on-screen display, Type:

make html

the generated html files can be found in build/html

The first build takes a long time, but information is cached and subsequent builds will be faster.

Requirements#

Build requirements are listed in the requirements file:

# Requirements for notebooks / Binderhub
numpy==2.2.5
scipy==1.15.2
matplotlib==3.10.1
pandas==2.2.3
scikit-learn==1.6.1
scikit-image==0.25.2
sympy==1.14.0
statsmodels==0.14.4
seaborn==0.13.2
pytest>=8.3
sphinx
sphinx-copybutton
coverage>=7.6
Pillow
pooch
ipython
pickleshare
requests
xlrd
openpyxl
jupytext
# For pretty rendering in local JupyterLab or JupyterLite.
jupyterlab_myst
# For glue markup in notebooks.
myst_nb

Ensure that you have a virtual environment or conda environment set up, then install requirements with:

pip install -r requirements.txt

Note that you will also need the following system packages:

  • Python C development headers (the python3-dev package on Debian, e.g.),

  • a C compiler like gcc,

  • GNU Make,

  • a full LaTeX distribution such as TeX Live (texlive-latex-base, texlive-latex-extra, texlive-fonts-extra, and latexmk on Debian/Ubuntu),

  • dvipng,

  • latexmk,

  • git.

Updating the cover#

Use Inkscape to modify the cover in images/, then export to PDF:

inkscape --export-filename=cover-2025.pdf cover-2025.svg

Ensure that the images/cover.pdf symlink points to the correct file.

Notes and admonitions#

Use ::: for <div> blocks (JupyterBook allows this): So, for example, prefer:

::: {note}

My note

:::

to the more standard Myst markup of:

<!-- #region -->
``` {note}

My note

```
<!-- #endregion -->

Note the region and endregion markup in the second form; this makes more sure that Jupytext does not confuse the {note} with a code block. One of the advantages of the ::: markup is that you don’t need these #region demarcations.

For the same reason, prefer the ::: form for other content blocks, such as warnings and admonitions. For example, prefer:

::: {admonition} A custom title

My admonition

:::

Exercises and solutions#

We use sphinx-exercise for the exercises and solutions.

Mark all exercises and solutions with gated markers, like this:

::: {exercise-start}
:label: my-exercise-label
:class: dropdown
:::

My exercise.

::: {exercise-end}
:::

::: {solution-start} my-exercise-label
:class: dropdown
:::

My solution.

::: {solution-end}
:::

The gated markers (of form solution-start and solution-end etc) allow you to embed code cells in the exercise or solution, because this allows code cells to be at the top level of the notebook, where Jupyter needs them to be.

The gated markers also make it possible to for the process_notebooks.py script to recognize exercise and solutions blocks, to parse them correctly.

A note on processing#

The pages are designed both as pages for pretty HTML output, and to be used as interactive notebooks in e.g. JupyterLite.

There is some markup that we need for the pretty HTML output that looks ugly in a Jupyter interface such as JupyterLite. To deal with this in part, we install the jupyterlab_myst extension by default, so that Myst markup (mostly) appears as it should inside JupyterLab when opened as a notebook. Another difference we want to see between the HTML and the notebook version is that we want to avoid putting the solutions in the notebook version, to allow more space for thought about the exercise. Both to modify any ugly formatting, and to remove the exercise solutions, we post-process the pages with a script _scripts/process_notebooks.py to load the pages as text notebooks, and write out .ipynb files with modified markup that looks better in a Jupyter interface. Some of the authoring advice here is to allow that process to work smoothly, because the process_notebooks.py file reads the input Myst-MD format notebooks using Jupytext before converting to Jupyter .ipynb files.

Tests#

There may well be cases where you will want to put cells in the rendered notebook that test values, as part of the exposition. For example, from the intro/scipy/index.md notebook / page:


```{code-cell}
A_upper = np.triu(A)
A_upper
```

```{code-cell}
np.allclose(sp.linalg.solve_triangular(A_upper, b, lower=False),
            sp.linalg.solve(A_upper, b))
```

Notice that, in this case, we do want the reader to see that test, as part of the exposition.

However, there are cases where the test would be useful, to, for example, detect changes in the output over versions of the packages being used. We want to avoid the situation where the text says one thing, but the values contradict it. But we may not want the reader to have to read such tests as part of the exposition. Here, for example, is a test from the intro/scipy/index.md notebook:


```{code-cell}
log_a = sp.special.gammaln(500)
log_b = sp.special.gammaln(499)
log_res = log_a - log_b
res = np.exp(log_res)
res
```

```{code-cell}
:tags: [remove-cell, test]
assert np.allclose(res, 499)
```

Note that the test confirms that Scipy is still giving the output implied in the text. Note too that we have given the testing code cell the tag remove-cell. This drops the cell from the HTML output, and our post-processing of the notebooks also drops these cells, so someone opening the notebook in e.g. JupyterLite will not see them. Accordingly, please make sure you are not defining anything in these test cells that the notebook will need in later cells.

Be judicious — testing the output of np.ones(3) is probably not useful — Numpy would have to break in order for that test to fail.

Development#

Run this once, in the repository directory:

pip install pre_commit
pre-commit install

Before each commit that you will push:

pre-commit run --all

Among other things, this runs the codespell check, also run by CI.

Contents