Module directories

In the modules introduction, we found that we could make a Python module with a single file that has a .py file extension. The .py file has to be in directory on the Python path.

This is the simplest form of a Python module — a single .py file.

These can be useful, but we often want to add many functions and attributes to our module. This would make the .py file very large.

Python gives us another way of creating a module that we will call module directories. This allows us to define a module with a directory that can have many .py files.

It is easiest to see this by example.

Note: Here we use Python and the notebook commands to create the module. This is just to show the process. Normally you would use your terminal and text editor to create the directories and files.

We will call our new module mydirmod (for My Directory Module).

Creating the directory

It’s a directory, so we first create the directory.

The directory may already exist from a prior run through of this notebook, so we start by deleting the directory, if it exists.

We use a pathlib Path object to check if the directory exists, and the rmtree function from the shutil module to remove it, if it does:

import os
from pathlib import Path
import shutil

# Remove mydirmod directory if it exists.
mod_path = Path('mydirmod')
if mod_path.is_dir():
    shutil.rmtree(mod_path)

With that out of the way, we create the directory.

mod_path.mkdir()

Next we put a .py file into that directory, using the %%file notebook command:

%%file mydirmod/some_module.py
""" A sub-module in mydirmod
"""

def myfunc(a):
    return a * 10

Writing mydirmod/some_module.py

Just to confirm, we show the files in mydirmod:

list(mod_path.glob('*'))  # Get, show all files

[PosixPath('mydirmod/some_module.py')]

But, our work here is not yet done. To make the mydirmod into a directory module, we have to do one more step.

Making mydirmod into a module

The key step to tell Python that mydirmod is a directory module, is to create an __init__.py file inside the directory. Notice the double underscores, indicating that this filename is special for Python. For the moment, let’s create a file that has (virtually) nothing in it:

%%file mydirmod/__init__.py
""" An __init__.py file that only has a docstring
"""

Writing mydirmod/__init__.py

Hey presto, we can import the module.

import mydirmod

However, tab-completion reveals that mymod has nothing inside it. In particular, it does not have the some_module.py file sub-module, nor does it have the myfunc function from that sub-module.

To import the sub-module, we need to do it explicitly, like this:

import mydirmod.some_module

# Use myfunc from the sub-module.
print(mydirmod.some_module.myfunc(9))

90

Actually, there is another way to get to functions in sub-modules, and that is to import the sub-module in the __init__.py file. Let’s do that:

%%file mydirmod/__init__.py
""" An __init__.py file that only has a docstring
"""

# Notice the . at the beginning of .some_module.
from .some_module import myfunc

Overwriting mydirmod/__init__.py

The . at the beginning of .some_module refers to the current directory, meaning, the directory containing the __init__.py file. It tells __init__.py to import the some_module.py file in its directory. Note: This is called a relative import, because some_module.py is in the directory . relative to the __init__.py file. These kinds of imports only work in module directories.

OK, let’s try that:

import mydirmod

# Use myfunc from the sub-module.
print(mydirmod.myfunc(9))

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[10], line 4
      1 import mydirmod
      3 # Use myfunc from the sub-module.
----> 4 print(mydirmod.myfunc(9))

AttributeError: module 'mydirmod' has no attribute 'myfunc'

Oh dear - it didn’t work. Why not? Because we need to Changing the module, reloading the module.

import importlib

importlib.reload(mydirmod)

<module 'mydirmod' from '/home/runner/work/dipy-textbook/dipy-textbook/mydirmod/__init__.py'>

# Use myfunc from the sub-module.
print(mydirmod.myfunc(9))

90

As your modules become more than slightly complicated, you will want to switch from using single .py file module, to directory modules like this one.

See also

Modules in the standard Python tutorial.