Publish Python Package to PyPi Jun 23rd 2020 Words: 1.2k

Publish your python package to PyPi so the world can import your package in their project.

Thousands of libraries created by diligent devs make Python project easy to start, and as a Python player, I use pip install and import thousands times a year.

Recently I create a simple plugin and want to publish it to PyPi, because may be someone would find it helpful.

Prepare package for publishing

I find this official instruction helpful on how to packaging a project.

Initially my project structure looks like this:

1
2
my-app-plugin
└── MyAppPlugin.py

I’d like to publish my child class MyAppPlugin as my-app-plugin, therefore anyone want to use the package simply run pip install my-app-plugin and add import MyAppPlugin in their Python file.

Make a Package

To create the package, a folder with a __init__.py file is first created to wrap the module:

1
2
3
4
my-app-plugin
└── my_app_plugin
    ├── __init__.py
    └── _MyAppPlugin.py

To make the package import controlled by __init__.py and prevent direct import of the modules, a prefix underscore is added to MyAppPlugin.

The import statement uses the following convention: if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered.

In short, __init__.py controls what content to exports, it can be empty (means export all as default).
The Python 3 document for packages provides more information. 

1
2
3
4
5
6
7
# __init__.py

from ._MyAppPlugin import MyAppPlugin

__all__ = [
    "MyAppPlugin"
]

Setuptools is a collection of enhancements to the Python distutils that allow developers to more easily build and distribute Python packages, especially ones that have dependencies on other packages.

setup.py

setup.py is the script used to install the package, it has the information about the package like author, version, dependencies. It usually contains a setup function.

The setup function can be imported from two packages, setuptools or distutils.core. I prefer setuptools.setup.

Setuptools is a collection of enhancements to the Python distutils that allow developers to more easily build and distribute Python packages, especially ones that have dependencies on other packages.

1
2
3
4
5
my-app-plugin
├── setup.py
└── my_app_plugin
    ├── __init__.py
    └── _MyAppPlugin.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# setup.py
from setuptools import setup, find_packages

setup(
    name="my-app-plugin",
    version="1.0.0",
    description="Lorem ispsum dolor sit amet",
    long_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis id lacinia sem. Nulla auctor lorem sit amet consequat maximus. ",
    long_description_content_type="text/markdown",
    license="BSD",
    url="https://github.com/username/my-app-plugin",
    packages=find_packages(),
    install_requires=["click>=7.0,<8"],
    classifiers=[
        "Development Status :: 4 - Beta",
        "Environment :: Console",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: BSD License",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.6",
        "Programming Language :: Python :: 3.7",
        "Programming Language :: Python :: 3.8"
    ],
    python_requires='>=3.6'
)

It is just an example, the document of setuptools describes the usage in detail.

Versioning

In some simple project it is okay to hard code the version string in the setup.py like the above code snippet. However, if the version string is used in multiple places, it will be difficult to change them later one by one. In practice, a file, usually named version.py or _version.py, which declares version as a global variable, is often used. Any module, including setup.py, import the version string from this version file. By using this technique, when the version changes only the version file needs to be modified. 

1
2
3
4
5
6
my-app-plugin
├── setup.py
└── my_app_plugin
    ├── __init__.py
    ├── _version.py
    └── _MyAppPlugin.py
1
2
3
4
# _version.py

__version__ = "1.0.0"

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# setup.py

from pathlib import Path
from setuptools import setup, find_packages

def get_version():
    version = {}
    with open("./click_constrained_option/_version.py") as f:
        exec(f.read(), version)
    return version["__version__"]

setup(
    name="my-app-plugin",
    version=get_version(),
    description="Lorem ispsum dolor sit amet",
    long_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis id lacinia sem. Nulla auctor lorem sit amet consequat maximus. ",
    long_description_content_type="text/markdown",
    license="BSD",
    url="https://github.com/username/my-app-plugin",
    packages=find_packages(),
    install_requires=["click>=7.0,<8"],
    classifiers=[
        "Development Status :: 4 - Beta",
        "Environment :: Console",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: BSD License",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.6",
        "Programming Language :: Python :: 3.7",
        "Programming Language :: Python :: 3.8"
    ],
    python_requires='>=3.6'
)

README.md and Wiki page (Optional)

Make the project friendly by creating a well structured README or wiki.

I generally include the following section in a README file:

  • Brief introduction (less than 3 sentence)
  • How to install/deploy it
  • How to use it
  • argument list/configuration file (if any)
  • Examples of some use case
  • Frequent Q&A

Frankly speaking, I am still struggled to write a document for my project, because I am the the developer who know the it too well to imagine what a user would encounter when attempt to use the tool for the first time.

Choose a license

Use Choose an open source license to help decide which license to use.

Test (Optional)

Constructing tests is exhausting, and it usually takes even more time. However, it helps you to deliver your code with confident, without diving to debug-fix loop again and again for each update. Plus watching the green ticks is pure satisfaction. ✅

I use unittest in Python and keep all my test instances under tests folder. 

1
2
3
4
5
6
7
8
9
10
my-app-plugin
├── setup.py
├── tests
│   ├── __init__.py
│   ├── test_a.py
│   └── test_b.py
└── my_app_plugin
    ├── __init__.py
    ├── _version.py
    └── _MyAppPlugin.py

Publish

Generating distribution archives

1
pip install --user --upgrade setuptools wheel

Now run this command from the same directory where setup.py is located:

1
python3 setup.py sdist bdist_wheel

Now you will have a dist folder contains a .whl file and a tarball.

Upload to TestPyPI (Optional)

You can try distribution tools and processes without affecting the real index on TestPyPI.

Create a PyPI account at Python Package Index, if you don’t have one yet.

Install twine, it’s the tool used for uploading. 

1
pip install --user --upgrade twine

Upload to TestPyPI:

1
twine upload --repository testpypi dist/*

The username and the password will be prompted.

After uploading to TestPyPI, the package can be installed by using the following command: 

1
pip install --index-url https://test.pypi.org/simple/ my-app-plugin

Upload to PyPI

After test the package on TestPyPI, now it’s time to bring it to the real world.

Create a PyPI account at Python Package Index, if you don’t have one yet. 

1
twine upload dist/*

Enter your username and password for pypi.org and complete the upload.

Congratulations, you have successfully published your python package.