Upload a Python Package to PyPi
If you're frequently programming in Python and have developed custom tools that are useful across multiple projects, it's a smart move to consider publishing them on PyPI. This blog post will show you how to do that. We'll also cover how to manage your release versions and keep a tidy changelog.
Table of Contents
- -> Project Structure
- -> Necessary Files
- -> Build the Project
- -> Upload the Project
- -> Versioning
- -> Wrapping Up
- -> References
Project Structure
In this tutorial, we will upload our recently written project named "chorizo" to PyPi.
There are two types of valid project structures:
src-layout
The project should contain a src directory under the project root, and all modules and packages meant for distribution are placed inside this directory.
chorizo
|-- src
| |-- chorizo
| | |-- __init__.py
flat-layout
The package directory is the project root itself.
chorizo
|-- chorizo
| |-- __init__.py
In this project, we will use the latter, because it is simpler, cleaner, and easier to explore on GitHub. And because I just hate naming something "src", I have a cool name like "chorizo".
Necessary Files
For a project to be uploaded to PyPi, it needs to have a few files in the root directory.
chorizo
|-- chorizo
| |-- __init__.py
|
|-- LICENSE
|-- pyproject.toml
|-- README.md
|-- requirements.txt
LICENSE
A license is a legal tool you can use to protect your intellectual property. For choosing the most suitable license for your project, check https://choosealicense.com/ for more information.
The most common license for open source projects is the MIT license, so we will use that one. You just need to copy the text and paste it to the LICENSE file.
Don't forget to change the year and the name of the author.
MIT License
Copyright (c) [year] [fullname]
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
A README.md file is used to describe your project. It is a good practice to include a description of your project, the installation, some examples...
# chorizo
An example project named chorizo.
## Installation
```bash
pip install chorizo
```
pyproject.toml
The pyproject.toml file is used to configure the build process of your project. There are multiple backends available to build your project, but I am lately using hatchling. The backend tool will be automatically installed when building, so use the one you like the most.
It is important to update the version of your package every time you upload a new release.
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "chorizo"
version = "0.0.1"
authors = [
{ name="Example Author", email="author@example.com" },
]
description = "An example project named chorizo."
readme = "README.md"
requires-python = "> =3.10"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
dependencies = [
"numpy",
"matplotlib",
"pandas",
]
[project.urls]
Homepage = "https://github.com/user/chorizo"
Issues = "https://github.com/user/chorizo/issues"
While uploading my projects, I ran into some problems due to having a directory named "assets" in the root. It turns out, backend tools often ignore certain directories like "examples" or "tests" at the root level, but "assets" isn't typically one of them. I struggled to resolve this with setuptools, but switching to hatchling did the trick.
If this is your case, you can add the following lines to the pyproject.toml file to specify which directories to include in the build process:
[tool.hatch.build.targets.wheel]
packages = ["chorizo"]
requirements.txt
It is also a good practice to leave a requirements.txt with the dependencies in case someone wants to build from source.
numpy
matplotlib
pandas
.gitignore
As a bonus tip, you can add a .gitignore file to your project to avoid uploading unnecessary files to GitHub.
**/__pycache__
/dist
/build
*.egg-info
Build the Project
Now it's time to build your project and create the files that will be uploaded to PyPi.
First of, you need to install two packages: build and twine:
- build is a package that will build your project, as the name suggests.
- twine is a package that will upload your project to PyPi super easily.
pip install build twine
Once everything is set up, you can build your project.
python -m build
New directories will appear in your project root: dist and chorizo.egg-info. dist contains the files that will be uploaded to PyPi, and chorizo.egg-info contains some metadata about your project.
Upload the Project
The process of uploading your project to PyPi is a bit tricky. Firstly, there are two PyPi repositories: TestPyPi and PyPi. The first one is used to test your project before uploading it to the second one. They are independent servers, so you need to create an account on both.
Important:
Once you upload a release, it cannot be deleted, so you need to be careful.
Upload to TestPyPi
It is a good idea to upload first to the test server to check if everything is working as expected. To register an account, go to https://test.pypi.org/account/register/.
Now you need a token to upload your project. To get a token, go to https://test.pypi.org/manage/account/#api-tokens and click on "Add API token". You will need to set up a Two-Factor-Autentification, which is a bit annoying, but follow the steps.
Finally, you can upload your project to the test server.
python -m twine upload --repository testpypi dist/*
Download and Test
You can download your project from the test server with pip.
pip install --index-url https://test.pypi.org/simple/ --no-deps chorizo
Your package will be installed with no dependencies, so you need to install them manually using your requierements.txt.
pip install -r requirements.txt
Upload to PyPi
Once you are sure that everything is working as expected, you can upload your project to the PyPi server.
You need now to register an account in https://pypi.org/account/register/ and follow the same steps as before: enabling 2FA and generating a token.
python -m twine upload dist/*
Now your project can be installed with pip.
Versioning
Now it is time to upload your project to GitHub. We will tag the project with the released version and push it to GitHub.
git tag -a v0.0.1 -m "Release v0.0.1"
git push origin v0.0.1
Then, I use the GitHub CLI to upload the release files to GitHub. If you don't have it installed, you can do it with your package manager of choice.
sudo apt install gh
sudo dnf install gh
You will need to log in to your GitHub account.
gh auth login
And finally, you can upload the release files to GitHub. In our case, the files are in the dist directory.
gh release create v0.0.1 dist/*
Now you will be asked to fill out all the information about the release. I usually write a changelog.md and paste it in the description of the release. You can skip that step on the CLI and edit the release later on GitHub.
Wrapping Up
Now you know how to upload your projects to PyPi. It is a bit tricky at first, but once you get used to it, it is pretty straightforward. I hope you find it useful!