Keeping in mind the current situation across the world. I hope everyone is doing good. Please take precautions and stay at home and keep your self busy in whatever way you want to be.
I was reading the book "Python for DevOps" and came across the topic "Packaging". In every business, packaging plays a big role while it comes to product distribution.
While it comes to IT software usually, below are the few things which should take care of :
- Descriptive Versioning
- In Python packages, the following two variants are used:
- major.minor
- major.minor.micro
- major - for backward-incompatible changes
- minor - adds features that are also backward compatible
- micro - adds backward-compatible bug fixes.
- The Changelog
- This is a simple file that keeps track of all the changes we will be doing for each version upgrade.
Not going in detail here, coming directly to implementation on how we can handle packaging in python using the "__init__.py" file.
The tool used here for packaging is "setuptools" python module.
Now we'll create python virtual environment and add "setuptools" there as below -
$ python3 -m venv /tmp/packaging $ source /tmp/packaging/bin/activate
$ pip3 install setuptools
Tip - you can cross check the list of installed modules using pip3 as below.
Now, let's see the code. I have simple hello-world examples as below
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $tree . . ├── README ├── hello_world │ ├── __init__.py │ ├── hello_python.py │ └── hello_world.py └── setup.py 1 directory, 5 files
Note -
- README - simple instructions
- hello_world(directory) - module name
- __int__.py - organize modules while keeping them in directory
- hello_*.py - two different module with different functionality
- setup.py - required by "setuptools" to build a package.
Source code is available on GitHub page as well.
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat hello_world/hello_world.py
def helloworld():
return "HELLO WORLD"
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat hello_world/hello_python.py
def hellopython():
return "HELLO PYTHON3"
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat hello_world/__init__.py from .hello_python import hellopython from .hello_world import helloworld
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat setup.py
from setuptools import setup, find_packages
setup(
name="hello_example",
version="0.0.1",
author="Example Author",
author_email="author@example.com",
url="example.com",
description="A hello-world example package",
packages=find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
Now, lets start packaging our hello world program:
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $python3 setup.py sdist running sdist running egg_info creating hello_example.egg-info writing hello_example.egg-info/PKG-INFO writing dependency_links to hello_example.egg-info/dependency_links.txt writing top-level names to hello_example.egg-info/top_level.txt writing manifest file 'hello_example.egg-info/SOURCES.txt' reading manifest file 'hello_example.egg-info/SOURCES.txt' writing manifest file 'hello_example.egg-info/SOURCES.txt' running check creating hello_example-0.0.1 creating hello_example-0.0.1/hello_example.egg-info creating hello_example-0.0.1/hello_world copying files to hello_example-0.0.1... copying README -> hello_example-0.0.1 copying setup.py -> hello_example-0.0.1 copying hello_example.egg-info/PKG-INFO -> hello_example-0.0.1/hello_example.egg-info copying hello_example.egg-info/SOURCES.txt -> hello_example-0.0.1/hello_example.egg-info copying hello_example.egg-info/dependency_links.txt -> hello_example-0.0.1/hello_example.egg-info copying hello_example.egg-info/top_level.txt -> hello_example-0.0.1/hello_example.egg-info copying hello_world/__init__.py -> hello_example-0.0.1/hello_world copying hello_world/hello_python.py -> hello_example-0.0.1/hello_world copying hello_world/hello_world.py -> hello_example-0.0.1/hello_world Writing hello_example-0.0.1/setup.cfg creating dist Creating tar archive removing 'hello_example-0.0.1' (and everything under it)
After this, you will see that the above command has created other folders as well and our packaged module has been stored in "dist" folder.
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $tree . . ├── README ├── dist │ └── hello_example-0.0.1.tar.gz ├── hello_example.egg-info │ ├── PKG-INFO │ ├── SOURCES.txt │ ├── dependency_links.txt │ └── top_level.txt ├── hello_world │ ├── __init__.py │ ├── hello_python.py │ └── hello_world.py └── setup.py 3 directories, 10 files
Now let's install and list down the modules using "pip3".
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $pip3 install dist/hello_example-0.0.1.tar.gz
Processing ./dist/hello_example-0.0.1.tar.gz
Installing collected packages: hello-example
Running setup.py install for hello-example ... done
Successfully installed hello-example-0.0.1
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $ pip3 list --format=columns
Package Version
------------- -------
hello-example 0.0.1
pip 20.0.2
setuptools 41.2.0
As the module has been installed, let's test this first using "ipyhon" console and then using the python program.
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $ipython3
/usr/local/lib/python3.7/site-packages/IPython/core/interactiveshell.py:931: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
warn("Attempting to work in a virtualenv. If you encounter problems, please "
Python 3.7.7 (default, Mar 10 2020, 15:43:03)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import hello_world as hw
In [2]: hw.hellopython()
Out[2]: 'HELLO PYTHON3'
In [3]: hw.helloworld()
Out[3]: 'HELLO WORLD'
In [4]:
Here you saw that I can call the function using my custom module "hello_world".
Using this module in the program-
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat example.py
import hello_world as hw
print("Calling Hello Python Function: "+hw.hellopython())
print("Calling Hello World Function: "+hw.helloworld())
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $python3 example.py
Calling Hello Python Function: HELLO PYTHON3
Calling Hello World Function: HELLO WORLD
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
Now, let's suppose you want to upgrade to version "0.0.2", then we will follow the below steps -
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat hello_world/hello_python.py
def hellopython():
return "HELLO PYTHON3 with version **0.0.2**"
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat hello_world/hello_world.py
def helloworld():
return "HELLO WORLD with version ** 0.0.2 **"
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $cat setup.py
from setuptools import setup, find_packages
setup(
name="hello_example",
version="0.0.2",
author="Example Author",
author_email="author@example.com",
url="example.com",
description="A hello-world example package",
packages=find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
Package it again - (packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $ python3 setup.py sdist running sdist running egg_info writing hello_example.egg-info/PKG-INFO writing dependency_links to hello_example.egg-info/dependency_links.txt writing top-level names to hello_example.egg-info/top_level.txt reading manifest file 'hello_example.egg-info/SOURCES.txt' writing manifest file 'hello_example.egg-info/SOURCES.txt' running check creating hello_example-0.0.2 creating hello_example-0.0.2/hello_example.egg-info creating hello_example-0.0.2/hello_world copying files to hello_example-0.0.2... copying README -> hello_example-0.0.2 copying setup.py -> hello_example-0.0.2 copying hello_example.egg-info/PKG-INFO -> hello_example-0.0.2/hello_example.egg-info copying hello_example.egg-info/SOURCES.txt -> hello_example-0.0.2/hello_example.egg-info copying hello_example.egg-info/dependency_links.txt -> hello_example-0.0.2/hello_example.egg-info copying hello_example.egg-info/top_level.txt -> hello_example-0.0.2/hello_example.egg-info copying hello_world/__init__.py -> hello_example-0.0.2/hello_world copying hello_world/hello_python.py -> hello_example-0.0.2/hello_world copying hello_world/hello_world.py -> hello_example-0.0.2/hello_world Writing hello_example-0.0.2/setup.cfg Creating tar archive removing 'hello_example-0.0.2' (and everything under it)Install new version -
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $pip3 install dist/hello_example-0.0.2.tar.gz
Processing ./dist/hello_example-0.0.2.tar.gz
Installing collected packages: hello-example
Attempting uninstall: hello-example
Found existing installation: hello-example 0.0.1
Uninstalling hello-example-0.0.1:
Successfully uninstalled hello-example-0.0.1
Running setup.py install for hello-example ... done
Successfully installed hello-example-0.0.2
Verify the installation - (packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $ pip3 list --format=columns Package Version ------------- ------- hello-example 0.0.2 pip 20.0.2 setuptools 41.2.0Test again using the python program -
(packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $python3 example.py Calling Hello Python Function: HELLO PYTHON3 with version **0.0.2** Calling Hello World Function: HELLO WORLD with version ** 0.0.2 ** (packaging) kulsharm2@WKMIN5257929:/tmp/hello_world$ ⚙️ $
