Helping Yocto Project work with Python 3
According to the statistics from StackOverflow Python is the fastest-growing major programming language. First released in 1991, Python is nowadays commonly used for various applications in multiple different industries. Python is a first class citizen of many embedded Linux systems.
The Yocto Project, a collaborative project of the Linux Foundation for creating custom Linux distributions for embedded devices, uses the OpenEmbedded build system and relies on layer meta-python from meta-openembedded to deliver Python 3 packages. Until recently, meta-python was providing both python 2 and python 3 versions of each package. The Python community decided that January 1, 2020, was the day to sunset Python 2. Since then Python 2 has been officially deprecated. This triggered major changes related to the support in Yocto and OpenEmbedded. All recipes for version 2 were moved to layer meta-python2 to provide legacy support after the end of life for this Python release. In meta-openembedded/meta-python, the OpenEmbedded community started efforts to remove all recipes for version 2 as well as to consolidate inc and bb files into a single bb file for version 3.
Konsulko Group engineers are regular contributors to various upstream open source projects, including meta-openembedded and more specifically to meta-python. In the past month, Leon Anavi joined the community efforts for consolidating Python 3 recipes in a single file as well as for upgrading various packages. Nowadays, most of the Python 3 recipes are utilizing the pypi bbclass which takes care for downloading and processing packages from pypi.org. This makes most of the upgrades to new releases of a Python package straight-forward. However, it is important to check the list of build and runtime dependencies as well as to ensure that bitbake still works fine with the upgraded recipe version for both x86-64 and ARM architectures prior to submission.
Let’s have a closer look at the recipe python3-protobuf. It has been recently upgraded from version 3.11.3 to version 3.12.2. Protocol Buffers, also known as protobuf, are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. In the Yocto and OpenEmbedded ecosystem, recipe python3-protobuf depends on recipe protobuf from layer meta-oe. Both meta-oe and meta-python are part of meta-openembedded. So to avoid version mismatch and to ensure that bitbake will be able to successfully build python3-protobuf version 3.12.2 an upgrade of recipe protofobuf to the same version was mandatory. We contributed both upgrades to the master branch of the git repository meta-openembedded. The maintainers took care of cherry-picking them to the dunfell branch which is compatible with the latest stable release of the Yocto Project as of the moment. As a result, if you checkout the latest stable release of Poky, the reference system of the Yocto Project, and meta-openembedded you will be able to quickly build the latest version of protobuf and python3-protobuf out of the box.
Konsulko engineers have been there since the earliest days of the OpenEmbedded build framework and the Yocto Project. We continue to regularly make upstream contributions to these open source projects. Please contact us if you need your “own” Linux distro for your own embedded product.
