We’re heading to Austria for Open Source Summit Europe, with a large contingent of our engineers, as well as four speakers for this always important conference.
September 16: AGL updates to Yocto Project 5.0
Principal Engineer Scott Murray will join the Linux Foundation’s Walt Miner to present “What’s Happening with Automotive Grade Linux and How Our Update to Yocto 5.0 Went.” Walt and Scott will share some lessons learned from upgrading from the 4.0 to 5.0 versions of the Yocto Project and provide an update on the latest AGL features included in the Ricefish release.
September 17: Rewriting zblock in Rust
Vitaly Wool, Principal Engineer and GM of Konsulko AB will discuss “Rusty Swapping: Rewriting a Zswap Backend in Rust.” Rust has gained popularity as the second Linux kernel high-level language. Recently, Vitaly rewrote a zswap backend called zblock in Rust. This talk will cover the main principles of zblock (which stay the same no matter which language is used), the obstacles met while implementing it in Rust, and finally the comparison of the two.
September 17: Best Practices for Scarthgap
Principal Engineer Tim Orling will present “Are You Ready For Scarthgap? Best Practices For The Latest Yocto Project LTS Release.” Tim will discuss setting up your own “distribution” and board-support package (BSP), and share techniques for managing and discovering layers. He’ll discuss best practices to ensure your public layers are ready for the Layer Index, and investigate meta-lts-mixin layers. He’ll look at the latest tools for Software Bill of Materials (SBOM), license compliance (SPDX), and software vulnerabilities (CVEs), and share ways to leverage new IDE tools added in this release.
September 19: A/B Update Solutions with Yocto Project
At Yocto Project Developer Day 2024 (co-located with OSEE) Senior Engineer Leon Anavi will provide a “Side-by-side Comparison of Dual A/B Update Solutions with the Yocto Project.” Leon’s talk will provide a detailed exploration of Mender, RAUC, and swupdate, comparing them on the same hardware platforms. He’ll discuss their advantages and disadvantages and how to select the most appropriate open-source solution for specific projects. Leon will delve into various use cases and practical examples, concluding with a side-by-side comparison.
We hope you’ll be able to join us in Vienna.
https://www.konsulko.com/wp-content/uploads/2024/08/vienna-2-graphic.png900900Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2024-09-02 15:21:352024-09-02 15:21:37Konsulko to speak at multiple OSSE events, September 16-19
Software over-the-air (OTA) updates are essential for any modern embedded Linux device. In part 1, we explored A/B software updates using RAUC and qbee.io. For our demonstrations, we utilized the new open source hardware Olimex’s iMX8MPlus System on Module (SOM) and Evaluation Board (EVB). The NXP i.MX8MP is a robust microprocessor, ideal for industrial-grade applications and widely adopted across various industries. In part 2, we will use Mender to update the same hardware.
About Mender
Mender.io is an open-source platform designed for managing and deploying over-the-air (OTA) updates to embedded devices. It provides a reliable and secure method to keep devices up-to-date, minimizing downtime and reducing the risks of manual updates. Mender supports an open-source A/B update scheme and offers an optional proprietary implementation for delta updates.
As a turnkey solution, Mender features a web interface for comprehensive device management and is available as a Software-as-a-Service (SaaS) for small and medium businesses, as well as hosted or on-premise solutions for large enterprises. It supports robust update strategies, rollback mechanisms, and add-ons for configuring, monitoring, and troubleshooting devices. Mender is a state-of-the-art solution for maintaining and managing fleets of connected devices across various industries.
Building an Image with Mender
Here are the steps to build core-image-base with Mender for Olimex:
Install the kas tool (optional: to install it globally for all users, run the installation as root or using sudo):
pip install kas
Clone meta-mender-community git repository for Yocto LTS release 5.0 (scarthgap):
Run the following command to start the build process:
kas build ../kas/olimex-imx8mp-evb.yml:debug-image.yml
Initiating the build process from scratch is a bit of a marathon, as kas and bitbake need to download all the source code and execute a plethora of tasks. Feel free to grab a cup of tea (or maybe a whole teapot) while you wait!
Flashing the Mender Image
Using kas and BitBake will result in the production of an image file. After the build process is complete, you will find the generated image at the following relative path: build/tmp/deploy/images/olimex-imx8mp-evb/core-image-base-olimex-imx8mp-evb.sdimg. This path indicates the location within the build directory where the final image is stored, ready for deployment to Olimex iMX8MP-SOM-4GB-IND and iMX8MP-SOM-EVB-IND.
The core-image-base-olimex-imx8mp-evb.sdimg file must be flashed onto a microSD card to be used with your device. This can be accomplished using various applications such as dd or bmaptool for command-line options. If you prefer a user-friendly application with a graphical interface, you can use Balena Etcher, which simplifies the flashing process and provides a straightforward GUI.
To set up and verify your Olimex iMX8MP-SOM-EVB-IND board, follow these steps:
Connect the USB-to-UART adapter to the A53_DBG1 connector on the Olimex iMX8MP-SOM-EVB-IND, and insert the Ethernet cable and the microSD card.
Plug the 5V power supply into the power jack of the iMX8MP-SOM-EVB-IND to power up the board.
Ensure the system boots successfully, then log in as the root user (no password required) and check the Mender status by executing the following commands:
A Mender artifact refers to a package format used by the Mender update manager for over-the-air (OTA) software updates. It contains all the necessary components such as firmware, scripts, configuration files, and metadata required to update a device’s software reliably and securely. The layer meta-mender/meta-mender-core is essential for integrating Mender’s functionality into the Yocto Project and OpenEmbedded build system as it provides required classes and scripts that automate the creation of Mender artifacts as part of the build process.
Follow the steps below to build a Mender Artifact for iMX8MP-SOM-EVB-IND that extends the system with the addition of the simple text editor nano:
Build both core-image-base and its corresponding Mender Artifact using:
kas build ../kas/olimex-imx8mp-evb.yml:debug-image.yml:update-image.yml
This process will generate a Mender Artifact containing nano at the file path build/tmp/deploy/images/olimex-imx8mp-evb/core-image-base-olimex-imx8mp-evb.mender.
Installing the Mender Artifact
Mender serves as a comprehensive update solution, featuring a central server that acts as a hub for storing and orchestrating software updates across fleets of devices via over-the-air deployment. Using Mender’s intuitive web UI or REST APIs, you can efficiently manage devices, upload software releases, and execute seamless deployments to distribute updates across your devices. Alternatively, Mender can operate in standalone mode, independent of a server.
To perform a manual standalone deployment using Mender in the terminal, follow the steps below. In this setup, no Mender Server is involved, and updates are initiated directly on the device.
Start a simple HTTP server in the directory with the Mender Artifact:
NOTE: Replace <server> with the IP address of the machine on which the Python3 HTTP server is running.
Reboot the embedded Linux device:
reboot
Login as root on the board and verify that nano text editor has been installed.
Ensure the new deployment becomes permanent:
mender-update commit
This example illustrates the seamless integration of Mender using the Yocto Project release Scarthgap on an embedded computer powered by the NXP i.MX8MP SoC. It demonstrates how you can effectively manage updates across fleets of devices using the Mender server. Furthermore, Mender also provides additional tools for remote troubleshooting, ensuring smooth operations in the field.
How do Mender.io and RAUC differ?
The Mender client is an application that runs on embedded devices. In a production setup, it connects to the Mender server to perform automatic updates by downloading and installing Mender Artifacts as they become available. Initially, the Mender client was developed in Go. However, a strategic decision was made to rewrite it in C++ to reduce the application’s footprint and support more platforms, including real-time operating systems (RTOS).
In comparison, RAUC, the alternative A/B open-source solution explored in part one, also has an application running on the embedded Linux device, but it is written in C. Unlike Mender, RAUC does not provide a server to manage devices, so a third-party solution such as qbee.io or Eclipse hawkBit is required.
Another notable technical difference is that RAUC integration through meta-rauc-nxp relies on a wks file where the A and B partitions are explicitly specified. In contrast, meta-mender-nxp uses classes and special variables provided by Mender to define those partitions, and BitBake generates a temporary wks file while building Mender-enabled images.
In terms of security, both Mender and RAUC support signing and verification of updates. Mender supports signing artifacts using RSA with a recommended key length of at least 3072 bits or ECDSA with curve P-256. RAUC employs X.509 cryptography for signing and verifying update bundles.
Leveraging the insights and experiences discussed in parts 1 and 2 of this article, here is a side-by-side comparison of the key features of Mender and RAUC:
Feature
Mender
RAUC
A/B updates
Yes
Yes
Roll-back
Yes
Yes
Configure add-on
Available
No
Monitor add-on
Available
No
Troubleshoot add-on
Available
No
Client implementation
C++
C
Client license
Apache 2.0
LGPL-2.1
Yocto Project integration
Yes
Yes
Management server
Yes
3rd-party
Mender.io provides a comprehensive, turnkey solution that covers everything from embedded devices to cloud-based software-as-a-service for managing fleets of connected devices. It also offers convenient add-ons and proprietary delta updates. Meanwhile, RAUC reliably integrates seamlessly with both in-house and third-party device management systems. The choice of update technology should be based on your specific requirements and use cases.
About Konsulko Group
Over the years, Konsulko engineers have made significant contributions to the community and crucial embedded Linux open-source projects, including the Yocto Project, OpenEmbedded, the Linux kernel, and U-Boot. We specialize in assisting customers in developing commercial products leveraging these technologies. With expertise in BSP bring-up on diverse hardware platforms for embedded devices, our services encompass a wide range of open-source solutions for software updates. Contact us to discuss the best update strategy and technology for your embedded product requirements.
https://www.konsulko.com/wp-content/uploads/2024/07/imx8MP-mender-graphic.png900900Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2024-07-30 09:58:172024-07-30 09:58:18Software Updates on i.MX8MP, Part 2: Mender & Yocto Project
Software over-the-air (OTA) updates are essential for embedded Linux devices as they ensure timely application of security patches, fix vulnerabilities, and enhance security features to protect against threats. They offer a convenient way to deliver new features, performance improvements, and bug fixes without needing physical access to the device, keeping it up-to-date and functional.
Several high-quality open-source OTA solutions are available today, allowing developers to leverage and customize existing systems rather than creating proprietary ones, saving both time and money. In this series of articles, we will explore and compare software update strategies using the A/B scheme, where two identical copies of the root filesystem are maintained, one active and one for the next update. Popular OTA update solutions like Mender, RAUC, and Swupdate implement this approach. This article will focus on RAUC using a typical development setup. In part 2 we will implement software updates with Mender. For our demonstration, we will use the i.MX8MP, a versatile microprocessor from NXP Semiconductors known for its industrial-grade reliability and popularity in smart home devices, industrial automation, medical equipment, and multimedia systems.
Qbee.io is a comprehensive cloud platform for managing and maintaining IoT and edge devices. Using the qbee-agent running on the embedded devices, it offers features such as configuration management, remote accesss, monitorning, security and OTA software updates based on RAUC. These capabilities enable businesses to efficiently oversee their distributed technology infrastructure, ensuring devices remain up-to-date, secure, and perform optimally. Earlier in 2024 Tim Orling, Konsulko Group Principal Software Engineer, implemented image update with qbee and RAUC on Raspberry Pi 5.
This technical article will guide you through the exacts steps to build a basic image for Olimex iMX8MP-SOM-4GB-IND and iMX8MP-SOM-EVB-IND using Yocto Project release 5.0 LTS (scarthgap) as well as to perform a software update using qbee and RAUC.
Building an Image
Download the long term support (LTS) release Scarthgap reference Yocto distribution, Poky:
git clone -b scarthgap https://git.yoctoproject.org/poky poky-olimex-imx8mp
cd poky-olimex-imx8mp
Click on your profile name at the top right corner and select Bootstrap keys.
Copy the key.
Replace <bootstrap_key> with the qbee bootstrap key and append to conf/local.conf:
QBEE_BOOTSTRAP_KEY = "<bootstrap_key>"
Build an image:
bitbake core-image-base
Creating an image from the ground up is a time-consuming process that requires numerous Yocto/OpenEmbedded recipes and configurations. Please be patient as bitbake systematically manages each step.
Flash tmp/deploy/images/olimex-imx8mp-evb/core-image-base-olimex-imx8mp-evb.rootfs.wic.gz to microSD card.
Attach the USB-to-UART adapter to connector A53_DBG1 Olimex iMX8MP-SOM-EVB-IND, plug the ethernet cable and the microSD card.
Plug 5V power supply to the power jack on iMX8MP-SOM-EVB-IND to turn on the board.
Verify that the system boots successfully, log in as user root without a password and check RAUC status:
Visit qbee.io, click Devices and verify that olimex-imx8mp-evb has successfully connected:
Creating a RAUC Update Bundle
A RAUC update bundle comprises the file system image(s) or archive(s) designated for system installation, accompanied by a manifest detailing the images for installation, encompassing options and meta-information. Additionally, it may include scripts designated for execution before, during or after the installation process. To sign and verify the update bundles RAUC uses SSL keys. Layer meta-rauc-beaglebone contains a keyring containing all keys and a recipe for a simple RAUC update bundle for demonstration purposes only.
Follow the steps below to create RAUC update bundle that extends the system by adding the popular text based editor nano:
Add to conf/local.conf:
IMAGE_INSTALL:append = " nano"
Build the RAUC update bundle:
bitbake update-bundle
Running RAUC Update from qbee.io
Please follow the steps below to upload the RAUC bundle to qbee.io and update the board:
Visit qbee.io, click File Manager and upload the RAUC bundle:
Select Devices, click on olimex-imx8mp-evb and go to tab Configure. From Settings > OTA enable RAUC image updates:
Select the RAUC bundle.
Click Save Changes, then click Commit Changes and enter a commit message:
Wait for the qbee agent to apply the RAUC update bundle. By default, the agent checks for changes every 5 minutes. To force an immediate check, click the Run agent button in the Device Overview. Allow a few minutes for the update bundle to be transferred and installed on the board.
After the update is finished, the board will automatically restart. You can verify that the active RAUC rootfs slot has been updated and nano is present:
To optionally verify the update of the embedded Linux device from the cloud service, select Devices, click on olimex-imx8mp-evb, navigate to the Logs tab, and review the logs related to the RAUC update.
With qbee.io, multiple IoT devices can be grouped together and managed as a fleet from the cloud service. In practical product development scenarios, enhancing the Yocto Project and OpenEmbedded workflow can be achieved through a few straightforward commands to streamline continuous integration (CI).
The second part of the article will detail the exact steps to build, boot, and update an image on the Olimex iMX8MP-SOM-4GB-IND and iMX8MP-SOM-EVB-IND hardware, this time using Mender instead of RAUC, with the Yocto Project release 5.0 LTS (scarthgap).
Konsulko engineers have played pivotal roles as contributors and mentors in the commercial product space from the early days of OpenEmbedded and the Yocto Project. Our team excels in utilizing RAUC, Mender, Swupdate and a range of other open-source tools to deliver comprehensive software update solutions. Contact us to discuss your specific product requirements and discover how Konsulko engineers can improve your embedded Linux development projects.
https://www.konsulko.com/wp-content/uploads/2024/07/imx8MP-qbee-yocto.png900900Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2024-07-22 11:42:102024-07-22 12:54:40Software Updates on the i.MX8MP, Part 1: qbee and RAUC
TuxCon is an annual open-source conference held in Plovdiv, Bulgaria, aimed to promote the adoption of open-source hardware and software, and organized by passionate volunteers. Since the conference’s inception in 2014, Konsulko Group engineers have participated and supported this important local event.
Konsulko Group junior engineer Atanas Bunchev spoke at the event, sharing his experience with Balena on Raspberry Pi and PHYTEC phyBOARD-AM62x. Balena.io, commonly known as Balena, is a platform that simplifies the deployment and management of embedded Linux devices using images based on the Yocto Project and OpenEmbedded. It leverages Docker containers for streamlined application deployment across various Internet of Things (IoT), offers centralized device management for monitoring and updates, and supports scalability from prototypes to large-scale projects. Atanas further elaborated on the utilization of Balena AutoKit for conducting automated QA testing on embedded Linux devices.
The second talk about the Yocto Project at TuxCon 2024 was delivered by Sadika Hasan, a senior-year student from “Paisii Hilendarski” University of Plovdiv. She demonstrated how to create a custom Linux distribution and Software Development Kit (SDK) by extending Poky, the Yocto Project’s reference distribution. Additionally, she covered the integration of the custom SDK with Visual Studio Code and the remote debugging of a C++ application on a Raspberry Pi Zero W 2.
Professor Tokmakov, with the assistance of engineer Nikolay Nedelev from Romit LTD, is actively involved in the ECOVEM project (European Center of Vocational Excellence in Microelectronics). As part of their efforts, two students from PU developed diploma theses for their Bachelor of Science degrees utilizing the Yocto Project. These theses were based on real-life problem solving and the practical experience of the Konsulko Group with industrial embedded Linux devices.
From the early days of OpenEmbedded and the Yocto Project, Konsulko engineers have actively contributed upstream and engaged with the community, offering their expertise in developing high-quality commercial products. If you are developing a new product, get in touch to see how Konsulko’s engineering expertise can enhance your project. Additionally, if you are a Linux software developer passionate about open source, we invite you to explore potential opportunities to join the Konsulko team.
https://www.konsulko.com/wp-content/uploads/2024/05/tuxcon-2024-graphic.png900900Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2024-05-28 11:28:342024-05-28 11:28:36Two technical talks about the Yocto Project at TuxCon 2024
Mender is an open-source over-the-air (OTA) software update technology for embedded Linux devices and Internet of Things (IoT) ecosystem. It allows developers to remotely manage and update software on embedded Linux devices, ensuring that they remain secure, up-to-date, and functional throughout their lifecycle. Mender simplifies the process of launching a new project by offering official and community-supported board integrations for a wide range of devices and operating systems. These integrations include support for Debian family and the Yocto Project, making it easier for developers to get started with their projects seamlessly.
The Yocto Project is an open-source collaboration project by the Linux Foundation to create custom Linux-based systems for embedded devices. It uses the OpenEmbedded build automation framework with the build tool bitbake and provides Poky as a reference Linux distribution. The Yocto Project follows a regular release cycle, typically with a new version every six months, alongside long-term supported releases available every two years. Currently, the most recent LTS release is version 5.0, codenamed Scarthgap, which became available on April 30, 2024.
Raspberry Pi 5 introduces significant hardware differences compared to its predecessors. To enable Mender updates on this platform, we require U-Boot version v2024.04 or later. Unfortunately, the Yocto Project release Scarphgap ships with U-Boot version v2024.01, which isn’t compatible. To address this compatibility issue and obtain the necessary U-Boot version, our setup utilizes the scarthgap/u-boot branch from the meta-lts-mixins layer. Tim Orling, Principal Software Engineer at Konsulko Group, contributed the patches in meta-lts-mixins for U-Boot v2024.04.
Raspberry Pi 5 is the first model of the famous single board computers that features a dedicated UART connector is a three-pin header compatible with the Raspberry Pi Debug Connector specification. It can be used with Raspberry Pi Debug Probe, a USB device that provides both a UART serial port and a standard Arm Serial Wire Debug (SWD) interface.
The article offers a practical guide, outlining the exact steps to build a Mender-enabled image for Raspberry Pi 5 and execute an A/B update. To streamline the build setup, we’ll utilize KAS. This Python-based open-source tool effectively handles various Yocto/OpenEmbedded layers. KAS executes builds within a Docker container to ensure consistency and reliable build outcomes, regardless of the primary GNU/Linux distribution on the build machine.
Image for Raspberry Pi 5
Follow the steps below to build core-image-minimal with Mender for Raspberry Pi 5:
Install the kas tool (optionally, you can install globally for all users. Run as root, respectively under sudo then):
mkdir -p meta-mender-community/mender-rpi5 && cd meta-mender-community/mender-rpi5
Use kas to build for the Raspberry Pi 5:
kas build ../kas/raspberrypi5.yml
Flash tmp/deploy/images/raspberrypi5/core-image-minimal-raspberrypi5.sdimg to a microSD card and boot it on Raspberry Pi 5.
Mender Artifact for Raspberry Pi 5
Follow the steps below to build a Mender Artifact for Raspberry Pi 5 that provides the simple text editor nano:
Enter KAS shell:
kas shell ../kas/raspberrypi5.yml
Append to the end of conf/local.conf:
IMAGE_INSTALL:append = " nano"
Build both core-image-minimal and a Mender Artifact for it:
bitbake core-image-minimal
As a result Mender Artifact containg nano will be generated as file tmp/deploy/images/raspberrypi5/core-image-minimal-raspberrypi5.mender
Update Raspberry Pi 5
As an end to end update solution, Mender provides aserver as the central hub for storing and orchestrating software updates across fleets of devices through over-the-air deployment. Through Mender’s user-friendly web UI or REST APIs, you can easily oversee device management, upload software releases, and create seamless deployments to distribute updates to your devices. However, it is also possible to use Mender in standalone mode without a server.
Follow the steps below to manually perform a standalone deployment with Mender in the terminal. In this scenario, no Mender Server is utilized, and the deployments are triggered directly at the device.
Start a simple HTTP server in the directory with the Mender Artifact:
python3 -m http.server
Login as root on Raspberry Pi 5 and install the Mender Artifact to perform an upgrade of the device:
NOTE: Replace <server> with the IP address of the machine on which the Python3 HTTP server is running.
Reboot Raspberry Pi 5:
reboot
Login as root on Raspberry Pi 5 and verify that nano text editor has been installed.
Make the deployment permanent:
mender-update commit
This straightforward example showcases the seamless integration of Mender with Raspberry Pi 5 using the Yocto Project release Scarthgap. You can follow up and manage updates of fleets of Raspberry Pi 5 devices through the Mender server. Additionally, Mender offers convenient add-ons for remote troubleshooting, ensuring smooth operations in the field.
Since the earliest days of the OpenEmbedded build framework and the Yocto Project, Konsulko engineers have been active contributors to the community, aiding customers in crafting commercial products using these technologies. Our expertise extends beyond Mender, encompassing various open-source solutions for software updates. Feel free to get in touch to explore how we can assist with your embedded product requirements.
https://www.konsulko.com/wp-content/uploads/2024/05/Raspberry-Pi-5-porting.png900900Leon Anavihttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngLeon Anavi2024-05-17 08:05:122024-05-17 08:13:31Porting Mender to Raspberry Pi 5 and Yocto Project Scarthgap
Balena.io, commonly known as Balena, is a platform simplifying IoT device deployment and management. It uses Docker containers for easy application deployment across various embedded Linux devices, offers centralized device management for monitoring and updates, and supports scalability from prototypes to large-scale projects. With over-the-air (OTA) updates and monitoring tools, Balena streamlines IoT application development and management. Balena supports over a 100 device types with robust and resilient remote updates, combined with a powerful toolset for monitoring, maintaining and debugging.
Recently PHYTEC took interest into having their AM62x-based development board added to the list of Balena-supported devices and Konsulko was on the task. The process of bringing Balena support for a new board consists of creating a custom Yocto-based Board Support Repository and having it pass an automated testing procedure run on the actual hardware.
At the heart of the phyBOARD-AM62x is the industrial PHYTEC phyCORE-AM62x. This versatile System-on-Module (SoM) is powered by Texas Instruments AM62x Sitara processor and is equipped with Ethernet, CAN, UART, I2C, SPI, dual display, MIPI CSI-2 camera and audio. The phyBOARD-AM62x offers a modern FTDI interface allowing software download and debugging, perfect for development. Paired alongside the M.2 connectorized WiFi and Bluetooth module extensions makes a perfect product for IoT devices.
Konsulko engineers created balena-phytec git repository which is currently publicly available at the BalenaOS GitHub organization. It is based on the Yocto/OpenEmbedded board support package (BSP) layers meta-ti and meta-phytec. The repository provides all necessary modifications to run BalenaOS (Balena’s specific Linux distribution) on the phyBOARD-AM62x, including a new Yocto/OpenEmbedded integration layer meta-balena-phytec that extends the recipes for the U-Boot bootloader and Linux kernel.
Texas Instruments AM62x SoCs come with a RTI/WWDT Windowed Watchdog Timer which would turn off the board when not serviced within a specific time interval. RTI only supports a windowed mode, where the watchdog can only be petted during the open window; not too early or not too late. However, due to the nature of systemd utilized by BalenaOS, it is unable to ping the watchdog within the designated open window period, as it attempts to ping at watchdog_timeout/2 ticks. Therefore the systemd watchdog has been disabled by a BitBake append file for systemd in layer meta-balena-phytec.
To finish the task Konsulko and Balena engineers ran a special test suite on the board using the Balena AutoKit. Shortened from Automation Kit, the AutoKit is a complete hardware solution for automated interaction with embedded Linux device. It features SD card multiplexing, Ethernet and power control, serial communication, HDMI capture and support for other USB peripherals.
The rest of the article covers a step-by-step guide for connecting the phyBOARD-AM62x to BalenaCloud and deploying a simple containerized application on it, as well as demonstration of some of the basic features provided by BalenaCloud’s Dashboard.
Downloading an Image
The first thing you have to do is to register at https://dashboard.balena-cloud.com/signup. Every user is allowed to have up to 10 devices with all features enabled for free. While the software that Balena develops is open source, the usage of their cloud infrastructure with more than 10 connected IoT devices requires a plan subscription, with better plans coming with more devices and better support response time.
Once you register and confirm your e-mail you’ll be welcomed with a screen providing an introduction to Balena and inviting you to create a fleet.
A fleet is a group of devices with identical configuration and with the same applications deployed on them. Create a new fleet and set the device type to phyBOARD-AM62x.
Once your fleet is created click the “Add device” button. Select the Development edition, as it includes a few quality of life modifications for developers, such as passwordless ssh access as root on port 22222.
Once you have picked the configuration for the new image you can download it by selecting the Download balenaOS option from the menu next to the Flash button and pressing on that button after that.
Alternatively if you have balenaEtcher installed you can directly click on the Flash button. That way balenaEtcher will start up with the correct URL for the image already selected, leaving only the output device selection to you.
The third option – downloading a configuration file – is applicable when having a custom BalenaOS image that you want to configure for a given fleet.
Flashing the Installation media
Once the image is downloaded you need to flash it on a microSD card. There are several ways of flashing an image. One way is to use a graphical tool for the task, such as balenaEtcher or Rufus(keep in mind that some of these will expect you to unzip the image in advance).
Another way to do is to use the Linux shell. Replace <your_image>.img.zip with the compressed image you downloaded and <your_image>.img with the extracted file. Replace /dev/sdX with the path to the microSD card.
Keep in mind that flashing an image on a microSD card will delete all data stored on the microSD card.
Installing Balena on the phyBOARD-AM62x
Note: The following procedure will delete all data on the internal eMMC storage of your phyBOARD-AM62x.
Plug in the newly-flashed microSD card into the phyBOARD-AM62x board. Make sure the boot switches are set to SD card mode. Power on the board and wait for the installation to the internal eMMC to finish.
Once the installation is complete the board will turn off. This is indicated by LEDs D11 and D12 turning off.
As the installation on the internal eMMC has finished, set the boot mode to eMMC. Disconnect the microSD card, connect phyBOARD-AM62x to Ethernet and power it on.
In few moments the board will show up on the fleet page at the Dashboard.
Clicking on phyBOARD-AM62x in the table will open a dedicated page for the board that shows details and current status information.
On the left you can see the fleet containing the phyBOARD-AM62x, its online status, its unique UUID, the Host OS and Balena Supervisor versions, the local and public IP address of the board. You can enable or disable the Public Device URL of the board, as well as use the Actions menu to manage the board.
The Identify action will make a specific LED on phyBOARD-AM62x blink for half a minute to make identifying the exact board easier – a very useful feature when having a bunch of similar boards lying around. On phyBOARD-AM62x this is User LED 1, a bright red LED at the left edge of the carrier board.
At the upper-right corner you can see the current resource usage on the board, as well as the temperature and free persistent storage space.
Below them are the Logs and Terminal sections, which can be used for advanced remote debugging and troubleshooting of the board.
Lastly, at the currently empty space at the bottom-left part of the screen we’ll be able to see the status of the containerized applications currently deployed on the board.
Pushing a hello-world application to phyBOARD-AM62x
The last section of this article contains a quick example of how to push a containerized application to a board that runs Balena. The example application that will be deployed is a Docker container consisting of a simple web page hosted with NodeJS Express.
DownloadBalena CLI tool, a Command Line Interface for balenaCloud or openBalena. Select the standalone binary for your workstation – the machine you’re going to use to deploy the application, not the board you’re going to deploy to.
As of the time of writing, the most recent version is balena-cli-v18.1.5-linux-x64-standalone.zip. If you are utilizing a newer release or a different operating system, kindly substitute the filename accordingly.
unzip balena-cli-v18.1.5-linux-x64-standalone.zip
cd balena-cli
You need to login in your profile from the CLI before you can push anything to your devices. For that task you can use the ./balena login command:
./balena login
_ _
| |__ __ _ | | ____ _ __ __ _
| '_ \ / _` || | / __ \| '_ \ / _` |
| |_) | (_) || || ___/| | | || (_) |
|_.__/ \__,_||_| \____/|_| |_| \__,_|
Logging in to balena-cloud.com
? How would you like to login? (Use arrow keys)
❯ Web authorization (recommended)
Credentials
Authentication token
I don't have a balena account!
Press Enter to select Web authorization and confirm the authentication request on the web page that opens:
Alternatively, select the Credentials option and insert your username and password.
git clone https://github.com/balena-io-examples/balena-nodejs-hello-world.git
cd balena-nodejs-hello-world
../balena push <fleet name>
Once the application is uploaded you can see it on the board’s page at the web dashboard.
The Hello World example is a web server hosting a single web page. To check it out, you can either visit the board’s local IP address if you’re in the same local network as phyBOARD-AM62x, or enable the PUBLIC DEVICE URL and open the hyperlink that appears next to the switch once enabled.
This article demonstrates how to setup and use the PHYTEC phyBOARD-AM62x development board with Balena, a container-based distribution designed for easy and rapid development of embedded applications, packed with useful device management and troubleshooting features as well as sophisticated monitoring and automation-oriented ones. Get in touch with us to discuss additional features. Our team is ready to help in development of new or already existing Linux embedded projects.
From the earliest days of OpenEmbedded and the Yocto Project, Konsulko engineers have actively contributed upstream and participated in the community, offering their expertise and guidance in developing high-quality commercial products. Our proficiency extends to Balena, RAUC, Mender, and other open-source solutions, ensuring seamless and reliable over-the-air updates. Contact us to explore how Konsulko engineers can contribute to the advancement of your embedded Linux product development.
___
https://www.konsulko.com/wp-content/uploads/2024/04/balenagraphic.png900900Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2024-04-06 08:13:322024-04-09 07:32:25Balena: Running Containerized Applications on phyBOARD-AM62x
Integrity Measurement Architecture (IMA-measurement) is a subsystem in the Linux kernel designed to provide a framework for maintaining the integrity of files on a system. The primary purpose of IMA is to ensure that only trusted code and data are executed on a system and that the integrity of critical system components is maintained.
IMA was merged into the mainline Linux kernel in 2004 with the release of version 2.6.30. It evolved over time, adding features such as TPM integration, extended support, and continued maintenance, becoming a key component for ensuring the integrity of files in Linux-based systems. IMA works by calculating cryptographic hashes of files at various points in their lifecycle, such as when they are accessed, executed, or modified.
This article shares the exacts steps to build a minimal Linux distribution with IMA support for QEMU x86-64 using the Yocto Project and OpenEmbedded. The Yocto Project is an open-source collaboration project that enables developers to create lightweight, optimized, and customizable Linux distributions for embedded devices while maintaining control over the software components and configurations included in the system. To enable IMA, we use Yocto/OpenEmbedded layers meta-security and meta-integrity. These layers offer a comprehensive suite of security tools and hardening utilities designed for Linux kernels, along with libraries that facilitate the implementation of robust security mechanisms.
The following steps outline the process of building an image with Integrity Measurement Architecture (IMA) using the Yocto Project and OpenEmbedded. This demonstration uses the default debug keys provided in the “data” directory of layer meta-integrity. Because everyone has access to these private keys, for devices in production you must create your own private keys and use them instead. Enabling the audit kernel subsystem provides additional informational integrity auditing messages useful for debugging any appraisal issues that may arise during the development process.
Kindly be aware that this article utilizes source code derived from the primary branches of associated Yocto/OE layers. Consequently, we are selecting specific git commits that have been confirmed to function reliably. These commits will be part of the next long-term support (LTS) release of the Yocto Project which is version 5.0 with codename Scarthgap. It is scheduled for release in April 2024 and will be supported for 4 years until April 2028.
Download the source code:
git clone git://git.yoctoproject.org/poky poky-qemu
cd poky-qemu
git checkout e31be0b0e6ed6855787ebfbacc15bdbf1b9e511c
git clone git://git.yoctoproject.org/meta-security
cd meta-security
git checkout 30e755c59204cbd64c3aa12e64ab33041f6f02c0q
git clone git://git.openembedded.org/meta-openembedded
cd meta-openembedded
git checkout 57db42741df341718b9bb4b1b8f20d2e2fcf7010
After building the image, we can launch it. QEMU, short for Quick Emulator, is an open-source virtualization software that allows users to emulate various hardware platforms and run operating systems on different host systems. We will use it to run and test the image. By utilizing the “nographic” option, QEMU disables the video console, setting the console to “ttys0”. This feature is particularly beneficial when remotely accessing a build server over SSH. To verify the effectiveness of the appraisal process, attempt modifying a file, then confirm that the verification of the altered file subsequently fails.
Launch the image in QEMU:
runqemu nographic
Login and root and verify the integrity of file /etc/hostname using evmctl:
This simple example serves as a demonstration of how Linux IMA operates, using QEMU as a platform. However, to implement Linux IMA on real-world devices, Konsulko Group offers assistance with hardware bring-up and integration of the suitable Yocto/OE BSP (Board Support Package) layers.
Since the inception of OpenEmbedded and the Yocto Project, Konsulko engineers have actively contributed to the community and provided guidance for developing commercial products. We specialize in U-Boot, Linux kernel, RAUC, Mender, and various other open source projects for embedded Linux devices. Contact us to explore how Konsulko engineers can assist with your embedded product development endeavors.
https://www.konsulko.com/wp-content/uploads/2024/02/IMA-Linux.png600600Leon Anavihttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngLeon Anavi2024-02-29 08:07:502024-02-29 08:07:52IMA-measurement with Yocto Project and OpenEmbedded
Konsulko Group has made many upstream contributions to OTA (over-the-air) update solutions for embedded Linux devices. RAUC is a popular open source option as it has been meticulously developed with a keen emphasis on stability, security, and adaptability. Notably, RAUC seamlessly integrates with major build systems such as Yocto Project/OpenEmbedded, Buildroot, and PTXdist.
Functioning across diverse usage scenarios, one of RAUC’s elementary yet impactful functionalities is the A/B update mechanism. In this setup, two identical root filesystems, denoted as A and B, are maintained. The device boots from one of these, while the other serves as the target for updates.
Post-update completion, the bootloader directs the system to boot from the freshly updated partition during the subsequent system startup. RAUC incorporates the ‘verity’ update bundle format. It extends the capabilities of RAUC by introducing built-in support for HTTP(S) network streaming, adaptive delta-like updates, and comprehensive update bundle encryption.
This article provides, as an example, the exact steps how to integrate RAUC with Yocto Project and OpenEmbedded for booting from a microSD card on BeagleBone Black.
Released in 2013, BeagleBone Black is a single-board computer (SBC) developed by the BeagleBoard.org Foundation. It was certified by the Open Source Hardware Association with OSHWA UID US000236. The chipset on BeagleBone Black is Texas Instruments Sitara AM3358 with 1GHz ARM Cortex-A8 CPU and SGX 3D graphics engine. Because of this the demonstrated integration is a suitable reference for other embedded devices equipped Texas Instruments chipsets.
Required Hardware
The hardware used for this step by step tutorial is:
RAUC, a robust and powerful open-source solution, demands advanced skills for initial integration. In general, to incorporate RAUC in a Yocto Project and OpenEmbedded image for BeagleBone Black the following actions have to be performed:
Use U-Boot as a bootloader
Enable SquashFS in the Linux kernel configuration
Use ext4 root file system
Design specific storage partitioning for the certain use case and configure RAUC accordingly
Provide a custom U-Boot script to properly switch between RAUC slots
Prepare a certificate and keyring to use for signing and verifying RAUC update bundles.
Leon Anavi has already done all these actions for core-image-minimal in Yocto/OpenEmbedded layer meta-rauc-community/meta-rauc-beaglebone. The layer is available at GitHub. Please follow the steps below to build core-image-minimal for BeagleBone Black with it:
Download the long term support (LTS) release Kirkstone reference Yocto distribution, Poky:
git clone -b kirkstone https://git.yoctoproject.org/poky poky-rauc-bbb
cd poky-rauc-bbb
Adjust conf/local.conf by appending the following configurations to the end of the file:
MACHINE = "beaglebone-yocto"
# Use systemd as init manager
INIT_MANAGER = "systemd"
# Add RAUC to the image
IMAGE_INSTALL:append = " rauc"
DISTRO_FEATURES:append = " rauc"
Build a minimal bootable image:
bitbake core-image-minimal
The image creation process from scratch is time-consuming, encompassing various Yocto/OpenEmbedded recipes and configurations. Kindly await completion as bitbake diligently executes each tasks.
Flash tmp/deploy/images/beaglebone-yocto/core-image-minimal-beaglebone-yocto.wic.xz to microSD card.
Attach the USB-to-UART adapter to BeagleBone Black, plug the ethernet cable and the microSD card.
Press and hold button S2 while plugging in the 5V DC power supply to turn on BeagleBone Black and boot from microSD card.
BeagleBone black board has an onboard button labeled as S2. It is situated near the microSD card slot. Press and hold it while powering the board to boot from microSD card.
Verify that the system boots successfully, log in as user root without a password and check RAUC status:
rauc status
On the screenshot BeagleBone Black has been booted from RAUC slot rootfs.0 (A) on the microSD card.
NOTE: The meta-rauc-beaglebone layer includes a core-image-minimal.bbappend file, housing essential configurations for RAUC functionality. Apply these configurations similarly to other images intended for use in your embedded Linux device.
Creating a RAUC Update Bundle
A RAUC update bundle comprises the file system image(s) or archive(s) designated for system installation, accompanied by a manifest detailing the images for installation, encompassing options and meta-information. Additionally, it may include scripts designated for execution before, during or after the installation process. To sign and verify the update bundles RAUC uses SSL keys. Layer meta-rauc-beaglebone contains a keyring containing all keys and a recipe for a simple RAUC update bundle for demonstration purposes only.
Follow the steps below to create RAUC update bundle that extends the system by adding the popular text based editor nano:
Add to conf/local.conf:
IMAGE_INSTALL:append = " nano"
Build the RAUC update bundle:
bitbake update-bundle
Following a successful execution, bitbake will produce the update-bundle-beaglebone-yocto.raucb file.
Updating BeagleBone Black with RAUC
Follow the steps below to update core-image-minimal running from a microSD card on BeagleBone Black:
Start a Python 3 web server on the build machine or another computer where the RAUC update bundle (aka update-bundle-beaglebone-yocto.raucb) is available and within the same network as BeagleBone Black, for example:
cd tmp/deploy/images/beaglebone-yocto/
pip3 install --user rangehttpserver
python3 -m RangeHTTPServer
On BeagleBone Black replace <IP> with the actual IP address of the computer on which the HTTP server is running and execute the following command to install the update:
The screenshot show successful installation of the RAUC updated bundle on BeagleBone Black.
Reboot BeagleBone Black to load the updated version:
reboot
NOTE: As alternative, instead of using an HTTP server, you can transfer the update bundle to BeagleBone Black and install it from local storage.
Verify that nano was added to the system:
which nano
Check RAUC status to confirm the system have booted from the second partition:
rauc status
On the screenshot, after sucessful installation of the RAUC update bundle, BeagleBone Black has been booted from RAUC slot rootfs.1 (B) on the microSD card. This slot contains nano.
In real-world product development, the Yocto Project and OpenEmbedded workflow can be enhanced with a few commands for easy implementation of continuous integration (CI).
From the dawn of OpenEmbedded and the Yocto Project, Konsulko engineers have been community contributors and guides for crafting commercial products. Our expertise spans RAUC, Mender, and various open-source solutions for top-notch software updates. Please get in touch with us to discuss how Konsulko engineers can help your own embedded product development.
https://www.konsulko.com/wp-content/uploads/2024/01/RAUC-BeagleBone-Black-2.png600600Leon Anavihttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngLeon Anavi2024-01-30 15:14:162024-04-01 10:26:02Integrating RAUC with Yocto Project on BeagleBone Black
Formerly known as the Yocto Project Developer Day, the Summit has an expanded scope to include both workshops for engineers building customized Linux distributions and applications, as well as an open forum where maintainers, trainers and users present papers on how the project is evolving and how they are using it.
Leon will examine the technical aspects — dependance on Neat VNC and AML libraries, TLS support and user authentication — followed by a step by step tutorial to demonstrate remote connection from another computer.
The presentation aims to simplify the process of setting up the VNC backend in Weston and empower developers to utilize this powerful feature seamlessly within their projects.
We hope you will join us for this presentation, and all of this new edition of Yocto Project Virtual Summit.
https://www.konsulko.com/wp-content/uploads/2023/10/YoctoVirtualSummit11-2023.png600600Konsulko Grouphttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngKonsulko Group2023-11-01 10:12:202023-11-22 09:25:58Konsulko Group engineer to speak at Yocto Project Summit
Adding a feature to a linux-yocto based kernel is fairly well documented. This makes sense because it is the most common thing you might want to do: “My board needs support for this sensor added to our BSP.”
Konsulko Group recently helped a customer that had exactly the opposite problem, a standard feature in linux-intel (which includes linux-yocto.inc and inherits the kernel-yocto class) needed to be removed. The SoC (in the Intel™ “Bay Trail” family) and the off-the-shelf industrial PC had a problem. When USB 3.0 (xHCI) support is enabled, the default BIOS settings (xHCI Mode = Auto) would cause the system to lock-up upon either warm reboot or shutdown. Given that these systems are deeply embedded in the field—where simply “hooking up a display, keyboard and mouse” is cost prohibitive—we needed to find an option that would not prevent OTA updates.
You might think you could apply a patch via SRC_URI to kmeta (yocto-kernel-cache), but this isn’t supported in the Yocto Project kernel tooling. Instead we can apply kernel config fragments that disable the problematic xHCI feature. The trade off is that USB 3.x devices won’t be able to run at full speed, but the systems in question have no need for USB 3.0 (they are a classic IoT gateway use case).
Determining your existing kernel configuration
The first thing you want to do in this situation is determine what your (default) kernel configuration is. In our case, Konsulko and our customer are using the ‘dunfell’ (3.1.x) branch of meta-intel with MACHINE="intel-corei7-64". Upon building an image or the kernel (e.g. bitbake virtual/kernel or bitbake linux-intel), the kernel configuration can be found at the following path:
Where the kernel version is 5.4.170, the (shortened) git commit hash of the kernel cache is 98cce1c95f and the (shortened) git commit hash of the kernel source is 36f93ff941.
Modifying the kernel configuration with menuconfig
The documented way to modify the kernel configuration is with:
bitbake -c menuconfig linux-intel
This approach works fine, but you must remember to copy the resulting .config to defconfig in your recipe’s SRC_URI.
Alternatively you can create your kernel config fragments in the kernel build directory and then add them to your kernel recipe. The fragments can be created with the help of the diffconfig script in the kernel source tree.
Also note that you must have already run:
$ bitbake -c kernel_configme -f linux-intel
or previously built the kernel in order for the .config to be present.
Modifying the kernel configuration with devtool
It will come as no surprise that my preferred way to modify the kernel configuration is to run:
$ devtool menuconfig linux-intel
But you will get an error:
ERROR: No recipe named 'linux-intel' in your workspace
So first we must get the kernel recipe into our workspace:
$ devtool modify linux-intel
Now we are able to run:
$ devtool menuconfig linux-intel
One benefit of this approach is that devtool will run the required steps that need to happen before menuconfig can be run (most notably the do_kernel_configme task).
Determining the changes needed
Regardless of which method you used to run menuconfig, you will now be presented with the text UI:
Since Konsulko and our customer already knew we needed to change the xHCI enablement, we can quickly </> for Search and then enter xHCI.
This gives us several results, but the ones that we care about have [=y](built-in) next to them:
CONFIG_USB_XHCI_HCD
CONFIG_USB_XHCI_PCI
CONFIG_USB_XHCI_PLATFORM
The top level item that needs to be disabled (set to N) is CONFIG_USB_XHCI_HCD:
After this we can spot check the other values are also disabled (by using </> for Search again): CONFIG_USB_XHCI_PCI and CONFIG_USB_XHCI_PLATFORM:
Satisfied that we have the needed change, we can save our configuration. Press the <E> key or click on < Exit > to exit the sub-menus until you are at the top of the stack. Press the <S> key or click on < Save > to save the configuration. At they prompt, press enter or click on <Yes>.
Now, the benefit of the devtool workflow comes into play, because we are rewarded with:
INFO: Updating config fragment <build>/workspace/sources/linux-intel/oe-local-files/devtool-fragment.cfg
The contents of this file are what you might expect:
# CONFIG_USB_XHCI_HCD is not set
Applying our configuration change
If we don’t already have one, we need a layer into which to put our changes:
Now—as long as our layer has high enough priority and no other recipes add kernel config fragments which conflict with our change—we should be able to build the kernel and inspect the resulting .config:
$ popd
$ bitbake linux-intel
$ grep -R XHCI tmp/work/corei7-64-intel-common-poky-linux/linux-intel/5.4.170+gitAUTOINC+98cce1c95f_36f93ff941-r0/linux-corei7-64-intel-common-standard-build/.config
# CONFIG_USB_XHCI_HCD is not set
# CONFIG_USB_ROLES_INTEL_XHCI is not set
We should also be able to run dmesg | grep xhci on the target and we would not expect to see any messages.
Summary
A Konsulko Group customer had a hardware problem that required us to remove a kernel feature. By using tools like devtool, we were able to fairly easily make a change to the kernel configuration and capture those changes in a persistent way with a .cfg fragment and a .bbappend. This approach solved the problem and allowed OTA updates to proceed to these deeply embedded devices in the field. Please contact us to discuss how Konsulko can help you with the unique requirements of your commercial project.
https://www.konsulko.com/wp-content/uploads/2022/02/How-KG-removes-a-feature-1.png12001200Tim Orlinghttps://www.konsulko.com/wp-content/uploads/2021/03/KonsulkoGroup-fullcolorlogo-340x156-1.pngTim Orling2022-02-21 18:53:112023-05-01 11:19:22Using kernel config fragments to remove an unwanted feature
