USING RUST WITH YOCTO PROJECT

Using Rust with Yocto Project

/by
by Paul Barker, Konsulko Group

Introduction

Rust is a modern programming language which emerged from Mozilla Research around 2010 and has been stable since 2015 (see the history section of the Wikipedia article). Rust is an excellent fit for embedded software development due to its focus on performance and safety without the overhead of garbage collection or a large language runtime. The zero-cost abstractions provided by Rust support developer productivity, allowing the use of traits, generics, functional programming styles and other high-level approaches in the knowledge that the resulting assembly code will perform just as well as more verbose, hand written lower level code.

Now is a great time to pick up Rust if you work in the Embedded space. The Rust Embedded Working Group has produced an Embedded Rust Book for an audience with existing embedded development knowledge and the Discovery Book for an audience with no embedded experience but a desire to learn. The Linux kernel community has began discussing the use of Rust within the kernel itself and the OpenEmbedded/Yocto Project community is planning to move Rust support into the openembedded-core layer during the next release cycle.

The benefits of Rust largely stem from the language’s memory model, the borrow checker and other features which will be new to most developers learning Rust. The above resources from the Rust Embedded WG along with several published books and online tutorials cover this material well and so it will not be repeated in detail here. However, if you wish to develop software in Rust for production deployment you should expect to invest at least a few weeks in learning the language, assuming existing familiarity with other programming languages like C, Python or Java.

This article will introduce a couple of simple example applications written in Rust, discuss how to compile and run these applications natively on a Linux system using the Cargo build system and show how to create Yocto Project recipes for these applications using the cargo-bitbake tool. Some basic knowledge of the Yocto Project is assumed but no previous experience of Rust should be required.

Getting Started with Rust

In this section we will install Rust on our Linux system and create two simple applications.

Note that it is not necessary to have Rust installed natively in order to build applications and libraries written in Rust under bitbake. However, installing Rust natively does allow us to develop, test and debug applications on the host before we try to cross-compile them and it is an important step to take when learning Rust.

Installing Rust

The recommended way of installing Rust on a Linux system is to download and run the official installation script. When prompted with the installation options it is usually sufficient to select option 1 to use the defaults. This will install Rust in the current user’s home directory and modify the profile files to set necessary environment variables at login.

$ curl https://sh.rustup.rs -sSf | sh

    <output snipped>

Current installation options:


   default host triple: x86_64-unknown-linux-gnu
     default toolchain: stable (default)
               profile: default
  modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>1

    <output snipped>

Rust is installed now. Great!

To get started you need Cargo's bin directory ($HOME/.cargo/bin) in your PATH
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run source $HOME/.cargo/env

After installing Rust you should log out and log back in to ensure that environment variables are set correctly.

Hello, World

As is traditional, we will begin by looking at a “Hello, World” application.

Each Rust application or library should have its own project directory. Within this directory, the source code is usually placed in a src directory and the top level directory contains the Cargo config file for the project
and any other collateral files such as a README file, LICENSE file, etc.

To create our project directory and populate it with initial files we can use the cargo init command:

$ cargo init hello-rs
     Created binary (application) package

Within our new hello-rs directory, this command creates an initial source file in src/main.rs with the following contents:

fn main() {
    println!("Hello, world!");
}

This makes our job of writing a Hello World application very simple – no code modifications are needed!

The cargo init command also creates a Cargo.toml file in our project directory to store project configuration and metadata:

[package]
name = "hello-rs"
version = "0.1.0"
authors = ["Paul Barker <pbarker@konsulko.com>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

We should test our new Hello World application natively before we try to cross-compile it under bitbake. To do this we can use the cargo build and cargo run commands:

$ cargo build
   Compiling hello v0.1.0 (/home/pbarker/Projects/Rust/rust/hello-rs)
    Finished dev [unoptimized + debuginfo] target(s) in 0.29s
$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/hello-rs`
Hello, world!

The last stage for our hello world application is to publish it as a git repository so that it can be fetched and built in our Yocto Project environment. We should create a LICENSE file and a README file in the top level of the project for the benefit of anyone who wants to reuse our code. My preference is for the Apache license version 2.0 but feel free to use any appropriate open source license here.

We should also add some extra metadata to the Cargo.toml file to specify the license, briefly describe the project and link to the location where we will be publishing the repository. This metadata will be used by cargo-bitbake when we generate a recipe for our application. In my case the repository will be published to my GitLab account.

--- a/Cargo.toml
+++ b/Cargo.toml
@@ -3,6 +3,9 @@ name = "hello-rs"
 version = "0.1.0"
 authors = ["Paul Barker <pbarker@konsulko.com>"]
 edition = "2018"
+license = "Apache-2.0"
+description = "Hello World application"
+repository = "https://gitlab.com/pbarker.dev/rust/hello-rs.git"
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

To finish up, commit the project files to a git repository and push to a location which can be reached when building using Yocto Project. Alternatively, you can make use of the hello-rs repository I published under my GitLab account.

The Hello World application above will demonstrate only part of the capabilities of the cargo-bitbake tool and the Rust support in Yocto Project. To see how Rust dependencies are handled in Yocto Project we’ll need an application which actually depends on a Rust library (known as a crate). The print-rand application will fill this role as it uses the rand crate to generate random numbers.

As before we will start by creating the project using cargo init:

$ cargo init print-rand
     Created binary (application) package

We will now modify the src/main.rs file to generate and print a random number:

use rand::prelude::*;

fn main() {
    let x: u32 = random();
    println!("x = {}", x);
}

We will also modify the Cargo.toml file to list the rand crate as a dependency:

[package]
name = "print-rand"
version = "0.1.0"
authors = ["Paul Barker <pbarker@konsulko.com>"]
edition = "2018"

[dependencies]
rand = "0.7"

We can build and test this application in a similar way to the previous Hello World app. Note that each dependency is downloaded and built along the way:

$ cargo build
   Compiling libc v0.2.77
   Compiling getrandom v0.1.15
   Compiling cfg-if v0.1.10
   Compiling ppv-lite86 v0.2.9
   Compiling rand_core v0.5.1
   Compiling rand_chacha v0.2.2
   Compiling rand v0.7.3
   Compiling print-rand v0.1.0 (/home/pbarker/Projects/Rust/rust/print-rand)
    Finished dev [unoptimized + debuginfo] target(s) in 3.60s
$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/print-rand`
x = 643356417

As before, we should add LICENSE and README files at this stage and extend the metadata in the Cargo.toml file:

--- a/Cargo.toml
+++ b/Cargo.toml
@@ -3,6 +3,9 @@ name = "print-rand"
 version = "0.1.0"
 authors = ["Paul Barker <pbarker@konsulko.com>"]
 edition = "2018"
+license = "Apache-2.0"
+description = "Print a random number"
+repository = "https://gitlab.com/pbarker.dev/rust/print-rand.git"
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

Lastly, commit the project files to a git repository and push to a public location. As before you may alternatively make use of the print-rand repository I published under my GitLab account.

Using cargo-bitbake

Now that we have created a couple of simple Rust applications and published them we can look at creating bitbake recipes using the cargo-bitbake tool. The first step of this process is to install the tool itself:

$ cargo install cargo-bitbake

    <output snipped>

   Compiling cargo-bitbake v0.3.14
    Finished release [optimized] target(s) in 12m 30s
  Installing /home/debian/.cargo/bin/cargo-bitbake
   Installed package `cargo-bitbake v0.3.14` (executable `cargo-bitbake`)

Before we run cargo bitbake for each our projects we should ensure that all changes are committed and pushed to the remote repository. We should also check the remote URL for each repository as this will be used to set SRC_URI. If the remote URL uses SSH access then you should ensure that this can be reached using an SSH key from the Yocto build environment or switch to HTTPS access.

Once we’re happy with the state of our repository we can run cargo bitbake in the top directory of our hello-rs project (note that we use a space not a dash in the command here):

$ cargo bitbake
No package.homepage set in your Cargo.toml, trying package.repository
Wrote: hello_0.1.0.bb

The message printed by cargo bitbake indicates that the repository entry in our Cargo.toml file was used as the homepage URL since we did not specify a separate homepage.

We can now look at the generated recipe file hello-rs_0.1.0.bb:

# Auto-Generated by cargo-bitbake 0.3.14
#
inherit cargo

# If this is git based prefer versioned ones if they exist
# DEFAULT_PREFERENCE = "-1"

# how to get hello-rs could be as easy as but default to a git checkout:
# SRC_URI += "crate://crates.io/hello-rs/0.1.0"
SRC_URI += "git://gitlab.com/pbarker.dev/rust/hello-rs.git;protocol=https;nobranch=1;branch=dev"
SRCREV = "cec42fb0d147e4a4b271256c475620ff627c8856"
S = "${WORKDIR}/git"
CARGO_SRC_DIR = ""


# please note if you have entries that do not begin with crate://
# you must change them to how that package can be fetched
SRC_URI += " \
"



# FIXME: update generateme with the real MD5 of the license file
LIC_FILES_CHKSUM = " \
    file://LICENSE;md5=86d3f3a95c324c9479bd8986968f4327 \
"

SUMMARY = "Hello World application"
HOMEPAGE = "https://gitlab.com/pbarker.dev/rust/hello-rs.git"
LICENSE = "Apache-2.0"

# includes this file if it exists but does not fail
# this is useful for anything you may want to override from
# what cargo-bitbake generates.
include hello-rs-${PV}.inc
include hello-rs.inc

As we noted above, the SRC_URI and SRCREV entries are based on the remote URI and the currently checked out commit of our local git repository. The SUMMARY, HOMEPAGE and LICENSE entries are based on the metadata in our Cargo.toml file. The LIC_FILES_CHKSUM entry is set based on the LICENSE file in our project.

We can also generate a recipe for the print-rand project by running cargo bitbake in the top directory of this project:

$ cargo bitbake
No package.homepage set in your Cargo.toml, trying package.repository
Wrote: print-rand_0.1.0.bb

The print-rand_0.1.0.bb recipe is very similar to the previous recipe for our hello world application. The SRC_URI, SRCREV, SUMMARY and HOMEPAGE are changed to the appropriate values for this project. The SRC_URI variable is extended with a list of the dependency crates required by this project, this includes not just the rand crate that we explicitly listed as a dependency in our Cargo.toml file but also all the recursive dependencies of this crate.

# Auto-Generated by cargo-bitbake 0.3.14
#
inherit cargo

# If this is git based prefer versioned ones if they exist
# DEFAULT_PREFERENCE = "-1"

# how to get print-rand could be as easy as but default to a git checkout:
# SRC_URI += "crate://crates.io/print-rand/0.1.0"
SRC_URI += "git://gitlab.com/pbarker.dev/rust/print-rand.git;protocol=https;nobranch=1;branch=dev"
SRCREV = "3397247f929f28d70adbb65d3990dc72699553bb"
S = "${WORKDIR}/git"
CARGO_SRC_DIR = ""


# please note if you have entries that do not begin with crate://
# you must change them to how that package can be fetched
SRC_URI += " \
    crate://crates.io/cfg-if/0.1.10 \
    crate://crates.io/getrandom/0.1.15 \
    crate://crates.io/libc/0.2.79 \
    crate://crates.io/ppv-lite86/0.2.9 \
    crate://crates.io/rand/0.7.3 \
    crate://crates.io/rand_chacha/0.2.2 \
    crate://crates.io/rand_core/0.5.1 \
    crate://crates.io/rand_hc/0.2.0 \
    crate://crates.io/wasi/0.9.0+wasi-snapshot-preview1 \
"



# FIXME: update generateme with the real MD5 of the license file
LIC_FILES_CHKSUM = " \
    file://LICENSE;md5=86d3f3a95c324c9479bd8986968f4327 \
"

SUMMARY = "Print a random number"
HOMEPAGE = "https://gitlab.com/pbarker.dev/rust/print-rand.git"
LICENSE = "Apache-2.0"

# includes this file if it exists but does not fail
# this is useful for anything you may want to override from
# what cargo-bitbake generates.
include print-rand-${PV}.inc
include print-rand.inc

We can now add these recipes to a Yocto Project layer and build them using bitbake. Support for the Rust toolchain in Yocto Project is provided by the meta-rust layer so this layer must be included before these recipes can be built. As there are many ways to put together a Yocto Project build we won’t cover this in this blog post. However, you can see a walkthrough of this process in the Using Rust with Yocto Project demo presented at theYocto Project Summit Europe 2020.

How Mender works

/by

by Tom Rini, VP Engineering

Software Update solutions are a key part of our services offering. For open source over-the-air updates, we often we recommend and work with mender.io. In fact, Konsulko Group is a Mender Authorized Referral Partner. Recently, a prospective customer expressed an interest in knowing more about how Mender works. Here’s the brief, informal introduction to member.io that I prepared, and now I’m sharing with you.

—

As a high level starting point, https://mender.io/how-it-works provides a good overview of what’s supported and what it covers. In short, Mender starts off by providing support for a traditional “A/B” approach to system updates, where if the update isn’t marked as valid (and there’s hooks for the application(s) to verify the system before this is done), it’s assumed invalid and the system will roll-back automatically. While “OTA” implies over the network, it can just as easily be done by providing (and validating) a USB key that contains an update.

One of the reasons we recommend Mender is that it has very good in-depth documentation. The starting point for all of that is https://docs.mender.io/2.4/ which covers all of the topic starting from how to implement Mender support in a device and including how to create your own server infrastructure if you don’t want to use their paid service. While there are a number of important pages there, one that I like to highlight is https://docs.mender.io/2.4/artifacts/state-scripts as it shows the state machine for an update and talks about some of the common use cases that come up for user interaction or dealing with failures.

Another place I want to call out is https://docs.mender.io/2.4/devices/update-modules which is also mentioned in the first link. This is how Mender is extended to provide updates for other parts of the system that are not the rootfs itself.  Since updating a Docker container is something that has been mentioned before I want to also note https://hub.mender.io/t/docker/324 as it is a well supported module for this specific case.

I hope you find this information useful. Please contact us if you have specific questions. We’re looking forward to talking to you about your own specific OTA needs.

Time to sign up for Virtual ELC Europe 2020

/by

Join us at the Linux Foundation’s Embedded Linux Conference Europe, the premier vendor-neutral technical conference for embedded Linux developers, presented virtually on October 26–29, 2020.

Konsulko Group’s Leon Anavi will give a talk on Software Update Solutions for Yocto Project and OpenEmbedded, exploring the integration in Yocto and OpenEmbedded of A/B and binary delta updates over the air or through a USB stick.

No matter where you are in the world, here’s a good chance to experience ELC Europe for a fraction of the cost of the in-person event.

Getting Started with RAUC on Raspberry Pi

/2 Comments/by

RAUC is a secure, robust and flexible open source software for A/B updates of Embedded Linux devices. It is appropriate for various use cases and it is compatible with all popular build systems: The Yocto Project/ OpenEmbedded, Buildroot and PTXdist.

Konsulko Group engineers have experience with all popular open source solutions for software over the air updates of embedded Linux devices, including Mender, SWUPdate, HERE OTA Connect based on OSTree and Aktualizr. In this article we will discuss the exact steps to integrate RAUC with the Yocto Project (YP) and OpenEmbedded (OE) for Raspberry Pi – the most popular single board computer among students, hobbyists and makers.

For the practical example in this article we will be using the latest and greatest Raspberry Pi as of the moment: Raspberry Pi 4 Model B. Versions with different RAM sizes are available on the market. Any of these Raspberry Pi 4 Model B versions are OK for this RAUC demonstration.

As long time developers and users of the Yocto Project and OpenEmbedded, both have become favorite tools for creating customized distributions for Konsulko engineers. We frequently use and support them commercially. The Yocto Project is a Linux Foundation collaborative open source project for creating custom Linux distributions for embedded devices. It is based on Poky, the reference distribution of the Yocto Project, using the OpenEmbedded build system. The Yocto Project releases on a 6-month cadence. As of the time of this writing, the latest stable release is Dunfell (3.1).

RAUC is a powerful and flexible open source solution that requires advanced skills for initial integration. To use RAUC in an image for Raspberry Pi built with the Yocto Project and OpenEmbedded, it requires:

  • U-Boot as a bootloader
  • Enabled SquashFS in the Linux kernel configurations
  • ext4 root file system
  • Specific partitioning of the microSD card that matches the RAUC slots
  • U-Boot environment configurations and a script to properly switch RAUC slots
  • Certificate and a keyring to RAUC’s system.conf

RAUC is capable of covering various use cases and scenarios, including advanced options for single or redundant data partitions. Upgrades are performed through the so called RAUC bundles. It is possible to install them over the air or using the old-fashioned method with a USB stick. For managing updates to a fleet of Internet of Things, it is possible to integrate RAUC with Eclipse hawkBit project that acts as a deployment server with a nice web user interface.

For the sake of simplicity, this article focuses on the most simple and straight-forward use case with 2 identical RAUC slots: A and B. For each slot we will have a separate partition on the microSD card for Raspberry Pi. We have already covered most of the RAUC requirements in an additional Yocto/OE layer called meta-rauc-raspberrypi. We will use it to put the pieces together. First we will build a minimal bootable image for Raspberry Pi 4 with RAUC. We will flash it to both A and B slots. After that we will build a RAUC bundle that adds the text editor nano. Finally we will install this RAUC bundle on the B slot, reboot and verify that nano is present.

Building a Linux Distribution with RAUC

Follow the steps below to build a minimal image for Raspberry Pi with Yocto, OpenEmbedded and RAUC as well as to perform a software update:

  • Download Poky, the reference distribution of the Yocto Project:

git clone -b dunfell git://git.yoctoproject.org/poky poky-rpi-rauc
cd poky-rpi-rauc

  • Download meta-openembedded layer:

git clone -b dunfell git://git.openembedded.org/meta-openembedded

  • Download Yocto/OE BSP layer meta-raspberrypi:

git clone -b dunfell git://git.yoctoproject.org/meta-raspberrypi

  • Download Yocto/OE layers for RAUC:

git clone -b dunfell https://github.com/rauc/meta-rauc.git

git clone -b dunfell https://github.com/leon-anavi/meta-rauc-community.git

  • Initialize the build environment:

source oe-init-build-env

  • Add layers to conf/bblayers.conf:

bitbake-layers add-layer ../meta-openembedded/meta-oe/
bitbake-layers add-layer ../meta-openembedded/meta-python/
bitbake-layers add-layer ../meta-openembedded/meta-networking/
bitbake-layers add-layer ../meta-openembedded/meta-multimedia/
bitbake-layers add-layer ../meta-raspberrypi/
bitbake-layers add-layer ../meta-rauc
bitbake-layers add-layer ../meta-rauc-community/meta-rauc-raspberrypi/

  • Adjust conf/local.conf for Raspberry Pi 4 with systemd and RAUC by adding the following configurations to the end of the file:
MACHINE = "raspberrypi4"

DISTRO_FEATURES_append = " systemd"
VIRTUAL-RUNTIME_init_manager = "systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_initscripts = ""

IMAGE_INSTALL_append = " rauc"

IMAGE_FSTYPES="tar.bz2 ext4 wic.bz2 wic.bmap"
SDIMG_ROOTFS_TYPE="ext4"
ENABLE_UART = "1"
RPI_USE_U_BOOT = "1"
PREFERRED_PROVIDER_virtual/bootloader = "u-boot"

WKS_FILE = "sdimage-dual-raspberrypi.wks.in"

  • Build a minimal bootable image:

bitbake core-image-minimal

NOTE: Building an image from scratch requires a lot of operations and takes some time so please patiently wait until bitbake completes all tasks.

  • Flash the image to a microSD card and boot it on Raspberry Pi 4:

sudo umount /dev/sdX*
bzcat tmp/deploy/images/raspberrypi4/core-image-minimal-raspberrypi4.wic.bz2 | sudo dd of=/dev/sdX
sync

  • Attach USB to UART debug cable to Raspberry Pi 4, plug ethernet cable and the microSD card. Turn on Raspberry Pi 4. Verify that the system boots successfully.
  • Now, let’s extend the image with the simple text editor nano by adding the following line to the end of conf/local.conf:

IMAGE_INSTALL_append = " nano"

  • Build a RAUC bundle:

bitbake update-bundle

  • Start a web server:

cd tmp/deploy/images/raspberrypi4/
python3 -m http.server

  • On the Raspberry Pi download the RAUC bundle, install it and reboot the board:

wget http://192.168.1.2:8000/update-bundle-raspberrypi4.raucb -P /tmp
rauc install /tmp/update-bundle-raspberrypi4.raucb
reboot

  • After successful upgrade with RAUC reboot the Raspberry Pi and verify that nano is now present:

which nano

  • Check RAUC status to confirm that now the second partition has been booted:

rauc status

For Internet of Things and other real-world products, the whole build procedure with the Yocto Project and OpenEmbedded can be optimized further to just a few commands for easy implementation of continuous integration (CI).

Konsulko engineers have been there since the earliest days of the OpenEmbedded build framework and the Yocto Project. We have experience with RAUC and various other open source solutions for software updates. Please contact us if you need your “own” rock-solid Linux distro for your own embedded product.