Tag Archive for: konsulko group

Konsulko Group brings security focus to ELCE Dublin

At the Embedded Linux Conference in Dublin, Ireland, September 13-16, 2022, Konsulko Group’s Tim Orling will present “Tales from the Crypt: Implementing Secure Boot and Disk Encryption on Tegra Platforms.” Learn about the challenges and successes implementing effective security on Nvidia’s Tegra.

Secure Boot challenges

“Secure boot” needs different implementations on different platforms. For Tegra platforms, secure boot involves a one-time only burning of keys into the on-device fuses. Tim will share a reliable approach to confidently secure boot into the vendor’s Ubuntu-based OS before creating a Yocto Project-built OS.

Disk encryption with LUKS and dm-crypt

Extending this approach to disk encryption – testing the vendor’s OS before moving on to creating our own, Tim attempted to stay as close to the vendor’s tools (luks-srv and luks-srv-app) and design as possible, to try to “future proof” the implementation for newer releases of Linux for Tegra.

Extending for Over-the-Air updates

A/B flashing for OTA updates (e.g. rauc or mender) added additional challenges, generalizing the approach for the meta-tegra community. The end solution must address the bootloader, initramfs, kernel command line, /etc/crypttab, /etc/fstab and more. Add in the complexity of the partition table layout and flashing tools for Tegra platform, and it can be a wild ride.

Dublin and ELCE

The city of Dublin, Ireland has a storied history of literature, music and beverage. In September, Dublin will play host to the Embedded Linux Conference, Europe (ELCE), part of the Linux Foundation’s Open Source Summit.

Launched in 2005, Embedded Linux Conference is for companies and developers using Linux in embedded products. It gathers the technical experts working on embedded systems and applications for education and collaboration, paving the way for transformation in these important and far reaching areas.

To attend, register for Open Source Summit. You’ll also get access to all the other events in the Open Source Summit collection. Hope you will be able to join us in person or virtually.

Pick the Right Software Update Solution for Your Product

Live Webinar:

Software Update Mechanisms: Selecting the Best Solution for Your Embedded Linux Device

August 18, 2022 at 1 pm EDT/7pm CET

Get the foundation you need to make an informed decision on the right solution for your software update needs. Leon Anavi of Konsulko Group and Jeff Tranter of ICS will examine several different industry approaches, including A/B updates with a dual-redundant scheme, delta updates, container-based updates and combined strategies, as well as the leading technologies that support these approaches.

You can find complete information and register here for the free webinar on the ICS website. Hope you’ll join us on August 18.

Join us at Embedded World in Nuremberg, June 21-23, 2022

Are you coming to Embedded World? Konsulko Group will be there, too, supporting our partners and friends.

Be sure to stop by the Automotive Grade Linux (AGL) stand in Hall 4 to see the AGL ecosystem and to say hello.

Better still, if you’d like to set up an appointment at the show to talk with us about your company’s embedded products and your software development requirements, please contact us to set up a meeting.

Konsulko Group speaking at Embedded Linux Conference Austin

Konsulko Group engineers will make two presentations at the Embedded Linux Conference North America (ELC) in Austin, Texas (and virtual), June 21-24, 2022.

Software Update Mechanisms

On Wednesday, June 22 at 2:35pm CDT (Texas time), Leon Anavi will present How to Choose a Software Update Mechanism for Embedded Linux Devices.

This talk will look at the advantages and disadvantages of widely-used industry approaches: A/B updates with dual redundant scheme, delta updates, container-based updates and combined strategies. Open source technologies such as Mender, RAUC and libostree-based solutions implement these strategies and provide tools to manage updates of multiple devices. Leon will discuss how to choose an appropriate open source solution to implement for a specific project.

Edge Computing with RISC-V and Linux

On Thursday, June 23 at 2:55pm CDT, Vitaly Vul and Maria Vul will present Edge Computing with RISC-V Platforms Running XIP Linux.

XIP stands for eXecute In Place, allowing code to be executed directly from flash without copying the code to RAM first, making it possible to run Linux on such RISC-V devices as Kendryte K210, which has only 8 MB of SRAM, not only for demonstration purposes but for real applications as well. This talk will cover running edge computing specifically on K210 under Linux with XIP enabled.

Part of Open Source Summit North America

These and many other excellent presentations at ELC are part of the Linux Foundation’s Open Source Summit. We hope you will join us in June.

Embedded Linux engineers have a choice to make in June

At Konsulko Group, we’ve participated in the “virtual” developer conferences created out of necessity over the past couple of years. We eagerly awaited the return of the face-to-face meetings of the embedded community.

Our first opportunity was the Embedded Linux Conference in Seattle last September. Although major components were still virtual, those who where able to attend in person experienced the event “together“ in ways that even the best remote simulation cannot achieve.

So we are very much looking forward to ELC (part of Open Source Summit North America 2022) in Austin, Texas, and the return of Embedded World in Nuremberg, Germany. Unfortunately, the pandemic initially forced rescheduling of both. Now they will occur on exactly the same dates (starting June 21, 2022) half a world apart, making it impossible to attend both onsite.

Which conference should you attend?

Unless you live in Texas or Germany, choosing by geography may not be the best option. The conferences are very different.

ELC is the premier event for companies and developers using Linux in embedded products. It gathers the technical experts working on embedded systems and applications for education and collaboration, historically paving the way for transformation in these far reaching areas.

ELC is also always a good time.

For 20 years, Embedded World has combined

  • an exhibition for engineers and technical management, and
  • a world-leading conference at the intersection of applied research and industrial applications.

The trade show is massive, filling several halls, with many of Konsulko’s friends and partners exhibiting year after year. (If you go, be sure to stop by the Automotive Grade Linux stand in Hall 4.)

We hope to see you at one of these important events in June.

Using kernel config fragments to remove an unwanted feature

Adding a feature to a linux-yocto based kernel is fairly well documented. This makes sense because it is the most common thing you might want to do: “My board needs support for this sensor added to our BSP.”

Konsulko Group recently helped a customer that had exactly the opposite problem, a standard feature in linux-intel (which includes linux-yocto.inc and inherits the kernel-yocto class) needed to be removed. The SoC (in the Intel™ “Bay Trail” family) and the off-the-shelf industrial PC had a problem. When USB 3.0 (xHCI) support is enabled, the default BIOS settings (xHCI Mode = Auto) would cause the system to lock-up upon either warm reboot or shutdown. Given that these systems are deeply embedded in the field—where simply “hooking up a display, keyboard and mouse” is cost prohibitive—we needed to find an option that would not prevent OTA updates.

You might think you could apply a patch via SRC_URI to kmeta (yocto-kernel-cache), but this isn’t supported in the Yocto Project kernel tooling. Instead we can apply kernel config fragments that disable the problematic xHCI feature. The trade off is that USB 3.x devices won’t be able to run at full speed, but the systems in question have no need for USB 3.0 (they are a classic IoT gateway use case).

Determining your existing kernel configuration

The first thing you want to do in this situation is determine what your (default) kernel configuration is. In our case, Konsulko and our customer are using the ‘dunfell’ (3.1.x) branch of meta-intel with MACHINE="intel-corei7-64". Upon building an image or the kernel (e.g. bitbake virtual/kernel or bitbake linux-intel), the kernel configuration can be found at the following path:

<build>tmp/work/corei7-64-intel-common-poky-linux/linux-intel/5.4.170+gitAUTOINC+98cce1c95f_36f93ff941-r0/linux-corei7-64-intel-common-standard-build/.config

Where the kernel version is 5.4.170, the (shortened) git commit hash of the kernel cache is 98cce1c95f and the (shortened) git commit hash of the kernel source is 36f93ff941.

Modifying the kernel configuration with menuconfig

The documented way to modify the kernel configuration is with:

bitbake -c menuconfig linux-intel

This approach works fine, but you must remember to copy the resulting .config to defconfig in your recipe’s SRC_URI.

Alternatively you can create your kernel config fragments in the kernel build directory and then add them to your kernel recipe. The fragments can be created with the help of the diffconfig script in the kernel source tree.

Also note that you must have already run:

$ bitbake -c kernel_configme -f linux-intel

or previously built the kernel in order for the .config to be present.

Modifying the kernel configuration with devtool

It will come as no surprise that my preferred way to modify the kernel configuration is to run:

$ devtool menuconfig linux-intel

But you will get an error:

ERROR: No recipe named 'linux-intel' in your workspace

So first we must get the kernel recipe into our workspace:

$ devtool modify linux-intel

Now we are able to run:

$ devtool menuconfig linux-intel

One benefit of this approach is that devtool will run the required steps that need to happen before menuconfig can be run (most notably the do_kernel_configme task).

Determining the changes needed

Regardless of which method you used to run menuconfig, you will now be presented with the text UI:

Since Konsulko and our customer already knew we needed to change the xHCI enablement, we can quickly </> for Search and then enter xHCI.

This gives us several results, but the ones that we care about have [=y](built-in) next to them:

  1. CONFIG_USB_XHCI_HCD
  2. CONFIG_USB_XHCI_PCI
  3. CONFIG_USB_XHCI_PLATFORM

The top level item that needs to be disabled (set to N) is CONFIG_USB_XHCI_HCD:

After this we can spot check the other values are also disabled (by using </> for Search again):
CONFIG_USB_XHCI_PCI and CONFIG_USB_XHCI_PLATFORM:

Satisfied that we have the needed change, we can save our configuration. Press the <E> key or click on < Exit > to exit the sub-menus until you are at the top of the stack. Press the <S> key or click on < Save > to save the configuration. At they prompt, press enter or click on <Yes>.

Now, the benefit of the devtool workflow comes into play, because we are rewarded with:

INFO: Updating config fragment <build>/workspace/sources/linux-intel/oe-local-files/devtool-fragment.cfg

The contents of this file are what you might expect:

# CONFIG_USB_XHCI_HCD is not set

Applying our configuration change

If we don’t already have one, we need a layer into which to put our changes:

$ bitbake-layers create-layer ~/Projects/meta-awesome-bsp

Add the layer to our active layers:

$ bitbake-layers add-layer ~/Projects/meta-awesome-bsp

Create a directory–following the pattern in openembedded-core–for our kernel changes:

$ mkdir -p ~/Projects/meta-awesome-bsp/recipes-kernel/linux

Finish our recipe:

$ devtool finish linux-intel ~/Projects/meta-awesome-bsp

Examine the resulting directory structure:

$ tree ~/Projects/meta-awesome-bsp
/home/<user>/Projects/meta-awesome-bsp
├── conf
│&nbsp;&nbsp; └── layer.conf
├── COPYING.MIT
├── README
├── recipes-example
│&nbsp;&nbsp; └── example
│&nbsp;&nbsp;     └── example_0.1.bb
└── recipes-kernel
    └── linux
        ├── linux-intel
        │&nbsp;&nbsp; └── devtool-fragment.cfg
        └── linux-intel_%.bbappend

Since this might not be the only change to the kernel we will need to make, let us give the fragment a better name:

pushd ~/Projects/meta-awesome/recipes-kernel/linux/linux-intel
mv devtool-fragment.cfg disable-xhci-hcd.cfg

And create an .scc file to give the Yocto Project kernel tooling better hints of how to apply our change:

cat << EOF >> disable-xhci-hcd.scc
# SPDX-License-Identifier: MIT
define KFEATURE_DESCRIPTION "Disable options for xhci (USB 3.0)"
define KFEATURE_COMPATIBILITY board

kconf hardware disable-xhci-hcd.cfg
EOF

And finally make changes to our linux-intel_%.bbappend to reflect these files:

$ cd ..
$ cat linux-intel_%.bbappend
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

SRC_URI += "\
    file://disable-xhci-hcd.cfg \
    file://disable-xhci-hcd.scc \
"

Now—as long as our layer has high enough priority and no other recipes add kernel config fragments which conflict with our change—we should be able to build the kernel and inspect the resulting .config:

$ popd
$ bitbake linux-intel
$ grep -R XHCI tmp/work/corei7-64-intel-common-poky-linux/linux-intel/5.4.170+gitAUTOINC+98cce1c95f_36f93ff941-r0/linux-corei7-64-intel-common-standard-build/.config
# CONFIG_USB_XHCI_HCD is not set
# CONFIG_USB_ROLES_INTEL_XHCI is not set

We should also be able to run dmesg | grep xhci on the target and we would not expect to see any messages.

Summary

A Konsulko Group customer had a hardware problem that required us to remove a kernel feature. By using tools like devtool, we were able to fairly easily make a change to the kernel configuration and capture those changes in a persistent way with a .cfg fragment and a .bbappend. This approach solved the problem and allowed OTA updates to proceed to these deeply embedded devices in the field. Please contact us to discuss how Konsulko can help you with the unique requirements of your commercial project.

Konsulko Group: The Year in Review 2021

Thanks to our customers, our partners and our dedicated team of engineers, 2021 was a very successful year for Konsulko Group. 

We’ve been chosen to work on important projects in consumer electronics, automotive, medical devices, agriculture, mining, finance, and autonomous vehicles. Our engineers helped our clients, new and old, build outstanding commercial products with Embedded Linux, Yocto Project and OpenEmbedded, as well as deploying Over-the-Air (OTA) software updating.

Partnerships

“Konsulko is a recommended and trusted partner for helping Mender customers and prospects succeed in the Embedded Linux space. They have an excellent team of professionals who will deliver on time and as agreed.”

Thomas Ryd, CEO, mender.io

We continued our strong relationships with mender.io, and with the Linux Foundation and Automotive Grade Linux (AGL). 

We also launched a new partnership with PHYTEC at the Embedded Linux Conference (ELC) in Seattle. Konsulko is providing support and development for PHYTEC customers using their Systems on Modules (SOMs) and Yocto Project Linux distribution.

Conferences

As in previous years, Konsulko engineers were active participants (often virtually) in conferences and developer gatherings, making presentations at ELC and Yocto Project Summits, and writing technical papers on Yocto Project, security, and OTA. We hope for a time in the not-too-distant future when it makes sense for developers from all over the world to gather again face-to-face to share ideas. 

Contributions

As many of our engineers have been working (and playing) with Open Source Software (OSS) for over 20 years, Konsulko Group is proud of our continuing commitment to contribute to the community. Last month we were happy to hear that Konsulko is #3 in contributions to Yocto Project “Non Core,” just behind industry giants ARM and Fujitsu, and ahead of every other software or hardware company.

This was even before we welcomed OSS veteran Tim Orling to the team in December. Tim was recently Core OS Architect and Yocto Project Architect for Intel Corporation, and serves as a maintainer of meta-python and many recipes in openembedded-core. He has authored hundreds of applied patches for openembedded-core and meta-openembedded. 

Doing business with Konsulko Group

As a team we have found that the best way to work with our customers and prospective customers is to provide a high level of transparency about our business processes. This year, we’ve published blogs on scoping the effort required to successfully complete a customer project, and another describing our two models of engagement

  • Konsulko Continuous Time Engagement™ for dedicated engineering resources
  • Konsulko OnDemand Time Engagement™ for high level consulting, on-demand support, and specific tasks within a larger project.

We hope you have found these useful, and we look forward to working with you in 2022.

Choosing the best model to engage with Konsulko Group

As a consulting and services company, Konsulko Group is committed to acting as our customer’s trusted advisor. That extends across our entire business – from the technologies and services we recommend, to our time tracking and billing practices. We’re committed to running our business with the utmost integrity and transparency, following the best industry practices. 

Over the past couple of years, we have written about Konsulko’s Globally Employable Engineers™ based throughout the world and working together remotely (something we have done successfully long before the pandemic). Members of our team have years of software engineering experience, excellent English language skills, and are comfortable working directly with our customers. 

Konsulko engineers could get good jobs anywhere, but they choose to live in their home country, or elsewhere in North America or Europe. Demand for their skills is strong and tapping their talent does not come at an offshore salary cost, so understanding Konsulko’s engagement models can be crucial to matching resources to the job at hand.

Konsulko Continuous Time Engagement™ 

Under this model, sometimes referred to as “dedicated engineering” or “staff augmentation” in the industry, Konsulko provides dedicated and thus guaranteed engineering time for an agreed period. We will allocate the resource(s) to you, whether it is on a full time basis or some other number of hours per week, and those resources will be dedicated to you. 

As such, we will start invoicing for our time as soon as the resources are allocated to you and there is a signed Master Service Agreement (MSA), Statement of Work (SOW) and purchase order (PO) in place. We will not accrue billable time if any of these resources are out for personal reasons, holidays or vacation.

We will, however, accrue all other time, even if we’re blocked from making progress for reasons outside of our control, such as, hardware problems, VPN or other IT issues, access to your internal infrastructure when necessary, or lack of adequate documentation. This guarantees that our engineering resource stays dedicated to your project, so as soon as we get “unblocked” the work can immediately resume.

The Continuous Time Engagement model works well for larger projects with multiple requirements and deliverables,  projects where the requirements are fluid and you need maximum flexibility, or any project where the schedule calls for dedicated resources for the duration of the work. Customers who are willing to get directly involved in managing the hand-offs and task alignment between their own engineering team and Konsulko engineers often find this an efficient way to work with us.

Konsulko OnDemand Time Engagement™

Under this model of engagement, you will be invoiced only for actual time we spend on the project. If we estimate a job will take six engineer weeks and it only takes five, you pay for five weeks not six. Startup time setting up your hardware or working with your IT department to enable access to your infrastructure is considered part of the project and thus billable hours. 

If we get blocked from making progress for reasons such as those mentioned above, we stop accruing time until we are unblocked, and our resources will likely be reassigned to another project. Although we’ll do our best to switch them back to your project as soon as we’re re-enabled, we cannot guarantee the length of the delay, or the time required to get back up to full speed. 

The OnDemand Time Engagement can be best for a large number of projects:

  • Specific tasks within your larger project 
  • Longer term projects without a hard deadline that can be stopped and started as necessary
  • Projects where you reasonably expect to stop and restart the work (such as bring-up of new hardware which almost always results in a respin of the PCB)
  • On-demand support
  • High level consulting

Which is best for you? When you contact us, we will work with you to determine which model works for your requirements and timeline, and do our best to make your engagement with Konsulko Group efficient and memorable for all the right reasons.

At Yocto Project Summit: A/B Linux updates with RAUC

Konsulko Group Senior Software Engineer Leon Anavi will be speaking about A/B Linux updates with RAUC and meta-rauc-community: now and in the future at the Yocto Project Virtual Summit on December 1, 2021 at 12:20 (UTC).

About the presentation

RAUC is a safe and secure open source software solution for A/B updates of embedded Linux devices. It supports the Yocto Project and OpenEmbedded, Buildroot and PTXdist. Upgrades are performed through RAUC bundles which can be installed either through the network or the old-fashioned way with a USB stick.

In 2020, the layer meta-rauc-community was created to provide examples of how to integrate the lightweight update client RAUC on various machines. Leon will talk about the evolution of meta-rauc-community and provide guidelines for porting to new machines using Yocto and OpenEmbedded BSP layers.

About the Summit

The Yocto Project Summit is a 3-day virtual technical conference for engineers, open source technologists, students and academia in the OSS space. The classes will be presented in Zoom. It will be highly interactive, with chat sessions, side rooms, teaching assistants, and hands-on exercises with live class accounts. Registration is $40 for the entire conference.

We hope you will be able to join us at this always important event.

Embedded Linux support and development for PHYTEC customers

PHYTEC provides off-the-shelf System on Modules (SOMs) in support of advanced Arm Cortex devices. PHYTEC SOMs are a rapid, agile and long-term solution that enables product developers to simplify processes, streamline development, reduce timelines and minimize risk.

Konsulko Group is partnering with PHYTEC to offer Embedded Linux support and development to PHYTEC customers, integrating up-to-date drivers and other community releases on PHYTEC SOMs using PHYTEC’s reference Yocto Project Linux distribution.

We hope to see you at the Embedded Linux Conference in Seattle (and virtually), September 27-30, 2021. Konsulko will be in and around PHYTEC’s booth during the conference. We are also making two presentations at ELC. If you can’t make it to the conference this year, please contact us for more information, and talk with us about how Konsulko Group can help with your next project.

Tag Archive for: konsulko group