Analysis of μC/OS-III & Zephyr Project
For a more in-depth analysis we select an established RTOS from each group.
In the IoT group the Zephyr Project stands out as feature-rich and rapidly evolving RTOS. Supported by the Linux Foundation, Google, Intel, NXP, Nordic Semiconductor, Facebook and many others the further development is certain. Zephyr is also of interest because its complex ecosystem, this gives insight on how to design a flexible architecture for a modular system. Due to the large community behind the Zephyr Project there many talks about certain parts of the system, as well as a thorough documentation available online.
μC/OS-III is selected as a second RTOS for the analysis because it has similar features as the other open-source RTOS from general purpose group, but it is also known for its extensive documentation from its author J. J. Labrosse.
μC/OS-III
The μC/OS-III book [4] explains the design and implementation of a preemptive round-robin scheduler, in addition IPC and memory management are also covered. It goes as far as specific list implementations for different task states. In this subsection I will cover some aspects of the implementation without retyping the book completely.
Task
In μC/OS-III a task consists of a piece of code that can be called from the scheduler, a stack and a task control block (TCB). The TCB contains all the information about a task a kernel needs for scheduling. Most importantly the priority, a function pointer to the task entry point, a name for debugging, the stack pointer and the state is stored in the TCB. The state is updated by the kernel according to the state-machine in the figure, it governs which tasks are put into ready list. Upon most transitions to/from the Running state a context switch occurs. [4, pp. 41]
Figure: Task states in μC/OS-III (simplified); bold lines mark context switches
Scheduling
μC/OS-III schedules tasks preemptively with round-robin. In preemptive scheduling a task is put on hold as soon a higher priority task reaches ready state, e.g. an interrupt routine adds a message to a queue a high priority task is waiting for. To ensure low latency a context switch occurs immediately and a lower priority tasks is preempted, as seen before in the figure.
Figure: Round-robin scheduling / time slicing
It is different for multiple tasks of the same priority: one task cannot preempt another. With round-robin or time slicing the CPU is shared between these tasks equally. After a certain amount of CPU cycles (time quanta), the scheduler is called
Context Switching
A context switch occurs every time another task put into running state. At the beginning of a context switch the kernel pushes CPU registers onto the task stack and stores the stack pointer in the TCB. It then proceeds to load the stack pointer and registers from the next task. [4, pp. 111]
Minimizing the overhead of context switch is critical, because it at least is called after every time quanta and on interrupts.
Critical Sections
When scheduling tasks or switching context no preemption from a task or interrupt is allowed to take place as it would corrupt the system. These are examples for critical sections. In that case they are protected by disabling all interrupts. Not only uses the scheduler critical sections, they are also needed to access synchronization objects. [4, pp. 35]
Critical sections should only be used if really necessary and only for short code blocks, because they increase the reaction time an RTOS. Long sections can lead to missing deadlines.
Software Timer
For a simple action it is not always necessary to spawn a new task. Periodic or one-shot actions can be run as callbacks from a timer task. One single timer task can hold many callback functions. These actions are run from the same stack with little overhead. [4, pp. 145]
Interrupt Service
Interrupt Service Routines (ISR) in μC/OS-III that the user writes have to use the assembly language. A message or synchronization primitive can be sent in one of two ways:
direct post after completion of the ISR the interrupted task or newly ready high priority task is called directly.
deferred post the ISR posts its message to an interrupt queue. The kernel then calls an ISR handler task. After the interrupt queue has been emptied, the system goes back to normal operation.
Inter-Process Communication
μC/OS-III implements 3 different synchronization primitives and message queues: [4, pp. 171]
semaphore is used to synchronize tasks or restrict access to a resource. In μC/OS-III all semaphores are implemented as counting semaphores. They can be used for resources that can be accessed multiple times. When creating a semaphore, the user chooses how many times the semaphore can be taken. A semaphore with a max count of one is called a binary semaphore. A semaphore can only be taken, if the internal count is greater than zero. In μC/OS-III semaphores cannot prevent an unbounded priority inversion.
mutex is a binary semaphore, but the difference is that in μC/OS-III a mutex features priority inheritance (the preempted task inherits the priority from the high priority task waiting for the mutex), which means that an unbounded priority inversion cannot occur. This mechanism makes a mutex slightly slower than a binary semaphore.
flag groups are used when a task pends on many events. A task can pend until all events flags are set (conjunctive synchronization) or until one of the event flags is set (disjunctive synchronization).
message queues are used to pass information from one task or ISR to another task. A message queue is set up with a finite size and in FIFO or LIFO manner. Every queue keeps a list of pending tasks, sorted by priority. When a new message arrives, it is consumed by the highest priority pending task. Alternatively a message can be sent as broadcast to all pending tasks.
To transfer messages efficiently the kernel only sends the pointer to the data. It is up to the user to keep the data in scope, a typical pitfall when using message queues. As a solution μC/OS-III suggests using its memory manager, to put the data on the heap.
Memory Management (Heap)
Dynamic memory allocation and freeing is prone to fragmentation, because embedded systems run for years without a reset. μC/OS-III provides a memory management system that supplies equal sized blocks of memory in partitions, resulting in the layout in the figure. The user can set the size of the blocks for each partition at runtime. Memory allocation is now constrained to a fixed size, thus preventing fragmentation. [4, pp. 279]
Figure: Memory partitions with fixed block sizes
Memory Safety Mechanism
In μC/OS-III the stack size of a task must be set by the user which typically in done as a rough estimate. Because lots of small stacks are used instead of a large one, stack overflows are a common issue. An overflow can be detected in software or hardware. One software solution is to check that the stack pointer is within the limits set in the TCB. Some CPUs have a stack overflow detection that can be used. Most ARM Cortex-M CPUs feature a Memory Protection Unit (MPU) that can detect access outside set memory regions. [4, pp. 50]
There are other RTOS that use a single stack to minimize the chance of stack overflow to one. A single stack also maximizes memory efficiency because no space is wasted on an unused task stack. With a dedicated task stack on the other hand an MPU can rule out manipulation from any other task.
Source Code
As of mid 2020 the source code of μC/OS-III and all additional components, as well as the full documentation have been published on GitHub [24].
Zephyr Project
The review of Zephyr is limited to its architecture and ecosystem to avoid redundancies with the μC/OS-III analysis.
The following information is based on the latest version (v2.4.99) of the Zephyr online documentation [29].
Architecture
Figure: Zephyr System Architecture (v.2.3.0) [29]
The Zephyr project is structured in layers. The base consists of the hardware platform, either on-chip (SoC peripherals, CPU core) or off-chip (sensors on the board). The description of the platform mainly includes addresses of the memory mapped registers. The scheduler and kernel services access the CPU core on the platform through a first abstraction: the architecture interface. These three layers make up the kernel.
Based on the kernel are the OS services. These are low level drivers for the hardware interfaces (I²C, SPI, UART) and basic services (file system, IPC), which are abstracted by the low level API. The OS Services also consist of 3 layers of the ISO/OSI model: the data link layer (BLE, IEEE 802.15.4, WiFi), the network layer (IPv6/IPv4) and the transport layer (TCP/UDP).
The session, presentation (TLS) and application (HTTP, MQTT) layer of the ISO/OSI model are put in the application services. The services can be accessed through the high level API. The available OS and application services show Zephyrs focus on networking and IoT applications.
The user application can therefore rely on the many features that are already implemented in the Zephyr project. The application itself is completely platform agnostic, as the hardware is abstracted and handled within Zephyr.
The structure of the architecture much resembles a general purpose OS such as GNU/Linux. Thus, managing all the components of Zephyr can be a complex task. To simply this task Kconfig is used to configure the components and the Devicetree is used to describe the hardware.
Devicetree
The Devicetree is data structure that serves as source for all hardware information. In addition to the hardware description the Devicetree also contains its boot-time configuration. The concept of the Devicetree in Zephyr is the same as in Linux, but the implementation and usage are vastly different.
The Devicetree for a board will contain many hierarchically structured files describing the individual peripherals of an SoC (interfaces, GPIOs, ADCs) as well as assignments of board level components to those interfaces. As an example we will take a top-down approach for STM32 Nucleo-F446RE (ARM Cortex-M4F) evaluation board:
zephyr/boards/arm/nucleo_f446re/nucleo_f446re.dts
/dts-v1/;
#include <st/f4/stm32f446Xe.dtsi>1
/*...*/
/ {
model = "STMicroelectronics STM32F446RE-NUCLEO board";
compatible = "st,stm32f446re-nucleo";
chosen {2
zephyr,console = &usart2;
zephyr,shell-uart = &usart2;
zephyr,sram = &sram0;
zephyr,flash = &flash0;
};
leds {
compatible = "gpio-leds";
green_led_2: led_2 {3
gpios = <&gpioa 5 GPIO_ACTIVE_HIGH>;
label = "User LD2";
};
};
/*...*/
};
&usart1 {4
pinctrl-0 = <&usart1_tx_pb6 &usart1_rx_pb7>;
current-speed = <115200>;
status = "okay";
};
/*...*/
Listing: Devicetree extract for the STM32 Nucleo-F446RE board [28]
An extract from the Devicetree of the STM32 Nucleo-F446RE board configuration is shown in the listing. At
To find the definition of usart1
zephyr/dts/arm/st/f4/stm32f446.dtsi
#include <st/f4/stm32f401.dtsi>1
/ {
soc {
usart3: serial|@|40004800 {2
compatible = "st,stm32-usart", "st,stm32-uart";
reg = <0x40004800 0x400>;
clocks = <&rcc STM32_CLOCK_BUS_APB1 0x00040000>;
interrupts = <39 0>;
status = "disabled";
label = "UART_3";
};
/*...*/
zephyr/dts/arm/st/f4/stm32f401.dtsi
#include <st/f4/stm32f4.dtsi>3
/ {
soc {
/*...*/4
zephyr/dts/arm/st/f4/stm32f4.dtsi
#include <arm/armv7-m.dtsi>5
/*...*/
/ {
/*...*/
soc {
usart1: serial|@|40011000 {6
compatible = "st,stm32-usart", "st,stm32-uart";
reg = <0x40011000 0x400>;
clocks = <&rcc STM32_CLOCK_BUS_APB2 0x00000010>;
interrupts = <37 0>;
status = "disabled";
label = "UART_1";
};
/*...*/
Listing: Devicetree extracts for the STM32F446RE MCU [28]
In a microcontroller family most peripherals are shared throughout the different products. Thus, the majority of the hardware structure of the STM32F446 is included from STM32F401, as illustrated at
The STM32F401 Devicetree source again defines its specific hardware
Because usart1 is available on all products of the STM32F4 family, it is defined in the STM32F4 device tree file serial driver is used. The special function register address and the clock source are also given. The data structures regarding the ARM Cortex-M4F CPU then again are defined in another file included at
From this example we can conclude that the Devicetree splits the hardware into a large hierarchical structure of files, which increases complexity. On the other hand code redundancies are removed, making the adoption of other hardware platforms easier than with manual hardware configuration in the code base.
Kconfig
Kconfig is a configuration tool for the Zephyr kernel and subsystem. The configuration is applied at build time in the manner Kconfig is used for the Linux kernel. Kconfig can be called from the command line as shows in the figure. Software support such as network stacks, device drivers, file system support and logging features are set in Kconfig. But the target board with its SoC configuration is selected beforehand. Assigning specific peripheral interfaces (e.g. SPI3) to a driver is done in the Devicetree.
Figure: Kconfig command line UI
APIs & Device Driver Model
The system architecture in the figure illustrates the many components Zephyr is made of. All those components are accessed through APIs. The online documentation lists all APIs with their stability status. Some examples are: Audio Codec, CAN, DMA, File Systems, GPIO, Kernel Services, Networking, Shell, User Mode and Watchdog. The different APIs serve as a generic base for the different implementations which are based on specific platforms and protocols. We will now look at the most generic API: the device driver model.
According to the device driver model, a device consists of a name, a device configuration, driver data and a driver API. Separating API and data allows for multiple instances of one driver within a system.
static const struct i2c_driver_api1 api_funcs = {
.configure = i2c_stm32_runtime_configure,
.transfer = i2c_stm32_transfer
};
/*...*/
#define STM32_I2C_INIT(name)2 \
/*...*/
static const struct i2c_stm32_config3 i2c_stm32_cfg_##name = { \
.i2c = (I2C_TypeDef *)DT_REG_ADDR(DT_NODELABEL(name)), \
STM32_I2C_IRQ_HANDLER_FUNCTION(name) \
.bitrate = DT_PROP(DT_NODELABEL(name), clock_frequency), \
/*...*/
}; \
/*...*/
static struct i2c_stm32_data4 i2c_stm32_dev_data_##name; \
/*...*/
DEVICE_DT_DEFINE5 (DT_NODELABEL(name), &i2c_stm32_init, \
device_pm_control_nop, &i2c_stm32_dev_data_##name, \
&i2c_stm32_cfg_##name, \
POST_KERNEL, CONFIG_KERNEL_INIT_PRIORITY_DEVICE, \
&api_funcs); \
/*...*/
Listing: STM32 I²C Device Driver extract [28]
A simplified extract of the STM32 I²C driver is shown in the listing. At i2c_driver_api and assigns the function implementations to the interface. The name, configuration and data is influenced by the device tree, thus the macro
West
West is a meta-tool used to manage Zephyr based projects. It is pluggable meaning a lot of convenient features can be or already have been added. West is used to:
-
Set up and update Zephyr based projects (
west initandwest update), the sources of Zephyr are cloned automatically from many git repositories -
Build an application (
west build -p auto -b <board> <application directory>), the build process itself is based on CMake and the Ninja build system -
Run or debug an application on the target hardware (
west flashandwest debug) -
Call the Kconf UI on the command line (
west build -t menuconfig) or in a graphical window (west build -t guiconfig)