Engineering evidence11 min read

A real-time kernel was not enough: what our 10 ms Linux control test showed

Two controllers, a one-millisecond jitter budget, and several failed configurations. Here are the numbers, the decision they support, and the work that remains.

Industrial controller timing test bench with two controllers, I/O modules, oscilloscope, and jitter histograms

About the author

Dr. Mehran Kiani-Oshtorjani

Industrial software and simulation engineer

Mehran Kiani-Oshtorjani is a mechanical engineer and software developer working across real-time simulation, fluid-power systems, industrial control, and connected software. His doctoral research at LUT University focused on computational approaches for real-time industrial systems.

Engineering takeaways

  • On these two pilot devices, PREEMPT_RT alone still missed the one-millisecond maximum-jitter criterion.
  • PREEMPT_RT plus SCHED_FIFO priority 80 passed four steady-state runs, with and without the defined load.
  • This is evidence for a narrow 10 ms pilot profile, not a general claim that Raspberry Pi hardware is a certified real-time controller.

Stop discussing jitter as an adjective

Calling a Linux system "deterministic" without a period, workload, percentile, maximum, and test duration is marketing, not engineering. A control loop can average 10.000 ms and still be unusable if rare wake-ups arrive several milliseconds late. The useful question is whether the observed worst case remains inside the machine's timing budget under a defined load.[3][4][5]

PREEMPT_RT reduces scheduling latency by making more kernel paths preemptible and moving interrupt handling into schedulable threads. SCHED_FIFO changes which runnable task the scheduler chooses. They solve related but different problems, so neither should be treated as a checkbox that makes timing risk disappear.[2][3]

Write the acceptance contract before running the test

Our Demo 1 pilot used two Raspberry Pi controllers and a 10 ms scan period. A run passed only when maximum jitter stayed at or below 1 ms, missed and overrun cycles remained zero, snapshot age stayed at or below 20 ms, maximum snapshot publication stayed at or below 1 ms, and resident-memory growth stayed within 2048 KB.[1]

The passing configuration and the numbers

The first configuration to pass on both controllers was PREEMPT_RT with the runtime scheduled as SCHED_FIFO at priority 80. Across the four recorded steady-state runs, maximum jitter ranged from 0.063612 ms to 0.094448 ms. No missed cycles, overruns, or resident-memory growth were observed.[1]

BootCtrl MCR Demo 1 passing steady-state runs, 10 ms profile
ControllerLoadMax jitterSnapshot agePublish maxMissed / overrun
ANo0.085877 ms1.982421 ms0.096610 ms0 / 0
BNo0.063612 ms9.277714 ms0.064073 ms0 / 0
AYes0.094448 ms3.851881 ms0.081295 ms0 / 0
BYes0.067025 ms9.068315 ms0.079147 ms0 / 0

The failed runs taught us more than the pass

The stock or tuned configuration produced maximum jitter between 2.258783 ms and 3.662989 ms. Moving to PREEMPT_RT without FIFO scheduling reduced the worst case substantially, but both controllers still failed the 1 ms gate at 1.014743 ms and 1.340064 ms. CPU affinity and kernel choice were not sufficient; the runtime also needed an explicit real-time scheduling policy.[1][2][3]

  • Measure the complete scheduling configuration, not only the kernel name.
  • Keep blocking I/O, logging, networking, and remote services outside the cyclic thread.
  • Record the worst result and every failed run; deleting inconvenient evidence makes the next regression harder to diagnose.

What this proves, and what it does not

The study supports one decision: continue the 10 ms pilot timing work on this hardware and configuration. It does not prove final Demo 1 acceptance, value correctness, electrical I/O timing, long-duration tails, thermal behaviour, or suitability for a different machine. It certainly does not turn a development board into a safety controller.[1]

The unclosed gaps are explicit: cold start under load, 100 ms profiles, same-core service contention, archived backend freshness evidence, and end-to-end value validation. Controller B's snapshot age remained within the 20 ms criterion but near 9 ms, so it deserves attention in longer runs.[1]

The next test should try to break the result

  1. Repeat from cold start while CPU, memory, storage, and network load begin together.
  2. Pin the service process on the runtime core as a normal SCHED_OTHER task to expose priority and contention mistakes.
  3. Run long enough to encounter thermal control, log rotation, reconnects, and storage housekeeping.
  4. Capture physical I/O or a reference clock externally; software timestamps alone cannot prove the whole signal path.
  5. Archive raw samples, configuration, commit, pass criteria, and failures with the published summary.

Our position is simple: compact Linux hardware can be a useful machine-control target when the application envelope is narrow and measured. The engineering advantage comes from knowing the envelope, not from calling the hardware real time.[5][1]

Frequently asked questions

Does PREEMPT_RT guarantee deterministic control?

No. It reduces important sources of kernel scheduling latency, but hardware, firmware, drivers, workload, priorities, memory behaviour, and application design still determine the measured result.

Why use SCHED_FIFO for the cyclic runtime?

SCHED_FIFO lets a higher-priority runnable control thread pre-empt normal tasks. It must be paired with bounded work, careful priority design, and watchdogs because a broken FIFO task can starve the system.

Are these results production approval for Raspberry Pi controllers?

No. They are four passing steady-state pilot runs on two devices under a defined 10 ms profile. The published limitations list the tests still required before final sign-off.

Sources and standards

  1. MCR Demo 1 steady-state 10 ms timing evidenceBootCtrl, 26 July 2026
  2. PREEMPT_RT theory of operationLinux kernel documentation
  3. sched(7): overview of CPU schedulingLinux man-pages project
  4. clock_nanosleep(2): high-resolution process sleepLinux man-pages project
  5. Cyclictest documentationLinux Foundation Real-Time Linux project

BootCtrl Engineering last reviewed the technical claims and source links on 26 July 2026. Product statements are checked against the current implementation repositories; roadmap work is not presented as released capability.

Define the timing budget before choosing the controller

Bring us your machine period, I/O path, load, and failure criteria. We will help turn them into a measurable pilot.

Discuss a control pilot