This wiki shows how to enable an Hailo-8 M.2 AI Acceleration Module in the router and use it within an LXC container. It also explains how to set up additional utilities within the container.
Enable the router to run its own AI modules locally, directly on the router, without having to use a hyperscaler. Your own data can be used to train the AI modules because it is not shared with third parties. The entire value chain remains under your control and can be flexibly developed further. Applications include intelligent energy management systems, dynamic traffic control systems, and production management. The AI modules can be trained and fine-tuned for their core tasks to achieve optimal results.
“ai-chip”“ai-chip”The wiki was tested with router software nrsw-image-2800-0.0.0.0.img and the LXC container with debina 13 armhf (trixie)
An adapter is required for the Hailo-8 AI chip to be used with the router. The PCI Express interfaces are implemented as Mini PCIe on the router board. The Hailo-8 AI chip requires a Mini PCIe to M.2 M-key adapter.
The NB2800's circuit board features a DC/DC converter that supplies 3.3 volts to the two Mini-PCIe slots. The maximum output current is 6 amps. A maximum of 12 watts is available for the Mini-PCIe modules. The Hailo-8 AI chip has a maximum power consumption of less than 5 watts.
The module can be installed in slot 1 or 2 on the NB2800.
The Hailo-8 AI chip was inserted into the first slot
To ensure communication between the chip and the router, the module must be configured as Wi-Fi in the BD.
You can check whether the module is present via the router's console.
The module was detected in the router: “02:00.0” (device PCI address: bus 02, device 00, function 0).
The module’s device class was correctly identified as “Class 0b40” (DSP/signal processing controller).
The last two entries come from the “lspci/pci.ids” database: 1e60 Hailo Technologies and device ID 2864 Hailo-8 AI Processor.
The first step is to download the software that includes driver support for the Hailo-8 AI chip. To do this, click the link NB2800 or NB2810 to download the appropriate software for the two respective routers.
The password for access is “ai-chip”
Save the downloaded router software to the “/tmp” subdirectory on the router, and then connect to the router via the console using SSH or the Windows program PuTTY.
You can install this image on the router using the command swupdate -fr nrsw-image-2800-0.0.0.0.img.
In this example, I've set it up for a Telekom SIM card.
You can use a ping test to check whether the router can establish an Internet connection via the WWAN connection.
Activate the LXC container on the router
In the second step, the image of the LXC container in this case, a Debina 13 (trixie).
The easiest way to do this is to install the whole thing over the Internet. However, the router requires a mobile connection to the Internet.
From a Jenkins server you get the current image for the container
* Jenkins Server Page image-debian / Link
Here you can download a Debian image for various systems and versions.
The armhf tab is important here, and trixie
The image is the “rootfs.tar.xz” which is required. With a right click you can copy the link.
It would be structured like this:
When the “Apply” button is pressed, Debian will be installed.
Set up the LXC container further. A network interface is enabled here.
The container has now also been assigned an IP address.
Always assign a fixed IP address for the LXC container via the DHCP server.
The MAC address of the LXC container is 00:FF:AA:00:44:00.
Log in to the LXC container and test whether it can access the Internet.
——
The container is installed and set up. Now you can set up further tools on the container.
Enable USB administration on the router
When the USB device is plugged in, it is mounted by the router and stored in the system as device part “/dev/sda1“.
In order to release the device in the container, I need to know what kind of device it is: “b” is a block device, major number “8” and minor number “1”.
With this information, we can adjust the LXC configuration so that we have access to the device from within the container.
This change is made directly in the router software. When updating the router software, this entry must be made again because it is not part of the official software.
Add these entries to the end of the config file.
~ $ vi /etc/lxc/guest0/config
# Setting up a USB flash drive in the LXC container
lxc.cgroup.devices.allow = b 8:1 rwm
lxc.mount.entry = /dev/sda1 /dev/sda1 none bind,optional,create=file
Simply add these four entries to the LXC configuration file. Please ensure that you fill them in with your own parameters. Restart the router for these changes to take effect.
Now the USB device can be connected to the container and used.
The “mkdir” command is used to set up an area in the file system for access.In the second step, change the permissions of the file.
Check whether the rights have been set.
—
Test this after rebooting the router to see if the interface is present in the router.
In the next step, the interface must be passed to the LXC container.
~ $ vi /etc/lxc/guest0/config # Allow Access (Device 246) lxc.cgroup.devices.allow = c 246:0 rwm # Mount the Hailo-8 device node in the container lxc.mount.entry = /dev/hailo0 dev/hailo0 none bind,create=file 0 0
If the “hailo0” interface under the LXC container is set to mode “(000),” you must check the settings in lxc config again to make sure they are correct.
——
The runtime software HailoRT was built for Debian 13 (trixie) using software version 4.24.0.
Download the runtime software file “hailort-4.24.0-armhf.tar.gz”
“hailort-4.24.0-armhf.tar.gz” from the link tar file Password for the download: “ai-chip”
Place the “.tar” file on the LXC container and then extract it directly there. After the library directories have been rebuilt using ldconfig, you can view what has changed in terms of access.
⇒ ⇒ ⇒ ⇒ ⇒
In the second step, we test the Hailor CLI to ensure it is functioning properly.
Run “hailortcli –version” to check the installed version.
Verify the presence of the “hailo0” interface from within the LXC container using “ls -l /dev/hailo0”.
The CLI command “hailortcli scan” locates the device on the PCIe bus.
Direct access to the AI chip from within the container is achieved using the command “hailortcli fw-control identify”.
– ——
If the CLI for the Hailo AI chip ran without any issues, you can now test the whole thing using an AI model.
A pre-compiled neural network for the Hailo-8 chip was used as the model. It contains images representing approximately 1,000 classes. The model was created using JupyterLab as a “.hef” file. The finished model, “resnet18.hef”, can be downloaded from the link below.
To monitor the temperature of the Hailo chip, I use a small C program with a library from Hailo. This allows the chip to be monitored even further while the AI model is running. To compile it, all you need is g+ + on the container.
Create these lines of code in the file temp.cpp and then compile them into a C program.
#include <ctime>
#include <csignal>
#include <algorithm>
#include <cstdlib>
using namespace hailort;
static volatile std::sig_atomic_t running = 1;
static void on_sig(int) { running = 0; }
// This option is selected only if get_health_information() is publicly accessible
template <typename D>
auto print_health(D &d, int) -> decltype(d.get_health_information(), void()) {
auto h = d.get_health_information();
if (!h) { std::cerr << "get_health_information failed: " << h.status() << "\n"; return; }
std::cout << "Zone: " << (int)h->current_temperature_zone
<< " (0=gruen, 1=orange, 2=rot)"
<< ", Temperatur-Throttling aktiv: " << h->temperature_throttling_active
<< ", Overcurrent-Schutz aktiv: " << h->overcurrent_protection_active << "\n"
<< "orange: " << h->orange_temperature_threshold
<< " C (Hysterese " << h->orange_hysteresis_temperature_threshold << ")"
<< ", rot: " << h->red_temperature_threshold
<< " C (Hysterese " << h->red_hysteresis_temperature_threshold << ")\n";
}
template <typename D>
void print_health(D &, long) {
std::cout << "Exceptions: get_health_information() is not public.\n";
}
int main(int argc, char **argv) {
int interval = (argc > 1) ? std::atoi(argv[1]) : 5;
if (interval < 1) interval = 5;
std::signal(SIGINT, on_sig);
std::signal(SIGTERM, on_sig);
auto dev = Device::create();
if (!dev) { std::cerr << "Device::create failed: " << dev.status() << "\n"; return 1; }
Device &d = *dev.value();
print_health(d, 0);
while (running) {
auto t = d.get_chip_temperature();
if (!t) { std::cerr << "get_chip_temperature failed: " << t.status() << "\n"; break; }
auto thr = d.get_throttling_state();
std::time_t now = std::time(nullptr);
char ts[16];
std::strftime(ts, sizeof(ts), "%H:%M:%S", std::localtime(&now));
std::cout << ts << " ts0=" << t->ts0_temperature << " ts1=" << t->ts1_temperature
<< " max=" << std::max(t->ts0_temperature, t->ts1_temperature) << " C";
if (thr) std::cout << " schutz_aktiv=" << (thr.value() ? "ja" : "nein");
std::cout << std::endl;
for (int i = 0; i < interval && running; + +i)
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return 0;
}
The first command line allows you to compile the source code from the file “temp.cpp” into a C program. Use the chmod command to change the file permissions, and then you can run the file.
When the program runs, it displays the current zone. It shows whether the protection feature has been activated for the AI chip and at what temperatures each zone becomes active. Every 5 seconds, the current temperature is displayed based on the two measurement points of the Halio AI chip.
A benchmark test is run for two minutes using the resnet18.hef model.
– …
* Hailo-8 M.2 AI Acceleration Module Webpage Hailo-8 AI Chip
* AI-Models for the Chip AI-Models
AI Agent for an Energy Management