====== Netmodule Routers with an Hailo-8 Module ======
===== Goal =====
This wiki shows how to enable an Hailo-8 M.2 AI Acceleration Module in the router and use it within an LXC container. It also explains how to set up additional utilities within the container.
===== The background to this project =====
Enable the router to run its own AI modules locally, directly on the router, without having to use a hyperscaler. Your own data can be used to train the AI modules because it is not shared with third parties. The entire value chain remains under your control and can be flexibly developed further. Applications include intelligent energy management systems, dynamic traffic control systems, and production management. The AI modules can be trained and fine-tuned for their core tasks to achieve optimal results.
----
=== Modules in the router ===
{{:app-notes:hailo8_chip.jpg?800|}}
----
===== Prerequisites =====
* NetModule NB2800/NB2810 Wireless Router with wireless connection
* NetModule Software with Hailo-8 AI Chip Support for NB2800 [[https://files.netmodule.com/share/vhtlzzda|NB2800]] Password for the download: ''"ai-chip"''
* NetModule Software with Hailo-8 AI Chip Support for NB2810 [[https://files.netmodule.com/share/gbse0zuK|NB2810]] Password for the download: ''"ai-chip"''
* Not available for legacy products NB1600, NB2700, NB3700, NB3710
The wiki was tested with router software nrsw-image-2800-0.0.0.0.img and the LXC container with debina 13 armhf (trixie)
----
===== - Installing the Module into the Router Hardware =====
An adapter is required for the Hailo-8 AI chip to be used with the router. The PCI Express interfaces are implemented as Mini PCIe on the router board. The Hailo-8 AI chip requires a Mini PCIe to M.2 M-key adapter.
{{:app-notes:hailo8_modul.jpg?800|}}
The NB2800's circuit board features a DC/DC converter that supplies 3.3 volts to the two Mini-PCIe slots. The maximum output current is 6 amps. A maximum of 12 watts is available for the Mini-PCIe modules. The Hailo-8 AI chip has a maximum power consumption of less than 5 watts.
The module can be installed in slot 1 or 2 on the NB2800.
{{:app-notes:ki_pcie.png?300|}}
The Hailo-8 AI chip was inserted into the first slot
To ensure communication between the chip and the router, the module must be configured as Wi-Fi in the BD.
=== BDINFO ===
bd/0/pd_module0: slot=0,caps=wlan-main:wlan-aux
bd/0/pd_module1: slot=1,caps=wlan-main:wlan-aux
bd/0/pd_module2: slot=2,caps=wwan-main:wwan-aux:gnss-active:voice
You can check whether the module is present via the router's console.
lspci
02:00.0 Class 0b40: 1e60:2864
The module was detected in the router: “02:00.0” (device PCI address: bus 02, device 00, function 0).
The module’s device class was correctly identified as “Class 0b40” (DSP/signal processing controller).
The last two entries come from the “lspci/pci.ids” database: 1e60 Hailo Technologies and device ID 2864 Hailo-8 AI Processor.
----
===== - Setting Up the Router Software on the Router =====
The first step is to download the software that includes driver support for the Hailo-8 AI chip. To do this, click the link ''[[https://files.netmodule.com/share/vhtlzzda|NB2800]]'' or ''[[https://files.netmodule.com/share/gbse0zuK|NB2810]]'' to download the appropriate software for the two respective routers.
{{:app-notes:ki_router_software.png?800|}}
The password for access is ''“ai-chip”''
Save the downloaded router software to the ''“/tmp”'' subdirectory on the router, and then connect to the router via the console using ''SSH'' or the Windows program ''PuTTY''.
BusyBox v1.35.0 () built-in shell (ash)
~ $ cd /tmp/
/tmp $ swupdate -fr nrsw-image-2800-0.0.0.0.img
This is swupdate (version 3.0) speaking
Will force things
root0 is currently our root partition
========== system info ==================
Model NB2800
Profile: NETBOLT
ROOTFS: 0.0.0.0 (20260806194249)
UBOOT: 5.0.0.100
LOGIC: 5.0.0.100 (1 file)
PATCH: none
Forcing installation of an unsigned image.
You can install this image on the router using the command ''swupdate -fr nrsw-image-2800-0.0.0.0.img''.
==== Next, set up a mobile (WWAN) connection on the router ====
In this example, I've set it up for a Telekom SIM card.
{{:app-notes:ki_wwan.png?800|}}
You can use a ''ping'' test to check whether the router can establish an Internet connection via the WWAN connection.
~ $ ping quad9.net
PING quad9.net (195.176.255.138): 56 data bytes
64 bytes from 195.176.255.138: seq=0 ttl=52 time=25.844 ms
64 bytes from 195.176.255.138: seq=1 ttl=52 time=39.836 ms
64 bytes from 195.176.255.138: seq=2 ttl=52 time=53.615 ms
----
===== - Set up the LXC Contianer =====
Activate the LXC container on the router
{{:pictures:lxc_aktiv.png?800|}}
In the second step, the image of the LXC container in this case, a Debina 13 (trixie).
{{:pictures:lxc_install.png?800|}}
The easiest way to do this is to install the whole thing over the Internet. However, the router requires a mobile connection to the Internet.
From a Jenkins server you get the current image for the container
* Jenkins Server Page image-debian / [[https://jenkins.linuxcontainers.org/view/Images/job/image-debian/|Link]]
Here you can download a Debian image for various systems and versions.
The ''armhf'' tab is important here, and ''trixie''
{{:app-notes:ki_lxc_trixie.png?800|}}
The image is the ''“rootfs.tar.xz”'' which is required. With a right click you can copy the link.
{{:app-notes:ki_trixie_image.png?800|}}
It would be structured like this:
https://jenkins.linuxcontainers.org/view/Images/job/image-debian/architecture=armhf,release=trixie,variant=default/4118/artifact/rootfs.tar.xz
When the ''“Apply”'' button is pressed, Debian will be installed.
Set up the LXC container further. A network interface is enabled here.
{{:pictures:lxc_netzwerk.png?800|}}
The container has now also been assigned an IP address.
~ $ lxc-info guest0
Name: guest0
State: RUNNING
PID: 2255
IP: 192.168.1.149
CPU use: 1.57 seconds
BlkIO use: 4.00 KiB
Memory use: 6.29 MiB
KMem use: 1.18 MiB
Link: virt0
TX bytes: 2.28 KiB
RX bytes: 853 bytes
Total bytes: 3.11 KiB
~ $
Always assign a fixed IP address for the LXC container via the DHCP server.
The MAC address of the LXC container is ''00:FF:AA:00:44:00''.
{{:pictures:lxc_ip_adresse.png?800|}}
Log in to the LXC container and test whether it can access the Internet.
~ $ lxc-attach guest0
root@LXCNAME:~# ping quad9.net
PING quad9.net (195.176.255.138) 56(84) bytes of data.
64 bytes from 195.176.255.138 (195.176.255.138): icmp_seq=1 ttl=51 time=37.0 ms
64 bytes from 195.176.255.138 (195.176.255.138): icmp_seq=2 ttl=51 time=40.9 ms
--- quad9.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 28.823/36.685/40.937/4.768 ms
root@LXCNAME:~#
The container is installed and set up. Now you can set up further tools on the container.
----
==== Adjust the LXC config so that the USB port can be used in the container ====
Enable USB administration on the router
{{:app-notes:lxc_usb_port.png?800|}}
When the USB device is plugged in, it is mounted by the router and stored in the system as device part ''“/dev/sda1"''.
Feb 12 09:13:38.515 NB2800 kern.info kernel: sda: sda1
Feb 12 09:13:38.515 NB2800 kern.notice kernel: sd 2:0:0:0: [sda] Attached SCSI removable disk
Feb 12 09:13:38.585 NB2800 local1.notice NB2800: add event for usb device /dev/sda
Feb 12 09:13:38.596 NB2800 user.notice event-manager[1831]: received event 'USB storage device has been added'
Feb 12 09:13:38.667 NB2800 local1.notice NB2800: add event for usb device /dev/sda1
Feb 12 09:13:38.671 NB2800 local1.notice NB2800: device add - device = /dev/sda1
Feb 12 09:13:38.675 NB2800 local1.notice NB2800: device add - type = usb
Feb 12 09:13:38.678 NB2800 local1.notice NB2800: device add - uid = 855A-2532
Feb 12 09:13:38.690 NB2800 user.debug : Add-event for (type=usb, device=/dev/sda1, id=855A-2532)
Feb 12 09:13:38.692 NB2800 user.debug : storage-add: device (path=/dev/sda1, uid=855A-2532) appeared
In order to release the device in the container, I need to know what kind of device it is: ''“b”'' is a block device, major number ''"8"'' and minor number ''“1”''.
{{:app-notes:lxc_device.png?700|}}
With this information, we can adjust the LXC configuration so that we have access to the device from within the container.
This change is made directly in the router software. When updating the router software, this entry must be made again because it is not part of the official software.
Add these entries to the end of the ''config'' file.
~ $ vi /etc/lxc/guest0/config
# Setting up a USB flash drive in the LXC container
lxc.cgroup.devices.allow = b 8:1 rwm
lxc.mount.entry = /dev/sda1 /dev/sda1 none bind,optional,create=file
Simply add these four entries to the LXC configuration file. Please ensure that you fill them in with your own parameters. ''Restart the router'' for these changes to take effect.
Now the USB device can be connected to the container and used.
The ''“mkdir”'' command is used to set up an area in the file system for access.In the second step, change the permissions of the file.
~ $ lxc-attach guest0
$ mkdir -p /mnt/home/data
$ chgrp -R users /mnt/home/
$ chmod -R 770 /mnt/home/
Check whether the rights have been set.
/mnt/home $ ls -l
total 4
drwxrwx--- 2 root users 4096 Aug 11 14:19 data
/mnt/home $
----
===== - Set up the Hailo-8 AI chip =====
Test this after rebooting the router to see if the interface is present in the router.
{{:app-notes:hailo8_schnittstelle.jpg?800|}}
In the next step, the interface must be passed to the LXC container.
~ $ vi /etc/lxc/guest0/config
# Allow Access (Device 246)
lxc.cgroup.devices.allow = c 246:0 rwm
# Mount the Hailo-8 device node in the container
lxc.mount.entry = /dev/hailo0 dev/hailo0 none bind,create=file 0 0
If the “hailo0” interface under the LXC container is set to mode “(000),” you must check the settings in lxc config again to make sure they are correct.
root@LXCNAME:~# ls -l /dev/ | grep dx*
crw------- 1 root root 246, 0 Oct 29 01:00 hailo0
----
==== The next thing you'll need is the HailoRT runtime software ====
The runtime software HailoRT was built for Debian 13 (trixie) using software version 4.24.0.
Download the runtime software file ''“hailort-4.24.0-armhf.tar.gz”''
* Download the file ''“hailort-4.24.0-armhf.tar.gz”'' from the link [[https://files.netmodule.com/share/hK3aMNuN|tar file]] Password for the download: ''"ai-chip"''
Place the ''“.tar”'' file on the LXC container and then extract it directly there. After the library directories have been rebuilt using ''ldconfig'', you can view what has changed in terms of access.
root@LXCNAME:~# cd /root
root@LXCNAME:~# tar -xzf hailort-4.24.0-armhf.tar.gz -C /
root@LXCNAME:~# ldconfig
root@LXCNAME:~# ldd /usr/local/bin/hailortcli
linux-vdso.so.1 (0xbeff8000)
libhailort.so.4.24.0 => /usr/local/lib/libhailort.so.4.24.0 (0xb69a0000)
libstdc++.so.6 => /lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb67f0000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6780000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6740000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6610000)
/lib/ld-linux-armhf.so.3 (0xb6f4e000)
In the second step, we test the Hailor CLI to ensure it is functioning properly.
Run ''“hailortcli –version”'' to check the installed version.
Verify the presence of the ''“hailo0”'' interface from within the LXC container using ''“ls -l /dev/hailo0”''.
The CLI command ''“hailortcli scan”'' locates the device on the PCIe bus.
Direct access to the AI chip from within the container is achieved using the command ''“hailortcli fw-control identify”''.
root@LXCNAME:~# hailortcli --version
HailoRT-CLI version 4.24.0
root@LXCNAME:~# ls -l /dev/hailo0
crw------- 1 root root 246, 0 Oct 29 00:00 /dev/hailo0
root@LXCNAME:~# hailortcli scan
Hailo Devices:
[-] Device: 0000:02:00.0
root@LXCNAME:~# hailortcli fw-control identify
Executing on device: 0000:02:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.24.0 (release,app,extended context switch buffer)
Logger Version: 0
Device Architecture: HAILO8
Serial Number: HLDDM2A234501431
Part Number: HM218B1C2FAE
Product Name: HAILO-8 AI ACC M.2 M KEY MODULE EXT TEMP
root@LXCNAME:~#
----
==== A prototype of the Hailo AI chip ====
If the CLI for the Hailo AI chip ran without any issues, you can now test the whole thing using an AI model.
A pre-compiled neural network for the Hailo-8 chip was used as the model. It contains images representing approximately 1,000 classes. The model was created using JupyterLab as a ''“.hef”'' file. The finished model, ''“resnet18.hef"'', can be downloaded from the link below.
* Download the file ''“resnet18.hef”'' from the link [[https://files.netmodule.com/share/LIjnJcke|hef file]] Password for the download: ''"ai-chip"''
To monitor the temperature of the Hailo chip, I use a small C program with a library from Hailo. This allows the chip to be monitored even further while the AI model is running. To compile it, all you need is ''g+ +'' on the container.
apt install g+ +
Create these lines of code in the file temp.cpp and then compile them into a C program.
#include
#include
#include
#include
using namespace hailort;
static volatile std::sig_atomic_t running = 1;
static void on_sig(int) { running = 0; }
// This option is selected only if get_health_information() is publicly accessible
template
auto print_health(D &d, int) -> decltype(d.get_health_information(), void()) {
auto h = d.get_health_information();
if (!h) { std::cerr << "get_health_information failed: " << h.status() << "\n"; return; }
std::cout << "Zone: " << (int)h->current_temperature_zone
<< " (0=gruen, 1=orange, 2=rot)"
<< ", Temperatur-Throttling aktiv: " << h->temperature_throttling_active
<< ", Overcurrent-Schutz aktiv: " << h->overcurrent_protection_active << "\n"
<< "orange: " << h->orange_temperature_threshold
<< " C (Hysterese " << h->orange_hysteresis_temperature_threshold << ")"
<< ", rot: " << h->red_temperature_threshold
<< " C (Hysterese " << h->red_hysteresis_temperature_threshold << ")\n";
}
template
void print_health(D &, long) {
std::cout << "Exceptions: get_health_information() is not public.\n";
}
int main(int argc, char **argv) {
int interval = (argc > 1) ? std::atoi(argv[1]) : 5;
if (interval < 1) interval = 5;
std::signal(SIGINT, on_sig);
std::signal(SIGTERM, on_sig);
auto dev = Device::create();
if (!dev) { std::cerr << "Device::create failed: " << dev.status() << "\n"; return 1; }
Device &d = *dev.value();
print_health(d, 0);
while (running) {
auto t = d.get_chip_temperature();
if (!t) { std::cerr << "get_chip_temperature failed: " << t.status() << "\n"; break; }
auto thr = d.get_throttling_state();
std::time_t now = std::time(nullptr);
char ts[16];
std::strftime(ts, sizeof(ts), "%H:%M:%S", std::localtime(&now));
std::cout << ts << " ts0=" << t->ts0_temperature << " ts1=" << t->ts1_temperature
<< " max=" << std::max(t->ts0_temperature, t->ts1_temperature) << " C";
if (thr) std::cout << " schutz_aktiv=" << (thr.value() ? "ja" : "nein");
std::cout << std::endl;
for (int i = 0; i < interval && running; + +i)
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return 0;
}
The first command line allows you to compile the source code from the file “temp.cpp” into a C program.
Use the chmod command to change the file permissions, and then you can run the file.
g+ + -std=c+ +17 temp.cpp -I/usr/local/include -L/usr/local/lib -lhailort -o temp
chmod a+x temp
./temp 5 | tee temp.log
Zone: 0 (0=gruen, 1=orange, 2=rot), Temperatur-Throttling aktiv: 1, Overcurrent-Schutz aktiv: 1
orange: 104 C (Hysterese 99), rot: 120 C (Hysterese 116)
15:32:23 ts0=32.6116 ts1=33.0163 max=33.0163 C schutz_aktiv=ja
15:32:28 ts0=32.5104 ts1=33.1175 max=33.1175 C schutz_aktiv=ja
When the program runs, it displays the current zone. It shows whether the protection feature has been activated for the AI chip and at what temperatures each zone becomes active. Every 5 seconds, the current temperature is displayed based on the two measurement points of the Halio AI chip.
A benchmark test is run for two minutes using the resnet18.hef model.
hailortcli benchmark resnet18.hef --time-to-run 120 & for i in 1 2 3 4 5 6; do ./temp; sleep 20; done; wait
[1] 156
Zone: 0 (0=gruen, 1=orange, 2=rot), Temperatur-Throttling aktiv: 1, Overcurrent-Schutz aktiv: 1
orange: 104 C (Hysterese 99), rot: 120 C (Hysterese 116)
07:13:35 ts0=17.5369 ts1=17.9416 max=17.9416 C schutz_aktiv=ja
Starting Measurements...
Measuring FPS in HW-only mode
07:13:40 ts0=20.3697 ts1=20.7239 max=20.7239 C schutz_aktiv=ja
07:13:45 ts0=21.2803 ts1=21.7862 max=21.7862 C schutz_aktiv=ja
07:13:50 ts0=21.7862 ts1=22.4438 max=22.4438 C schutz_aktiv=ja
07:13:55 ts0=22.0897 ts1=22.7473 max=22.7473 C schutz_aktiv=ja
07:14:00 ts0=22.4944 ts1=23.152 max=23.152 C schutz_aktiv=ja
07:14:05 ts0=22.6967 ts1=23.3037 max=23.3037 C schutz_aktiv=ja
07:14:10 ts0=22.7979 ts1=23.5061 max=23.5061 C schutz_aktiv=ja
07:14:15 ts0=22.8991 ts1=23.6579 max=23.6579 C schutz_aktiv=ja
07:14:20 ts0=23.0002 ts1=23.8602 max=23.8602 C schutz_aktiv=ja
07:14:25 ts0=23.2026 ts1=23.8602 max=23.8602 C schutz_aktiv=ja
07:14:30 ts0=23.2532 ts1=24.012 max=24.012 C schutz_aktiv=ja
07:14:35 ts0=23.3543 ts1=24.1131 max=24.1131 C schutz_aktiv=ja
07:14:40 ts0=23.4049 ts1=24.2143 max=24.2143 C schutz_aktiv=ja
07:14:45 ts0=23.5567 ts1=24.2143 max=24.2143 C schutz_aktiv=ja
07:14:50 ts0=23.6073 ts1=24.3155 max=24.3155 C schutz_aktiv=ja
07:14:55 ts0=23.6579 ts1=24.4672 max=24.4672 C schutz_aktiv=ja
07:15:00 ts0=23.759 ts1=24.5178 max=24.5178 C schutz_aktiv=ja
07:15:05 ts0=23.8602 ts1=24.619 max=24.619 C schutz_aktiv=ja
07:15:10 ts0=23.9108 ts1=24.6696 max=24.6696 C schutz_aktiv=ja
07:15:15 ts0=23.9108 ts1=24.7202 max=24.7202 C schutz_aktiv=ja
07:15:20 ts0=24.0625 ts1=24.7202 max=24.7202 C schutz_aktiv=ja
07:15:25 ts0=24.0625 ts1=24.8213 max=24.8213 C schutz_aktiv=ja
07:15:30 ts0=24.2143 ts1=24.9225 max=24.9225 C schutz_aktiv=ja
07:15:35 ts0=24.1637 ts1=24.9225 max=24.9225 C schutz_aktiv=ja
Network resnet18/resnet18: 100% | 155385 | FPS: 1294.80 | ETA: 00:00:00
Measuring FPS (and Power on supported platforms) in streaming mode
----
===== - Additional information =====
* Hailo-8 M.2 AI Acceleration Module [[https://hailo.ai/products/ai-accelerators/hailo-8-m2-ai-acceleration-module/|Webpage Hailo-8 AI Chip]]
* AI-Models for the Chip [[https://github.com/hailo-ai/hailo_model_zoo_genai AI-Models|AI-Models]]
[[app-notes:ai-agent|AI Agent for an Energy Management]]
----