Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
app-notes:ein-mqtt-server-fuer-iot-telemetry-sensoren [2023/11/09 14:17] gray [2.2 Make telemetry data usable via SDK scripts Subscribe] |
app-notes:ein-mqtt-server-fuer-iot-telemetry-sensoren [2024/03/21 14:40] (current) gray [2.1 Collect telemetry data with the help of SDK scripts] |
||
|---|---|---|---|
| Line 19: | Line 19: | ||
| Grafana is used for display and analysis. Here short term and long term evaluations are well visualized for the user. | Grafana is used for display and analysis. Here short term and long term evaluations are well visualized for the user. | ||
| - | |||
| - | For signaling information an XMPP server is used. Over this messages are sent to a cell phone or computer. | ||
| ---- | ---- | ||
| Line 68: | Line 66: | ||
| if(DEBUG) printf("%s\n", MESSAGE); | if(DEBUG) printf("%s\n", MESSAGE); | ||
| /* publish the message to mqtt broker Temperature */ | /* publish the message to mqtt broker Temperature */ | ||
| - | ret = nb_mqtt_publish(HOST, PORT, USERNAME, PASSWORD, TOPIC, QOS, RETAIN, MESSAGE); | + | ret = nb_mqtt_publish(HOST, PORT, USERNAME, PASSWORD, TOPIC, QOS, RETAIN, MESSAGE);; |
| if (ret<0){ | if (ret<0){ | ||
| nb_syslog("Failed to publish mqtt message"); | nb_syslog("Failed to publish mqtt message"); | ||
| Line 92: | Line 90: | ||
| if(DEBUG) printf("%s\n", MESSAGE4); | if(DEBUG) printf("%s\n", MESSAGE4); | ||
| /* publish the message to mqtt broker Sys */ | /* publish the message to mqtt broker Sys */ | ||
| - | ret = nb_mqtt_publish(HOST, PORT, USERNAME, PASSWORD, TOPIC, QOS, RETAIN, MESSAGE4); | + | ret = nb_mqtt_publish(HOST, PORT, USERNAME, PASSWORD, TOPIC, QOS, RETAIN, MESSAGE4); |
| if (ret<0){ | if (ret<0){ | ||
| nb_syslog("Failed to publish mqtt message"); | nb_syslog("Failed to publish mqtt message"); | ||
| Line 130: | Line 128: | ||
| - | + | In this script, the values from the memory cells are also used to calculate the absolute final humidity. | |
| - | + | ||
| - | Under this script, values from the memory cells are used to calculate the absolute humidity. | + | |
| <code c shellyplusht subscribe_SDK.are> | <code c shellyplusht subscribe_SDK.are> | ||
| Line 335: | Line 331: | ||
| ---- | ---- | ||
| - | Output of the unformatted data receive the temperature value. This is taken out of the string and then stored in a memory cell. Additionally the information of the change is stored as datun and time. | + | Below this example you can see how the script takes the temperature value from the data block. |
| <konsole> | <konsole> | ||
| Line 353: | Line 349: | ||
| </konsole> | </konsole> | ||
| - | When all the information is extracted from the unformatted data, a data block of memory cells is obtained. In addition, the absolute humidity is calculated from the data and also stored in the memory cell. | ||
| - | <konsole> | ||
| - | Ausgabe unforamtierten Daten | ||
| - | {"id":123,"src":"shellyplusht-ID","dst":"shellies/shelly-ID","result":{"id": 0,"battery":{"V":0.43, "percent":0},"external":{"present":true}}} | ||
| - | Batteryspannungswert battery ist entalten | ||
| - | Hier wird die laenge des Daten String bestimmt: 162 Zeichen | ||
| - | Neuer Daten String verkleinert: | ||
| - | battery":{"V":0.43, "percent":0},"external":{"present":true}}} | ||
| - | Bestimmen des batteryblock: 18 Zeichen | ||
| - | Ausgabe des batteryblock | ||
| - | battery":{"V":0.43 | ||
| - | Nur der Battery Wert 8 Zeichen | ||
| - | Der Shelly wird über USB-Netzteil versorgt: 0.43 Volt nur. | ||
| - | Ausgabe von custom Speicherzelle_1 Temperatur: 22.5 | ||
| - | Ausgabe von custom Speicherzelle_2 Datum Zeit: 30/10/2023_21:19:00 | ||
| - | Ausgabe von custom Speicherzelle_3 Luftfeuchtewert: 49.3 | ||
| - | Ausgabe von custom Speicherzelle_4 Batteryspannungswert: 0.43 | ||
| - | Ausgabe von custom Speicherzelle_5 Prozentsatz: n/a | ||
| - | Ausgabe von I1: und I2: | ||
| - | float: 22.5 | ||
| - | float: 49.3 | ||
| - | 9.847469223 | ||
| - | Absolutendluftfeuchtigkeit Berechnung: 9.847469223 g/m³ | ||
| - | Ausgabe von custom Speicherzelle_result Absolutendluftfeuchtigkeit: 9.847469223 g/m³ | ||
| - | </konsole> | ||
| ---- | ---- | ||
| - | ===== - Further telemetry data via other ways ===== | ||
| - | |||
| - | Here are further descriptions how telemetry data can be transferred directly to the MQTT broker | ||
| - | |||
| - | * [[configuration:iot-geraete-und-mqtt-server|IoT devices sending data]] | ||
| - | |||
| - | ---- | ||
| - | |||
| - | A further description how a smart meter can send its data to a MQTT broker via the program tool Node-RED. The whole thing was implemented under a LXC container. | ||
| - | |||
| - | |||
| - | * [[configuration:ein-smartmeter-auf-einem-lxc-container|Smartmeter, Node-Red and LXC Container]] | ||
| - | |||
| - | ---- | ||
| ===== - Bring data from the MQTT broker structured into a database ===== | ===== - Bring data from the MQTT broker structured into a database ===== | ||
| - | Create an influx database via the console and add a user for access. | + | An Influx database is created via the console. Only the access and the name of the thank you bank need to be stored. A Python script retrieves the data from the MQTT broker and also builds the structure based on the data. |
| - | + | ||
| - | The structure in the database is then created with a Python script, using a Json format. | + | |
| - | + | ||
| - | <code c > | + | |
| - | json_body = [ | + | |
| - | { | + | |
| - | 'measurement': sensor_data.measurement, | + | |
| - | 'tags': { | + | |
| - | 'location': sensor_data.location | + | |
| - | }, | + | |
| - | 'fields': { | + | |
| - | 'value': sensor_data.value | + | |
| - | } | + | |
| - | } | + | |
| - | ] | + | |
| - | influxdb_client.write_points(json_body) | + | |
| - | </code> | + | |
| - | + | ||
| ---- | ---- | ||
| - | |||
| - | |||
| - | With the help of the Python script, you can specifically query data from the broker and store it in the database in a structured way. | ||
| <code c shellyplusht subscribe_datenbank.py> | <code c shellyplusht subscribe_datenbank.py> | ||
| Line 559: | Line 495: | ||
| ---- | ---- | ||
| - | The Python scripts are started and monitored via a cronjob. | + | A cronjob starts the Python script and monitors whether it is still running. |
| - | + | ||
| - | <konsole> | + | |
| - | + | ||
| - | # Das Skript wird alle 1 Minuten plus 20 sekunden gestartet | + | |
| - | * * * * * sleep 20; /home/korris/plus_ht.sh | + | |
| - | + | ||
| - | </konsole> | + | |
| - | + | ||
| - | The script checks if the process is still running. If it is, it does nothing. If the process was terminated it is started again. The Python scripts run endlessly. The cronjob only checks if they are still running. | + | |
| <konsole> | <konsole> | ||
| Line 603: | Line 530: | ||
| From now on you have access to the data that is already in the database and that will be added over time. | From now on you have access to the data that is already in the database and that will be added over time. | ||
| - | Which measurements have been deposited in the database: | ||
| - | |||
| - | <konsole> | ||
| - | :~ $ influx | ||
| - | Connected to http://localhost:8086 version 1.8.10 | ||
| - | InfluxDB shell version: 1.8.10 | ||
| - | :> use Shelly | ||
| - | Using database Shelly | ||
| - | :> show series | ||
| - | key | ||
| - | --- | ||
| - | power,location=shellyplug-s-C8C9A3B8E8C6 | ||
| - | rh,location=shelly-08b61fcf3380 | ||
| - | tC,location=shelly-08b61fcf3380 | ||
| - | tC,location=shellydw2-73BBC6 | ||
| - | tC,location=shellyflood-244CAB430495 | ||
| - | temperatur,location=shellyplug-s-80646F81A273 | ||
| - | temperatur,location=shellyplug-s-80646F81AD26 | ||
| - | temperatur,location=shellyplug-s-80646F81F06B | ||
| - | temperatur,location=shellyplug-s-C8C9A3A4DE17 | ||
| - | value,location=shellydw2-73BBC6 | ||
| - | value,location=shellyflood-244CAB430495 | ||
| - | voltage,location=shellydw2-73BBC6 | ||
| - | voltage,location=shellyflood-244CAB430495 | ||
| - | </konsole> | ||
| ==== - The first visualization under Grafana ==== | ==== - The first visualization under Grafana ==== | ||
| Line 651: | Line 553: | ||
| With this function Grafana Discover I would try more function under the programs. One comes here quickly on solutions that are helpful in the implementation in the dashboard. | With this function Grafana Discover I would try more function under the programs. One comes here quickly on solutions that are helpful in the implementation in the dashboard. | ||
| - | |||
| - | If one is then with the conversion for a dashboard, one can fall back on information here. | ||
| - | |||
| - | {{:configuration:configuration:granfana_dashboard.png|}} | ||
| - | |||
| - | ---- | ||
| - | |||
| - | ===== - Pass on information from the MQTT network to a free messenger. ===== | ||
| - | |||
| - | An XMPP server is used as the messenger. Because a standardized open protocol is used. There is an XMPP client for almost everything. | ||
| - | |||
| - | On the LXC container a XMPP Cleint is used under the command line. With it you can send messages over the XMPP protocol. | ||
| - | |||
| - | [[https://manpages.ubuntu.com/manpages/focal/en/man1/sendxmpp.1p.html|link Manpage sendxmpp]] | ||
| - | |||
| - | |||
| - | If the command line program is installed on the container you can test it with these commands. It is important that the user of ''sendxmpp'' is also created on the XMPP server with password. The sendxmpp program is not an IM client. It can only send messages but not receive them. | ||
| - | |||
| - | <konsole> | ||
| - | echo 'Hello World' | sendxmpp -n -t -u "XMPP_Benutzer_Sender" -j Server_Adresse | ||
| - | -p Passwort_vom_Benutzer_Sender Benutzer_Empfänger@xmpp_Server_Adresse | ||
| - | </konsole> | ||
| - | |||
| - | The LXC container runs a TCP server under a Python script. | ||
| - | |||
| - | <code py tcp_server tcp-server.py> | ||
| - | # a simple tcp server | ||
| - | import socket,os | ||
| - | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | ||
| - | sock.bind(('192.168.143.2', 60100)) | ||
| - | sock.listen(5) | ||
| - | while True: | ||
| - | connection,address = sock.accept() | ||
| - | buf = connection.recv(1024) | ||
| - | if (str(buf)) != "b''": | ||
| - | # print("ja") | ||
| - | file = open('xmpp_text.txt','w') | ||
| - | temp = str(buf, 'utf-8') | ||
| - | file.write(str(temp)) | ||
| - | # print(file.write(str(buf))) | ||
| - | file.close() | ||
| - | # else: | ||
| - | # print("ist gleich") | ||
| - | connection.send(buf) | ||
| - | connection.close() | ||
| - | </code> | ||
| - | |||
| - | ---- | ||
| - | |||
| - | This waits for messages generated by SDK script on the router. New messages are then stored in a file "xmpp_text.txt". | ||
| - | |||
| - | About a cronjob every 10 seconds a script is started. This checks whether a new message is available. If so, this message is sent via xmpp. | ||
| - | |||
| - | <konsole> | ||
| - | :~# crontab -e | ||
| - | |||
| - | # Das Skript wird alle 10 Sekunden gestartet | ||
| - | * * * * * /root/xmpp_skript.sh | ||
| - | * * * * * sleep 10; /root/xmpp_skript.sh | ||
| - | * * * * * sleep 20; /root/xmpp_skript.sh | ||
| - | * * * * * sleep 30; /root/xmpp_skript.sh | ||
| - | * * * * * sleep 40; /root/xmpp_skript.sh | ||
| - | * * * * * sleep 50; /root/xmpp_skript.sh | ||
| - | </konsole> | ||
| - | |||
| - | The cronjob script: | ||
| - | |||
| - | <code sh xmpp-nachricht xmpp_skript.sh> | ||
| - | #!/bin/bash | ||
| - | if [ -r /root/xmpp_text.txt ] | ||
| - | then | ||
| - | sendxmpp -m /root/xmpp_text.txt -n -t -u "benutzer" -j server -p passwort empfänger@server | ||
| - | sleep 1s | ||
| - | rm /root/xmpp_text.txt | ||
| - | fi | ||
| - | </code> | ||
| ---- | ---- | ||