From 38dcd7310062662ac8584ce671d004c39d9f153c Mon Sep 17 00:00:00 2001 From: Joachim Hummel Date: Fri, 14 Nov 2025 11:43:03 +0000 Subject: [PATCH] Add MQTT/OwnTracks workflow for location tracking MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add tracker-mqtt.json: New n8n workflow for MQTT-based location data - MQTT workflow subscribes to owntracks/# topic - Transforms OwnTracks data format to NocoDB schema - Stores locations in same database as Telegram workflow - Update CLAUDE.md with MQTT workflow documentation - Add OwnTracks integration guide and data mapping details 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- CLAUDE.md | 207 ++++++++++++++++++++++++++++++++++++++-------- tracker-mqtt.json | 185 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+), 34 deletions(-) create mode 100644 tracker-mqtt.json diff --git a/CLAUDE.md b/CLAUDE.md index dfef554..a3e5cd6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,36 +4,80 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Overview -This repository contains an n8n workflow configuration (`tracker.json`) that implements a Telegram-based location tracking system without a database. The workflow stores location data in a simple JSON file (`/tmp/n8n-locations.json`) on the n8n server. +This repository contains three n8n workflow configurations for location tracking: + +1. **tracker.json** - Telegram-based with file storage using `/tmp/n8n-locations.json` (simpler, no database required) +2. **tracker-db.json** - Telegram-based with NocoDB storage (production-ready, persistent storage) +3. **tracker-mqtt.json** - MQTT-based with NocoDB storage (for OwnTracks/MQTT location data) + +The Telegram workflows share the same architecture but differ in storage backend. The MQTT workflow integrates OwnTracks-compatible location data into the same NocoDB database. ## Workflow Architecture -The workflow has two main execution paths: +### Telegram-based Workflows -### 1. Location Capture Flow (Telegram Trigger → Storage) +Both Telegram workflows have two main execution paths: + +**1. Location Capture Flow (Telegram Trigger → Storage)** + +tracker.json (File-based): +- Telegram Trigger → Hat Location? → Location verarbeiten → Lade existierende Daten → Merge mit History → Speichere in File → Telegram Bestätigung + +tracker-db.json (NocoDB-based): +- Telegram Trigger → Hat Location? → Location verarbeiten → Speichere in NocoDB → [Hole letzten Eintrag + Zähle Einträge] → Merge → Bereite Bestätigung vor → Telegram Bestätigung + +Key nodes: - **Telegram Trigger**: Receives incoming Telegram messages - **Hat Location?**: Filters messages containing location data -- **Location verarbeiten**: Extracts and formats location data (lat/lon, user info, timestamp) -- **Lade existierende Daten**: Reads existing locations from `/tmp/n8n-locations.json` -- **Merge mit History**: Combines new location with existing data (keeps last 100 entries) -- **Speichere in File**: Writes updated JSON back to `/tmp/n8n-locations.json` -- **Telegram Bestätigung**: Sends confirmation message with location details and map link +- **Location verarbeiten**: Extracts and formats location data (lat/lon, user info, timestamp) using JavaScript +- **Storage**: Either file-based (shell commands) or NocoDB API calls +- **Telegram Bestätigung**: Sends confirmation with location details and map link -### 2. Location API Flow (Webhook → JSON Response) -- **Webhook - Location API**: Exposes `/location` endpoint with CORS enabled -- **Lade Daten für API**: Reads location data from file -- **Format API Response**: Formats data into structured JSON (current location, history, metadata) -- **JSON Response**: Returns JSON with CORS headers +**2. Location API Flow (Webhook → JSON Response)** - See section below + +### MQTT-based Workflow + +**tracker-mqtt.json** has a simpler, single-path architecture: + +MQTT Trigger → Ist Location? → MQTT Location verarbeiten → Speichere in NocoDB + +Key nodes: +- **MQTT Trigger**: Subscribes to MQTT topic `owntracks/#` (OwnTracks-compatible) +- **Ist Location?**: Filters for messages with `_type: "location"` +- **MQTT Location verarbeiten**: Transforms MQTT/OwnTracks data format to match NocoDB schema +- **Speichere in NocoDB**: Stores location in same database as Telegram data + +### Location API Flow (Shared) + +**tracker.json (File-based)**: +- Webhook - Location API → Lade Daten für API (shell: cat) → Format API Response → JSON Response + +**tracker-db.json (NocoDB-based)**: +- Webhook - Location API → Lade Daten aus NocoDB → Format API Response → JSON Response + +Both expose `/location` endpoint with CORS enabled, returning current location, history, and metadata. The MQTT workflow shares the same NocoDB database and thus the same API. ## Key Technical Details ### Data Storage -- Storage location: `/tmp/n8n-locations.json` + +**tracker.json (File-based)**: +- Location: `/tmp/n8n-locations.json` - Format: JSON array of location objects - Max retention: 100 most recent locations (oldest automatically removed) -- Data persistence: File-based (survives n8n restarts but may be lost on system restart due to `/tmp` location) +- Persistence: Survives n8n restarts but may be lost on system restart due to `/tmp` location + +**tracker-db.json & tracker-mqtt.json (NocoDB)**: +- Location: NocoDB database (Project: `pdxl4cx4dbu9nxi`, Table: `m8pqj5ixgnnrzkg`) +- Format: NocoDB records (no client-side limit) +- Persistence: Full database persistence +- API: Uses n8n's NocoDB node with token authentication (ID: `6fNBtcghMe8wFoE5`) +- Both workflows write to the same database table + +### Location Object Structure (NocoDB Schema) + +The NocoDB database uses the following schema for all location sources: -### Location Object Structure ```json { "latitude": number, @@ -49,6 +93,20 @@ The workflow has two main execution paths: } ``` +**Data Mapping for MQTT/OwnTracks**: +- `latitude` ← `lat` +- `longitude` ← `lon` +- `timestamp` ← `tst` (Unix timestamp converted to ISO 8601) +- `user_id` ← `0` (MQTT has no user ID) +- `first_name` ← `tid` (tracker ID, e.g., "le") +- `last_name` ← `source` (e.g., "fused") +- `username` ← `tid` +- `marker_label` ← Combination of `tid` and optionally `SSID` +- `display_time` ← Formatted timestamp +- `chat_id` ← `0` (MQTT has no chat ID) + +Additional MQTT data (accuracy, altitude, battery, velocity, etc.) is available in the transformation node but not stored in the database. + ### API Response Structure ```json { @@ -61,34 +119,115 @@ The workflow has two main execution paths: ``` ### Web Interface -The workflow sends users a link to `https://web.unixweb.home64.de/tracker/index.html` for viewing locations on a map. This frontend is hosted separately and not included in this repository. +The workflow sends users a link to `https://web.unixweb.home64.de/tracker/index.html` for viewing locations on a map. + +The `index.html` file in this repository provides a standalone web interface: +- **Map Library**: Leaflet.js (loaded from CDN) +- **API Endpoint**: Configured to `https://n8n.unixweb.eu/webhook/location` (index.html:85) +- **Features**: Auto-refresh (5 second interval), location history polyline, marker popups +- **Hosting**: Can be hosted on any web server or opened as a local file ## Workflow Configuration -- **Telegram Credentials**: Uses "Telegram account n8n-munich-bot" (ID: dRHgVQKqowQHIait) +### Shared Configuration (Both Workflows) +- **Telegram Credentials**: "Telegram account n8n-munich-bot" (ID: `dRHgVQKqowQHIait`) - **Webhook IDs**: - Telegram trigger: `telegram-location-webhook` - Location API: `location-api-endpoint` -- **Error Workflow**: ID `PhwIkaqyXRasTXDH` (configured but not included in this export) - **Execution Order**: v1 - **Caller Policy**: workflowsFromSameOwner +- **Status**: Both workflows set to `active: true` by default -## Modifying the Workflow +### tracker.json Specific +- **Storage**: Shell commands (cat, echo) for file I/O +- **Error Workflow**: ID `PhwIkaqyXRasTXDH` (configured but not in this export) -When editing this workflow: -1. The workflow is active by default - test changes carefully to avoid disrupting live tracking -2. JavaScript code nodes use n8n's code execution environment (not vanilla Node.js) -3. Shell commands execute in n8n's runtime environment - ensure `/tmp` is writable -4. CORS is configured for `*` (all origins) - restrict if needed for security -5. Date formatting uses `de-DE` locale - adjust if different locale needed -6. The 100-entry limit prevents unbounded growth - increase if more history is needed +### tracker-db.json Specific +- **NocoDB Credentials**: "NocoDB Token account" (ID: `6fNBtcghMe8wFoE5`) +- **Project ID**: `pdxl4cx4dbu9nxi` +- **Table ID**: `m8pqj5ixgnnrzkg` +- **Sort Order**: Uses array structure `[{field: "timestamp", direction: "desc"}]` for sorting locations -## Deployment +### tracker-mqtt.json Specific +- **MQTT Credentials**: Requires MQTT broker credentials (placeholder ID: `MQTT_CREDENTIAL_ID`) +- **Topic**: `owntracks/#` (subscribes to all OwnTracks topics) +- **Message Filter**: Only processes messages with `_type: "location"` +- **NocoDB Config**: Same as tracker-db.json (shares database) +- **Status**: Set to `active: false` by default (activate after configuring MQTT credentials) -To use this workflow: -1. Import `tracker.json` into n8n instance -2. Configure Telegram bot credentials -3. Ensure n8n has write permissions to `/tmp/n8n-locations.json` +## Modifying the Workflows + +### Important Considerations +1. **Both workflows are active by default** - test changes carefully to avoid disrupting live tracking +2. **JavaScript code nodes** use n8n's execution environment (not vanilla Node.js) +3. **Date formatting** uses `de-DE` locale - change in "Location verarbeiten" node if needed +4. **CORS** is configured for `*` (all origins) - restrict for production security + +### tracker.json Specific +- Shell commands execute in n8n's runtime environment - ensure `/tmp` is writable +- The 100-entry limit prevents unbounded growth - adjust in "Merge mit History" node +- Consider moving from `/tmp` to persistent storage for production (see README.md) + +### tracker-db.json & tracker-mqtt.json Specific +- NocoDB sorting requires array structure: `[{field: "timestamp", direction: "desc"}]` +- No client-side entry limit (relies on database capacity) +- Requires valid NocoDB credentials and accessible database +- Both workflows write to the same NocoDB table + +### tracker-mqtt.json Specific +- Requires MQTT broker configuration before activation +- Topic pattern can be adjusted to match your MQTT setup +- Data transformation maps OwnTracks format to NocoDB schema +- MQTT data fields (accuracy, battery, velocity) are extracted but not persisted to database + +### Common Modifications + +**Change history limit (tracker.json only)**: +In "Merge mit History" node, change `locations.slice(0, 100)` to desired limit + +**Change date format**: +In "Location verarbeiten" node, change `.toLocaleString('de-DE')` to desired locale + +**Restrict CORS**: +In "Webhook - Location API" node, change `Access-Control-Allow-Origin: *` to specific domain + +**Update web interface URL**: +In "Telegram Bestätigung" node and `index.html:85`, update API endpoint URL + +## Repository Contents + +- **tracker.json** - Telegram with file-based storage (simple, no database) +- **tracker-db.json** - Telegram with NocoDB storage (production-ready) +- **tracker-mqtt.json** - MQTT/OwnTracks with NocoDB storage (IoT devices) +- **index.html** - Leaflet.js web interface for map visualization +- **locations-example.csv** - Example data format for testing +- **README.md** - Detailed German documentation with setup and usage instructions + +## MQTT/OwnTracks Integration + +The **tracker-mqtt.json** workflow is designed for OwnTracks-compatible MQTT location data. + +### Setup Requirements +1. Configure MQTT broker credentials in n8n +2. Update the credential ID in tracker-mqtt.json (currently placeholder: `MQTT_CREDENTIAL_ID`) +3. Import workflow into n8n 4. Activate the workflow -5. Send a location via Telegram to test the capture flow -6. Access the webhook endpoint to verify API functionality + +### OwnTracks Configuration +Configure your OwnTracks app/device to publish to the same MQTT broker: +- **Topic pattern**: `owntracks/user/device` (workflow subscribes to `owntracks/#`) +- **Mode**: MQTT +- **Expected message format**: JSON with `_type: "location"` + +### Data Flow +1. OwnTracks device publishes location to MQTT broker +2. n8n MQTT trigger receives message +3. Filter checks for `_type: "location"` +4. Data transformation maps MQTT fields to NocoDB schema +5. Location stored in same database as Telegram locations +6. Available via same `/location` API endpoint + +### Distinguishing Data Sources +In the database: +- **Telegram entries**: Have real `user_id` and `chat_id` values, `first_name`/`last_name` from Telegram profile +- **MQTT entries**: Have `user_id: 0` and `chat_id: 0`, use `tid` (tracker ID) in name fields diff --git a/tracker-mqtt.json b/tracker-mqtt.json new file mode 100644 index 0000000..49d6504 --- /dev/null +++ b/tracker-mqtt.json @@ -0,0 +1,185 @@ +{ + "name": "MQTT Location Tracker - NocoDB", + "nodes": [ + { + "parameters": { + "topics": "owntracks/#", + "options": {} + }, + "id": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d", + "name": "MQTT Trigger", + "type": "n8n-nodes-base.mqttTrigger", + "typeVersion": 1, + "position": [ + 80, + 128 + ], + "credentials": { + "mqtt": { + "id": "MQTT_CREDENTIAL_ID", + "name": "MQTT Broker" + } + } + }, + { + "parameters": { + "conditions": { + "string": [ + { + "value1": "={{ $json._type }}", + "value2": "location" + } + ] + } + }, + "id": "b2c3d4e5-6f7a-8b9c-0d1e-2f3a4b5c6d7e", + "name": "Ist Location?", + "type": "n8n-nodes-base.if", + "typeVersion": 1, + "position": [ + 304, + 128 + ] + }, + { + "parameters": { + "jsCode": "// Extrahiere MQTT Location-Daten für NocoDB\nconst items = $input.all();\nconst results = [];\n\nfor (const item of items) {\n const data = item.json;\n \n // Timestamp (tst = Unix timestamp in seconds)\n const timestamp = new Date(data.tst * 1000).toISOString();\n const displayTime = new Date(data.tst * 1000).toLocaleString('de-DE');\n \n // Erstelle marker_label aus tid (tracker ID)\n const trackerLabel = data.tid || 'Unknown';\n \n // Optional: Verwende SSID als zusätzliche Info wenn vorhanden\n const markerLabel = data.SSID ? `${trackerLabel} (${data.SSID})` : trackerLabel;\n \n results.push({\n json: {\n latitude: data.lat,\n longitude: data.lon,\n timestamp: timestamp,\n user_id: 0, // MQTT hat keine User-ID, default auf 0\n first_name: trackerLabel,\n last_name: data.source || 'mqtt', // z.B. \"fused\"\n username: data.tid || '',\n marker_label: markerLabel,\n display_time: displayTime,\n chat_id: 0, // MQTT hat keine Chat-ID, default auf 0\n // Optional: Zusätzliche MQTT-Daten als Referenz\n mqtt_data: {\n accuracy: data.acc,\n altitude: data.alt,\n battery: data.batt,\n velocity: data.vel,\n course: data.cog,\n connection: data.conn,\n bssid: data.BSSID\n }\n }\n });\n}\n\nreturn results;" + }, + "id": "c3d4e5f6-7a8b-9c0d-1e2f-3a4b5c6d7e8f", + "name": "MQTT Location verarbeiten", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 528, + 112 + ] + }, + { + "parameters": { + "authentication": "nocoDbApiToken", + "operation": "create", + "projectId": "pdxl4cx4dbu9nxi", + "table": "m8pqj5ixgnnrzkg", + "fieldsUi": { + "fieldValues": [ + { + "fieldName": "latitude", + "fieldValue": "={{ $json.latitude }}" + }, + { + "fieldName": "longitude", + "fieldValue": "={{ $json.longitude }}" + }, + { + "fieldName": "timestamp", + "fieldValue": "={{ $json.timestamp }}" + }, + { + "fieldName": "user_id", + "fieldValue": "={{ $json.user_id }}" + }, + { + "fieldName": "first_name", + "fieldValue": "={{ $json.first_name }}" + }, + { + "fieldName": "last_name", + "fieldValue": "={{ $json.last_name }}" + }, + { + "fieldName": "username", + "fieldValue": "={{ $json.username }}" + }, + { + "fieldName": "marker_label", + "fieldValue": "={{ $json.marker_label }}" + }, + { + "fieldName": "display_time", + "fieldValue": "={{ $json.display_time }}" + }, + { + "fieldName": "chat_id", + "fieldValue": "={{ $json.chat_id }}" + } + ] + } + }, + "id": "d4e5f6a7-8b9c-0d1e-2f3a-4b5c6d7e8f9a", + "name": "Speichere in NocoDB", + "type": "n8n-nodes-base.nocoDb", + "typeVersion": 2, + "position": [ + 752, + 112 + ], + "credentials": { + "nocoDbApiToken": { + "id": "6fNBtcghMe8wFoE5", + "name": "NocoDB Token account" + } + } + }, + { + "parameters": { + "height": 320, + "width": 896, + "color": 5 + }, + "type": "n8n-nodes-base.stickyNote", + "position": [ + 0, + 0 + ], + "typeVersion": 1, + "id": "e5f6a7b8-9c0d-1e2f-3a4b-5c6d7e8f9a0b", + "name": "Sticky Note" + } + ], + "pinData": {}, + "connections": { + "MQTT Trigger": { + "main": [ + [ + { + "node": "Ist Location?", + "type": "main", + "index": 0 + } + ] + ] + }, + "Ist Location?": { + "main": [ + [ + { + "node": "MQTT Location verarbeiten", + "type": "main", + "index": 0 + } + ] + ] + }, + "MQTT Location verarbeiten": { + "main": [ + [ + { + "node": "Speichere in NocoDB", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "active": false, + "settings": { + "executionOrder": "v1" + }, + "versionId": "mqtt-location-tracker-v1", + "meta": { + "instanceId": "12d864c68e4fb5dfd100dc0c683b95f43cd55af7e9efa82e25407fac5a3824a5" + }, + "id": "MqttLocationTracker01", + "tags": [] +}