Zigbee Device Monitoring Flow

Ok some quick instructions, so pay attention, kids.

In the check device health and global state, the “on start” tab is where you list your devices that you want checked outside of the default. Meaning, all zigbee devices will be monitored and checked every 2 hours according to the default set at the top. I’ve left mine in there. (which you will ultimately want to remove or comment out). The numbers are expressed in seconds. You can vary the length of time before it times out. REMEMBER this is expressed in seconds. Don’t edit anything that says DO NOT EDIT Unless you’re taking responsibility for your own code. This is proven to work. We used this early on in CORE development. It’s not clean. It’s not been improved, but CORE handles it like a champ.

Here is the first part.

[{"id":"1090fa3ec2993b28","type":"mqtt in","z":"20ad3b97e56645fd","d":true,"name":"","topic":"zigbee2mqtt/#","qos":"2","datatype":"auto","broker":"3f792c51bc3d4765","nl":false,"rap":true,"rh":0,"inputs":0,"x":190,"y":1100,"wires":[["29fd1b5b9e9f9a46","4d8933ef6de47276"]]},{"id":"bc0a4ca3ff6dbd3c","type":"comment","z":"20ad3b97e56645fd","name":"--- DEVICE HEALTH and GLOBAL STATE ---","info":"","x":250,"y":1000,"wires":[]},{"id":"29fd1b5b9e9f9a46","type":"function","z":"20ad3b97e56645fd","name":"Check Device Health & Save Global State","func":"// -- DO NOT EDIT BELOW THIS LINE --\n// Actual code\nconst defaultTimeout = context.get(\"defaultTimeout\");\nconst deviceOverrides = context.get(\"deviceOverrides\");\n\nif(msg.topic.startsWith(\"zigbee2mqtt/bridge/\")) {\n    // These messages we don't need, in general, but renames are important\n    if(msg.topic == \"zigbee2mqtt/bridge/response/device/rename\") {\n        node.log(msg);\n        const payloadObj = JSON.parse(msg.payload);\n        flow.set(\"dh*_\" + encodeURI(payloadObj.data.to.replace(/\\s/g, '_')), flow.get(\"dh*_\" + encodeURI(payloadObj.data.from.replace(/\\s/g, '_'))));\n        flow.set(\"dh*_\" + encodeURI(payloadObj.data.from.replace(/\\s/g, '_')), null);\n        flow.set(\"dh*_\" + encodeURI(payloadObj.data.from), null);\n    }\n    return null;\n} else {\n    if(msg.topic == \"PING\" || msg.topic == \"STATUS\") {\n        // Cleanup code, only used temporarily\n        // for(const savedDeviceNameEncoded of global.keys()) {\n        //     if(savedDeviceNameEncoded.includes('%20')) {\n        //         global.set(savedDeviceNameEncoded, undefined);\n        //     }\n        // }\n        let deviceStatus = {};\n        let deviceOffline = {};\n        let allOnline = true;\n        const now = new Date().getTime();\n        // const now = new Date().getTime();\n        msg.now = now;\n        const lastSeenTime = new Date(null);\n        for(const deviceNameEncoded of flow.keys()) {\n            if(deviceNameEncoded.startsWith(\"dh*_\")) {\n                if(deviceNameEncoded.includes(\"%20\")) {\n                    // This is an old and incorrect property\n                    flow.set(deviceNameEncoded, undefined);\n                } else {\n                    const lastSeen = flow.get(deviceNameEncoded);\n                    if(lastSeen !== null) {\n                        const deviceName = decodeURI(deviceNameEncoded.substring(4).replace(/_/g, ' '));\n                        let timeout = defaultTimeout;\n                        if(deviceOverrides.hasOwnProperty(deviceName)) timeout = deviceOverrides[deviceName];\n                        const timeLapsedNum = Math.round((now - lastSeen) / 1000)\n                        msg.timeLapsedNum = timeLapsedNum;\n                        lastSeenTime.setSeconds(timeLapsedNum);\n                        const timeLapsedStr = lastSeenTime.toISOString().substr(11, 8);\n                        if(timeout == -1) {\n                            deviceStatus[deviceName] = \"SKIPPED: \" + timeLapsedNum;\n                        } else if(timeLapsedNum < timeout) {\n                            deviceStatus[deviceName] = \"ONLINE: \" + timeLapsedNum + \", \" + lastSeen;\n                        } else {\n                            deviceStatus[deviceName] = \"OFFLINE: \" + timeLapsedNum + \", \" + lastSeen;\n                            // Clean up offline devices\n                            global.set(\"zigbee_motion_\" + encodeURI(deviceName.replace(/\\s/g, '_')), undefined);\n                            global.set(\"zigbee_contact_\" + encodeURI(deviceName.replace(/\\s/g, '_')), undefined);\n                            global.set(\"zigbee_\" + encodeURI(deviceName.replace(/\\s/g, '_')), undefined);\n                            allOnline = false;\n                            deviceOffline[deviceName] = timeLapsedNum;\n                        }\n                    }\n                }\n            }\n        }\n        if(allOnline) {\n            node.status({fill:\"green\",shape:\"dot\",text:\"online\"});\n        } else {\n            node.status({fill:\"red\",shape:\"ring\",text:\"offline\"});\n        }\n        msg.payload = deviceStatus;\n        if(Object.keys(deviceOffline).length > 0) {\n            msg.deviceOffline = deviceOffline;\n        } else {\n            msg.deviceOffline = null;\n        }\n        if(msg.topic == \"PING\") {\n            if(msg.deviceOffline === null) {\n                return null;\n            } else {\n                msg.topic = \"DeviceOffline\";\n            }\n        }\n        return msg;\n    } else {\n        const deviceNameRegExp = /zigbee2mqtt\\/([^\\/]*)/;\n        const deviceNameExtendedRegExp = /zigbee2mqtt\\/([^\\/]*)\\/([^\\/]*)/;\n        let deviceName = deviceNameRegExp.exec(msg.topic);\n        let deviceNameExtended = deviceNameExtendedRegExp.exec(msg.topic);\n        if(deviceName !== null && deviceName.length == 2 && (deviceNameExtended === null || deviceNameExtended.length < 3)) {\n            const deviceNameEncoded = encodeURI(deviceName[1].replace(/\\s/g, '_'));\n            if(msg.payload != '') {\n                let payload = JSON.parse(msg.payload);\n                if(payload.hasOwnProperty(\"last_seen\")) {\n                    flow.set(\"dh*_\" + deviceNameEncoded, payload.last_seen);\n                }\n                // Save the device state to global variables\n                if(payload.hasOwnProperty(\"occupancy\")) {\n                    global.set(\"zigbee_motion_\" + deviceNameEncoded, payload.occupancy);\n                    // Also make a combined state available\n                    if(payload.occupancy == true) {\n                        global.set(\"zigbee_any_motion\", true);\n                    } else {\n                        let motion = false;\n                        for(const savedDeviceNameEncoded of global.keys()) {\n                            if(savedDeviceNameEncoded.startsWith(\"zigbee_motion_\") && global.get(savedDeviceNameEncoded) == true) {\n                                motion = true;\n                                break;\n                            }\n                        }\n                        global.set(\"zigbee_any_motion\", motion);\n                    }\n                }\n                if(payload.hasOwnProperty(\"contact\")) {\n                    global.set(\"zigbee_contact_\" + deviceNameEncoded, payload.contact);\n                }\n                global.set(\"zigbee_\" + deviceNameEncoded, payload);\n            }\n        }\n    }\n}\nreturn null;","outputs":1,"noerr":0,"initialize":"// Code added here will be run once\n// whenever the node is started.\n// Settings\n// IMPORTANT: Device NAMES can't contain a forward slash \"/\"!!!\nconst defaultTimeout = 2 * 3600;     // 4 hours = 4 * 3600 seconds\nconst deviceOverrides = {\n    \"EXAMPLE Repeater - Balcony (IKEA)\": -1,    // -1 = Do not check for Device Health\n    \"EXAMPLE SillySensor\": 8 * 60 * 60,         // Checks in once every 8 hours expressed in seconds\n    \"BasementSpareLamp\": 4 * 3600, \n    \"MasterBedroomCloset1\": 4 * 3600,\n    \"MasterBedroomCloset2\": 4 * 3660,\n    \"KitchenCabinetsW\": 4 * 3600,\n    \"Doorbell\": 8 * 60 * 60,\n    \"RadioRGBW\": 4 * 3600,\n    \"SPAREBedroomLampL\": 4 * 3600,\n    \"BasementLivingRoomLampWest\": 4 * 3600,\n    // \"Button - Study Alice (Aqara)\": 4 * 3600,\n    // \"Balcony - Contact Door\": 4 * 3600,\n    // \"Study - GU10 Gledopto\": 24 * 3600,\n    // \"Livingroom - Yellow Lamp\": 24*3600,\n    // \"Study - Bulb (IKEA 1 color)\": -1,\n    // \"Study - Alice Lamp (IKEA)\": -1,\n    // \"AqaraCubeUnassigned2\": -1,\n    // \"T&H - Study (Aqara T1)\": -1,\n    // \"Livingroom - Coffee Machine Water Leak\": -1,\n    // \"Kitchen - Main Gas Valve\": 12*3600,\n    // \"Kitchen - Sink Water Sensor\": 4*3600,\n    // \"Kitchen - Contact Door\": 4*3600,\n    // \"Corridor - Temp & Humidity\": 24*3600,\n    // \"Entrance - Motion\": -1\n    // \"Study - iHorn Motion\": -1,\n}\n// -- DO NOT EDIT BELOW THIS LINE --\n// Actual code\ncontext.set(\"defaultTimeout\", defaultTimeout);\ncontext.set(\"deviceOverrides\", deviceOverrides);\nlet deviceStatus = {};\nlet deviceOffline = {};\nlet allOnline = true;\n// const now = new Date().getTime();\nconst now = new Date().getTime();\nfor(const deviceNameEncoded of flow.keys()) {\n    if(deviceNameEncoded.startsWith(\"dh*_\")) {\n        if(deviceNameEncoded.includes(\"%20\")) {\n            // This is an old and incorrect property\n            flow.set(deviceNameEncoded, undefined);\n        } else {\n            const lastSeen = flow.get(deviceNameEncoded);\n            if(lastSeen !== null) {\n                const deviceName = decodeURI(deviceNameEncoded.substring(4).replace(/_/g, ' '));\n                let timeout = defaultTimeout;\n                if(deviceOverrides.hasOwnProperty(deviceName)) timeout = deviceOverrides[deviceName];\n                if(timeout == -1) {\n                    deviceStatus[deviceName] = \"SKIPPED\";\n                } else if((now - lastSeen) < (timeout * 1000)) {\n                    deviceStatus[deviceName] = \"ONLINE\";\n                } else {\n                    deviceStatus[deviceName] = \"OFFLINE\";\n                    allOnline = false;\n                    deviceOffline[deviceName] = Math.round((now - lastSeen) / 1000);\n                }\n            }\n        }\n    }\n}\nif(allOnline) {\n    node.status({fill:\"green\",shape:\"dot\",text:\"online\"});\n} else {\n    node.status({fill:\"red\",shape:\"ring\",text:\"offline\"});\n}","finalize":"","libs":[],"x":520,"y":1100,"wires":[["ea547aeecca0720e"]]},{"id":"87626bcaf5cfe2bc","type":"debug","z":"20ad3b97e56645fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":970,"y":1180,"wires":[]},{"id":"8ede87446457a244","type":"inject","z":"20ad3b97e56645fd","d":true,"name":"Check Health Ping","props":[{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"7200","crontab":"","once":true,"onceDelay":"10","topic":"PING","payload":"PING","payloadType":"str","x":220,"y":1220,"wires":[["29fd1b5b9e9f9a46"]]},{"id":"dea944b0f42a56a7","type":"inject","z":"20ad3b97e56645fd","d":true,"name":"Get Health Status","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":"10","topic":"STATUS","x":200,"y":1180,"wires":[["29fd1b5b9e9f9a46"]]},{"id":"ea547aeecca0720e","type":"switch","z":"20ad3b97e56645fd","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"DeviceOffline","vt":"str"},{"t":"neq","v":"DeviceOffline","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":790,"y":1100,"wires":[["311bab92c731f793"],["87626bcaf5cfe2bc","6bd665e4060ea18d"]]},{"id":"b6f2a3e0c5dcff4c","type":"link out","z":"20ad3b97e56645fd","name":"->Pushover","links":["d9e4c8d5d6471847"],"x":1535,"y":1100,"wires":[]},{"id":"afab0efa86e28552","type":"function","z":"20ad3b97e56645fd","name":"Format Device Offline Message","func":"msg.topic = \"Device Offline!\";\nlet payload = \"Zigbee Device Health\\n\";\npayload +=    \"--------------------\\n\";\nconst lastSeenTime = new Date(null);\nfor(const [offlineDevice, lastSeen] of Object.entries(msg.deviceOffline)) {\n    lastSeenTime.setSeconds(lastSeen);\n    payload += offlineDevice + \": \" + lastSeenTime.toISOString().substr(11, 8) + '\\n';\n}\nmsg.payload = payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1330,"y":1100,"wires":[["87626bcaf5cfe2bc","b6f2a3e0c5dcff4c","6bd665e4060ea18d"]]},{"id":"4d8933ef6de47276","type":"debug","z":"20ad3b97e56645fd","name":"MQTT debug","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":450,"y":1060,"wires":[]},{"id":"6bd665e4060ea18d","type":"debug","z":"20ad3b97e56645fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":990,"y":1220,"wires":[]},{"id":"311bab92c731f793","type":"throttle","z":"20ad3b97e56645fd","name":"Limit Offline Messages","throttleType":"time","timeLimit":"30","timeLimitType":"minutes","countLimit":0,"blockSize":0,"locked":false,"x":1020,"y":1100,"wires":[["87626bcaf5cfe2bc","afab0efa86e28552","6bd665e4060ea18d"]]},{"id":"3f792c51bc3d4765","type":"mqtt-broker","name":"CORE Clean1","broker":"","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]

I’ve had the first nodes disabled, so you will want to enable them once you put your information in. Enter your core server information into your mqtt node. The inject nodes will allow you to check at any time. This outputs to the debug window.

I pushed the output to a pushover node. Easy enough.

Challenge accepted? @Mike @relaxton

OK, so I chucked it in and it seems to be working. Do you use another flow for low battery warnings? I do have a few of devices offline that I need to fix, remove or add to the ignore list but like it so far.

1 Like

I pushed it into pushover so that I could get the notification in a format I could walk around and fix the problems.

Yes, I did the same, while I did not want to use Pushover for my notifications, I set it up quickly just to get something working. I also linked it in with my emergency mode (e.g. leaks detected) so that not only do all coloured lights go red, but I get the text of the specific emergency sent to my phone.

Actually, most of the notifications were false positives. Is there any reason why this code is not using the availability data that zigbee2mqtt is maintaining? It looks like it is just doing a simple “elapsed time since last seen check” which is possibly a bit naive?

Am I missing something?

At the time this was created that was not very useful. At this point in time that may be a valid appooach though.

This has worked well for me, once the elapsed time for some devices has been modified it has been very good. I’m sure it could use a lot of tweaking by now, but it still works for me. I’ve not needed it for a long time but enabled it now, looks to work as it did, if I see something which needs changing I’ll update it.

Sweet. I will look at whether I could simplify by using the z2m availability and report back. I will also look at integrating battery reporting into the feed.

Definitely a good start, even if only because it inspired me to finally get my notification system working.