Edge Analytics
Objective #
The goal of this project was to build an on-device, rolling analytics system. The system ran on a ceiling mounted sensor cluster and had onboard environmental sensors such as: temperature, humidity, motion, and luminosity sensors as well as microphones.
My Role #
My task was to design the analytics system and serve as the technical lead for the development team. This feature was implemented using Python and C and used SQLite as the storage backend.
- UTC Timestamped sensor measurements were periodically stored in the database using pre-determined base units
- A rolling mechanism that only kept the most recent 30 days of data (since it is running on an embedded device, storage space is limited)
- An API that provided detailed hourly and daily metrics, exposing things like: min/mean/max temperature, humidity, sound level, and occupancy utilization (i.e. the fraction of time was the room in the occupied state). The API was called from other parts of the system.
- The API would convert the units according to the user's preferred units and timezone on the way out.
- The computed metrics are accessible via MQTT
- The system had to very light weight in terms of compute, RAM, and storage. Python Generators were used to minimize RAM.
- The system exposed a test interface for the quality assurance team.
Impact #
- The edge analytics system allows for computing analytics right at the source of where the data is acquired.
- Reduced latency allowing near real-time analysis of data
- Distributed computing, increased scalability
- Decreased bandwidth since you don't have to send raw data upstream
- Can feed metrics into centralized system for long term analytics