Detecting Presence Using Machine Learning
Objective #
The goal was to improve the accuracy and reliability of detecting whether or not occupants are present in a space using a ceiling mounted sensor cluster. The idea was to combine motion sensing (PIR) and sound (MEMS microphones) to detect presence instead of just detecting motion.
To make the distinction clear, traditional motion sensing uses PIR sensors to detect whether there is motion in the space. Motion sensors work by detecting the IR energy as a warm foreground object (such as a person) moves through an background environment. This requires the foreground object to be in motion, since a voltage is generated in proportion to rate of IR change across multiple IR detectors in the sensor (pyroelectric effect). If the foreground object remains stationary (eg. occupant seated at desk and not moving much), the motion sensor will not activate. If the lighting in the room is triggered based on motion sensors (as is very commonly the case for energy savings purposes) then the lights often annoyingly turn off even though there may be occupants in the room - requiring occupants to wave their arms to generate enough motion to turn the lights back on.
To solve this problem, we needed to detect the presence of occupants and not just their motion.
My Role #
As the senior machine learning engineer, my role was to design, develop, and deploy this feature in collaboration with the product software team. I was responsible for:
- Designing the specifications in accordance to high level user stories provided my product management
- Designing a practical solution that could easily run on a low end, compute constrained embedded device.
- Collecting training, validation, and test data
- Developing the presence detection model using Python, Scikit-Learn, SciPy, and Pandas.
- Implementing the ability for the model to adapt to different background noise levels to accommodate different room types and scenarios.
- Optimizing the model such that it could run with low memory requirements and without requiring external dependencies.
- Implementing the inference pipeline in firmware (using C), including a short time fourier transform which took advantage of ARM vector operations for increased performance.
- Validating the performance and assuring reliability of the system.
Impact #
- This feature allowed the ceiling mounted sensor to reliably determine presence in the space.
- This allowed much more reliable presence-based activation and deactivation of lighting, and HVAC systems, reducing energy
- The system was ported to other products at the company (wall mounted sensors), improving the reliability and performance of multiple products.