How Your Input Improves Device Detection

Even by modern data aggregation standards, the Sense monitor is doing something enormously complex as it collects detailed, sub-second level data from inside your home’s electrical panel. It’s reading multiple subsystems simultaneously, analyzing subtle changes in electrical signals, and reporting in real-time to the Sense app. But the collection and analysis part of the process only paints part of the picture, as we also rely on crowdsourced input from our community to help accurately identify devices.

Before we dive in, we want to make it clear that we take user privacy very seriously, and no personally identifiable information is ever shared with us or anyone else. So, while it might be interesting to see how much TV our coworkers here in the office watch, it’s not happening. 

Device detection: Listening for a voice in a noisy room

Sense’s Lead Data Scientist Ghinwa Choueiter, who has her doctorate in electrical engineering and computer science from MIT, describes how the device identification process begins: “Once the monitor is installed in the home and connected to the Sense app, we start getting data. Our unsupervised machine learning algorithms then begin to run and look for what’s in the home, and when it’s being used.” The algorithms analyze the timing, frequency, phase and magnitude of the electrical signals, among other things, and sift through the noise in order to detect individual devices in the home.

Think of this like trying to listen for one voice in a noisy room. When lots of people are talking, picking out a unique voice in a room can be tough, but that’s only half the battle. Once a distinct voice is distinguished, you still need to figure out who it’s coming from. With this challenge, sometimes Sense identifies exactly what a device is when it’s detected (eg. Garage Door Opener), while other times Sense will detect a separate device but may not be able to name it exactly, so it's displayed as "Unknown Heat 1", "Motor 3", etc.

It can take months to collect enough data for the machine learning algorithms to accurately identify the patterns of some of the trickier devices and separate them from the noise, and that part of the job is hard to integrate with human input or training. However, community input in the naming of recognized devices, and calling out when Sense makes is a mistake, is critically important. “We rely on users labeling devices that Sense has detected, but hasn’t yet identified correctly,” says Ghinwa.

This user input is incredibly valuable. When you rename a device, you not only help make Sense more useful in your home, but you help build our growing database of device labels. This database helps inform our device detection algorithms and better identify devices in homes across the Sense community. Thanks to users renaming devices, we’re able to identify certain electrical signatures as vacuums rather than just ‘Motor.’

Additionally, integrations with other connected devices in your home like smart plugs, Philips Hue, and Ecobee have helped the data science team collect aggregated ground truth data for thousands of devices, which helps inform the development of new device models, and refinement and testing of existing ones.

Easy vs. Tricky Devices

The statistical modeling that Ghinwa and the team perform across thousands of homes makes detection of some devices easy. “We have many data points on refrigerators since they’re a common appliance and their compressors constantly cycles on and off. We only need a few days of data from a new user’s home to identify one,” she says. On the other hand, appliances such as dishwashers say, are much more complex. They consist of more than one sub-part that interact differently with each other depending on the mode of operation and are used far less frequently relative to a fridge.

Pictured: A refrigerator's start up signal.

Most devices with a limited number of components and modes of operation are manageable. But what about something with multiple major components and modes of operation, like a bread maker? It is a very complex device because it has a heater, motor and electronics which operate in many different combinations throughout 10 or so modes.

In these instances where complexity grows exponentially, the combined efforts of machine learning and user input shine. Sense can identify the device, or individual components within the device, and the user can merge the components and rename the aggregate as a bread maker because he or she understands the behavioral context of its use, when it was last turned on, for instance. So if you’re a user who is labeling your appliances, whether it's a whole appliance or a sub-part, the data science team appreciates your help.

Even relatively simple devices can be difficult to distinguish from one another. Take for example toaster ovens. While they have a limited feature set, coffee makers are similar heat sources that can draw the same wattage and run for roughly the same time. In these cases, more data, like user labeling allows us to make more accurate predictions.

Put It In (Behavioral) Context

Being aware of your own routines and habits, believe it or not, can help with device detection. For instance, using the historical data from the Sense app, you may see a heat device that turns on every morning, 10 minutes before you wake up. Since you’re familiar with the routine in your home, you can easily identify the device as a coffee maker.

When you are trying to identify a device, some useful sources of information (besides your brain) include the timeline, where you can see when a device has turned on, and device statistics, where you can examine a device’s average run time and monthly usage. You can also set notifications for devices so you’re alerted when turn on.

The more of these devices you help identify, the better Sense becomes. You truly have a hand in improving detection in your home and homes across the Sense community.