By collecting 4 million data points every second, Sense builds up a very detailed library of device signatures over time that let it recognize which devices are on in the home and when. Many of you have asked us if you can help speed up this process by manually identifying devices. Here, we’ll explain the data science behind our approach, and how you can help train Sense and contribute to the accuracy and speed of our device identification algorithms.
Context provides important clues
In order to accurately identify a device, Sense needs to see repeated occurrences of the device in its typical operating context. From the standpoint of an electrical signature, many devices are not simply on or off — rather, they operate in a range of different states. Your typical dishwasher could be on, and the motor just kicked on; or it could be in rinse, drain, or standby mode. Each of these states looks very different to Sense.
Seeing a device’s full cycle and varied states helps our data science team identify it. Sense also uses its knowledge of other identified devices to determine which one is which. For example, your dryer is likely to run after your washer. Sense looks for these sorts of contextual time patterns to help identify your devices. The real-world operation of some devices contains useful clues that may be absent during an artificial training process, like enabling a “training mode” and then turning an appliance on and off multiple times in a row. For example, microwave ovens usually are activated for multiples of 30 seconds, but a purposeful “training session” would probably run the oven for much shorter bursts in consecutive order. That would remove the important contextual cues that can help make device detection more powerful.
More time = more data = better detection
Sense needs to see many occurrences of a variety of your home appliances turning on and off before it begins to distinguish them as unique items. This is because Sense is learning about your devices in the context of your particular home. Sense uses unsupervised machine learning to detect your devices. The qualifier “unsupervised” refers to the fact that there is no “ground truth”; there are no alternative methods to definitively determine each device that is on or off in your home. Sure, you might be able to tell Sense with certainty when your toaster is on, but doing this for every single device, each possible state for every device, and every possible combination of devices that might be on and off within your home at a given time, would be quite an undertaking! However, leveraging this advanced machine learning technology to allow the algorithms to “teach themselves” opens up an incredibly bright future for this technology.
Sense is like cat videos. No really.
To provide a related example, scientists at Google X trained an “artificial brain” to identify cat videos on YouTube by feeding it over 10 million videos. Instead of conducting pattern matching against videos labeled with specific features such as “human”, “cat,” or “cars,” — referred to as the training set — the system classified the images based on the patterns it found in the data alone. Again, it is important to note that Google X did this by using the images in their natural context. The process would simply not work if, say, a set of cat drawings had been used for that goal. To get a feeling for the concept of unsupervised learning, imagine you’re a little kid helping your family with the laundry and you’re surrounded by an assortment of socks. You don’t need to know the names of colors to begin to group blue socks with blue, white socks with white, and so on. Although device detection is considerably more difficult than sorting laundry, thankfully we don’t need quite as much data as Google in order to train Sense!
When you consider the number of household devices, the many states of each device, and the variation between devices of a single type; it starts to become clear why Sense takes time to learn.
How can I help train Sense?
In the meantime, there are a number of ways your input can help improve Sense’s device detection. By categorizing, naming and identifying the make/model of “mystery” devices that might just be labeled “Heat 1” or “Motor 2”, you’re helping to add more data points to the Sense database that can enable more accuracy and granularity to device detection in the future. You can also help catch problems where the algorithm misfires, or sees a devices turn on, but doesn’t catch it turning off, by sending a “Device is Not On” report. This can be done with just a couple of clicks right from the Device screen. We can already see large benefits from such feedback, and as Sense gets deployed in many more homes, device detection will keep improving with your help.