The Bits Behind the Bubbles

We sat down with Jonah, embedded software engineer, to hear how he came into engineering and how he helps to make that little orange box run.

Without our top-notch engineering team, we wouldn’t have Sense. From the bubbles you see everyday in the app, all the way down to the code on the orange box itself, the programmers here at Sense make it all possible. To celebrate Programmer’s Day  — yes, it’s a holiday! — we sat down with software engineer Jonah Petri to learn a bit more about how he got here, some of his favorite tools, and what he does to make the Sense monitor tick. 

Sense: Where did you go to school and what did you study?

Jonah: I did my undergraduate work at Boston College. I was pretty sure from the start that I wanted to be a computer science major, but I also got interested in math and physics near the end of my undergrad years. I ended up getting a CS degree and a math minor. One of my regrets was not getting deeper into math earlier in my undergrad career — I find software engineering very easy to teach myself, but I pick up math much faster when I can learn from a teacher, and I ended up running out of time to take all the classes I wanted to. There are many areas of my work for which I need a certain amount of math, so I've had to get better at reading math papers. It still doesn't come very naturally to me!

Besides Sense, what are some other products you’ve worked on?

I graduated in 2002, which was right in the midst of the first dot-com crash.  It was a tough time to get work as a very junior engineer, especially one without a degree from a famous technology school. After about 9 months of searching, I decided that I would need to try something different. So I crashed a career fair at MIT — just walked in like I belonged there — and found myself sitting for an interview with an Apple engineer. It went really well, and within a few months I had moved out to California to work in Cupertino! I spent 7 years at Apple, starting from a job in localization and build tools, then working on the Spotlight desktop search engine, and then spent my last 2 years working on the iPhone. However, the lack of any work/life balance eventually caught up with me. After working for nearly 4 months straight with no weekends, I decided I needed to move on and try something different.  

I left Apple, and moved back to Boston, and started working at iZotope on software for film/tv audio, and for music creators. I learned about digital signal processing, and writing realtime low-latency software, cross-platform development, C++. Eventually I became the software architect for the company. We won an Emmy award for our audio processing suite called RX — that was definitely a highlight! I also decided that smaller companies were a better fit for me. I love having opportunities to work on a variety of systems, and in my experience those opportunities are much more available at smaller companies. To wit, iZotope started an effort to build a hardware device to let people make multitrack recordings really easily. It used our world-class DSP combined with some slick new ideas in UI design to tie it all together. I was lucky enough to be able to shift to work on that project, which was where I learned how to work with embedded linux and the basics of how to be a firmware engineer.

What do you do here at Sense?

At Sense, I work on the embedded software — the code which actually runs on the orange Sense Monitor in your power panel. The monitor does a lot of heavy lifting. It processes more than 4 megabytes of samples from the current sensors and line voltage sensors each second. It does feature extraction on that data, power calculations, and evaluates that data stream against a probabilistic model of the possible states of the house. The monitor also handles Wi-Fi connectivity, local network device discovery, buffering and uploading your house's data, and other housekeeping tasks like software updates and Bluetooth-based setup.

Why do you enjoy the work you do here?

I've always been very interested in energy use from an environmental sustainability perspective. My generation will likely be the first to see most of the truly awful effects of climate change, and I decided I needed to do something about it. I felt like there weren't a lot of opportunities for someone like me who isn't a chemist or mechanical engineer to work directly on end-user-facing aspects of a climate change-related product, but Sense is an exception! I was really excited to find the company, and the experience here has been great. I feel very connected to the mission of the company, and I feel like I definitely make a difference in our success. The work we're doing is very important! As we scale up our deployed monitor fleet, we can start making a serious impact on climate change. That's my goal, and we're on the right path to hit it. 

Also, Sense has a really great engineering work culture. People are engaged and work hard, but the company from the top down is run with the presumption that you get the best out of people by respecting their time away from work, and building trust and collaboration during the day. The result is a friendly, high-functioning, low-drama, cross-disciplinary team, and it’s great fun to be a part of it.

Can you talk about why it's challenging?

Hah, how shall I pick what challenges to talk about? Merely deploying an IoT device into people's houses which has no direct UI and is generally installed inside the electrical panel (by an electrician) is a challenge. People should never be asked to touch the device, for safety, so we have to have a lot of anti-bricking fail-safes in place, from a hardware watchdog up through many software recovery mechanisms. Security is also something we are always working on. Any device on a house's internal network needs to be locked down, and all communications of customer data need to be encrypted, and we put a lot of effort into getting that right. Also just keeping the whole monitor within its CPU and memory budget requires a good amount of effort. Fortunately, I'm lucky to have the help of some very smart data scientists who have been making great improvements to the components which evaluate what's going on in the home.

Any favorite tools or languages that you like to use or are excited about?

I am pretty excited about Rust. I have plenty of experience in lower-level systems languages like C and C++, and also higher-level memory-safe languages like Python, and all have their weaknesses. For my job here at Sense, any error which I can move to compile-time is a huge win for me. It's worlds easier to fix a compilation error than to diagnose a crash happening in a customer's installed monitor halfway across the country. Rust also helps me keep my memory use and CPU within budget. Plus cargo is awesome for its built-in build system and unit testing.  There's a whole class of issues from other languages which I just don't have to think about!  (This post paid for by the Rust Evangelism Strike Force™)

Can you share a story of a specific challenge here at Sense and how you overcame it?

Sure! One challenge that pops into my head is handling the high data rates within the monitor.  We use a signaling protocol called SPI to move data from the ADCs in our analog front end into main memory. In total, we need to move about 4.5 megabytes per second into memory, which is pretty high but nothing earthshaking. However, we ran into some unexpected difficulties getting that much data transferring continuously. We used the on-board DMA engine on our SOC, of course, but we would still have occasional dropped samples, which is definitely a non-starter for DSP. Any missing samples will cause corruption in the spectral energy of the incoming signal, which will mess up the downstream machinery which is tracking your home's state. A human would hear a dropped sample as a click, if they were listening to music, and it's very disruptive! So, I had to do a pretty deep dive into the hardware to make it work right. I ended up disassembling and reverse engineering the ROM code on the DMA controller itself to figure out what was going on. I found a way to modify the DMA controller's code to cut the amount of bus traffic in half, but I had to write a customized DMA program in assembler to get it functioning. I wrote up what I found at my blog for those who are curious to learn more!


From now on, you can thank Jonah and the rest of the engineering team every time you open up the Sense app. Thanks, Jonah and team!