Fun with Monitoring, Part 1

When I moved into this house three years ago, I wanted to dabble with networked sensors. I wanted to play with the Internet of Things (IoT), or at least to play with my home network of sensors!

Based on my 20 years of experience as a networking professional, I have observed many reasons why a wireless signal might not arrive at its destination, so I prefer wired Ethernet whenever possible. I do not want my home network to disappoint me! This house has at least one Ethernet port in every room (closets, bathrooms, and storage rooms excepted), so the physical infrastructure meets my preference for Ethernet.

As a networking professional, I have also suffered through the difficulty of troubleshooting NAT. Obscuring addresses is not security, just a complication that breaks applications. Again, I do not want my home network to frustrate me! For the sheer number of things expected on the Internet of Things, I believe the greatly expanded address space of IPv6 is the obvious addressing choice for those things. As a member of an IPv6 advocacy organization, I also have an obligation to use IPv6 whenever possible, or to file bug reports so that IPv6 usage becomes possible. IPv6 is the fundamental requirement for sensors on my home network.

Hardware

I started with temperature sensors. I selected the HTU21D after reading some reviews that made me question the stability and durability of Dallas Semiconductor 1-wire temperature sensors. One way to gauge support and ease of use is to check Adafruit or Sparkfun. The HTU21D appeared to fit my theme of frustration avoidance: it is well supported, it reports humidity as well as temperature, and it can be polled more frequently than 1-wire although I don't use that feature. I am still happy with this hardware choice.

The HTU21D needs to be connected to a microcontroller or to a computer via I2C. Since I wanted to have many sensors, I wanted a low cost, low power host. I started with the Arduino Nano because it is a small, low cost Arduino. The Ethernet shield from Arduino uses a Wiznet W5100 for the Ethernet controller that implements IPv4 (only) in hardware and does not support IPv6. The ENC28J60 is often discussed as the alternative Ethernet controller for Arduino. So I went off to find libraries for the ENC28J60 supporting IPv6. Despite the mention of IPv6 that convinced me I could use this library, I was unable to get IPv6 working without any example code. I read through the code of the actual library, and I didn't find IPv6 examples. Programming the Arduino for IPv6 support without example code was more of a challenge than I wanted in a project at home. [update: Arduino IPv6 is possible!]

After the Arduino Nano, I moved on to the Raspberry Pi. A kind colleague suggested that I should get a full-sized Raspberry Pi for development in addition to the Pi Zero that I thought was what I wanted. At the time, that was a Raspberry Pi 2 Model B, and I am happy to report that it was essentially trivial for the Raspberry Pi to poll the HTU21D and send the data to my monitoring software. Success at last! The Arduino was fruitless, pun intended, but the Raspberry Pi worked easily.

The Raspberry Pi 2B has built-in Ethernet, so I used that. Using it as a development platform was definitely a great idea that saved me some frustration! The problems started when I tried to move the sensor setup to the Pi Zero with a micro-USB OTG cable to a USB wireless module (one stated to work with Pi!). Remember that IPv6 is my basic requirement for the network portion because NAT always brings its own pain. Well, the Pi Zero wouldn't pick up an IPv6 address until I put the wireless interface into promiscuous mode. I tried the USB wireless on the Pi 2B, and it failed IPv6 in the same way. As soon as the wireless interface goes into promiscuous mode once, the Pi (either one) listens to multicast and obtains an IPv6 address; meanwhile, Ethernet on the Pi 2B runs IPv6 easily. Of course I could write a startup script to activate promiscuous mode (I had to do that for a critical Solaris server at work a number of years ago), but it's a kludge (been there, done that, and I expect better from my hardware now!). The USB wireless worked as expected on my desktop running Xubuntu 15.10 at the time, so I believe without further investigation that it's an issue between wireless and Raspberry Pi (probably Raspbian). [update: unable to replicate issue after wireless access point upgrade]

Although I did not do this segment, there is an amusing detour. Since DDC automatic monitor detection is based on I2C, with a few spare parts that I had kicking around (a VGA extension cable, four jumper wires, and an external battery pack because I didn't have a 5V-to-3.3V step-down on hand at that time), my colleague was able to read temperature and humidity from one of the HTU21Ds through a VGA port! As long as you have a computer with DDC near where you want to measure temperature, you don't need many parts.

The current status is that I do not have a low-cost I2C host to connect to and poll the HTU21D. I am satisfied with the HTU21D temperature and humidity sensor. My experience with Arduino and Raspberry Pi convinced me that I personally will be happier and less frustrated with a tiny computer running Linux than I will be with a microcontroller. An Odroid C2 wandered through, but it was neither stable enough (both stock Odroid Ubuntu build and Odrobian tested) nor cheap enough to be considered.

Software

For data collection, I selected collectd because, for many use cases, the client configuration file is identical across all of the clients. I prefer making my tweaks on just one server, not on every (!) single (!) client (!). Also, collectd has a library of plugins to extend what data it gathers, and it has guidelines to write your own. For the data display, I selected Grafana because the dashboards are gorgeous and easy to customize.

Since Grafana only reads and does not store data, I needed software to accept data from collectd and store it in any format that Grafana reads. Although Grafana is extensible, I didn't want to write a data source plugin. I selected graphite for data storage for many reasons: it automatically creates a new file when new metrics arrive (no manual configuration!), it has a standard retention policy that keeps file sizes known (no surprises about disk utilization!), and I found the hierarchical model exposed in Grafana very understandable. I did try alternatives to graphite, but the alternatives were more complicated in Grafana; I want the component I look at the most to have the best user interface. The reason why I so seriously looked at alternatives is that graphite has scalability issues (dropped data) that it doesn't even log! Dropping data is horrible enough, but not even mentioning the pain point in a log message is worse! However, this is mitigated by scalable replacements to any of graphite's parts.

Although "run this command for metrics" was easy in collectd, my colleague wrote a custom plugin for the HTU21D; this custom plugin ran unmodified with the sensor connected to a VGA cable connected to a non-Intel graphics card (Linux). I love when stuff just works!

I am very happy with Grafana and collectd software. I haven't seen any problems with graphite either, and alternatives exist for any component not keeping up that I might encounter.

Conclusion

So, what else have I learned beyond the hardware and software selection? Well, I used to think I woke up for temperature changes, like waking up hot on a summer night; I now know it's usually an increase in humidity instead.

References

Next >