Dipping a toe into Z-Wave: OpenZWave in Go with the Ninja Blocks wrapper library

Having installed OpenZWave and successfully set up my UZB key as a controller, the next part of the process was to bridge the OpenZWave library to my Belphanior home automation core. Belphanior communicates with its servants via an HTTP-based RPC protocol; easiest route seemed to be to set up a web server to handle requests from Belphanior and translate them into Z-Wave commands.

OpenZWave is a C++ library; while I can work in C++, for web servers and the like I prefer a less detailed abstraction. Fortunately for me, an Australian firm called Ninja Blocks put together an excellent Go-language wrapper available on their github repo, which was exactly the tool for the job.

Here, I document what I liked about go-openzwave and my experience getting it working with Belphanior.

Dipping a toe into Z-Wave with the UZB controller hardware

Quite some time ago, I laid down some things I'm looking for in a home automation solution. I've recently come to the conclusion that Z-wave is probably mature enough to begin experimenting with. It's a closed protocol owned by a single company, but the company has been around long enough and has affiliated with enough consumer partners that it's unlikely to vanish tomorrow. It may not be secure (it looks like there are security features in the protocol, but I don't know yet if all devices support them), but it should support the debugging and status features I'm looking for that I was never able to get working with INSTEON. To that end, I purchased a UZB stick from Z-Wave.Me and a GE Dimmable Lamp Module and started experimenting. This will be an ongoing trip; we'll see where we end up.

Understanding USB Serial on Arduino Uno (There Is No Magic)

(source: http://gordonscruton.blogspot.com/2011/06/no-magic-please-part-1-learning.html)

A professor I had was fond of saying that the first thing to learn about computers is that there is no magic. From time to time, I am reminded of that truth. So let me tell you a story.

I've been playing around with a lighting project using some Supernight LED strips (unlike NeoPixels, these light strips allow you to control the hue but don't allow for individual LED clusters to be controlled; the entire strip takes on one color). After having some fun with them using the packaged-in controller, I decided to wire up my own based on some previous work and a tutorial from Make on how to control these lights (they're basically four-channel: one channel is +12v, and the other three can accept a PWM signal for color intensity). I used an Arduino UNO as the driver for the light control circuit. Everything was going well: I did a breadboard test, wired it all up, and put it in a nice project enclosure box.

To control the lights, I'm running a web service on a Raspberry Pi that communicates via USB serial to the Aruduino (in theory, the Pi could send the PWM signals directly, but I'm squeamish about wiring my harder-to-replace Pi to a 12 volt signal). I wrote a small state machine to accept some serial signals on the Arduino, and a small go service to take web requests and translate them into opening the serial port, sending the data, and closing the port.

When it works, it works great!

Trouble is, it didn't work. I could see my server sending the signals, and it believed they were sent correctly, but the Arduino wouldn't respond. I could even go into the Serial Monitor in the Arduino IDE and send the characters individually; they'd trigger the lights, but when I sent them from my server, the lights would go out. I couldn't debug by printing messages back over the serial connection, because I was using it to control the board (PROTIP: always add a couple LEDs to your enclosure; you never know when you're going to need the ability to just get a simple status signal from your Arduino without using serial).

The breakthrough came from adding a bit of code to the setup() function that caused the lights to quickly pulse red-green-blue to indicate setup was complete. I quickly learned that every time I tried to send a signal to change the light color from the Pi, the lights were pulsing---the Arduino was doing a full reboot. Why?

I had assumed that the Arduino was (at the hardware level) talking directly to the USB, and the Serial library I used emulated a serial connection in software. In reality, that's not the case at all. The secret is described clearly (well, clearly enough ;) ) in the wiring diagram for the UNO itself.

(Source)

The main processor is an ATMega 328P, but there's an entire second processor on the chip---an ATMega16U2 that sits between the main processor and the USB hardware. That processor has (normally) non-modifiable firmware that translates the USB serial protocol to a traditional 5v serial logic protocol. It also has a useful feature, sitting right in the middle of the diagram:

"There's yer problem."

It's a bit hard to tell in all the visual noise, but that's pin 13 on the USB controller wired to RESET on the main processor. So when does the USB controller reset the main processor? When a USB connection is initiated, the controller sends a DTR signal on pin 13. This signal reboots the processor. Every time my web service opened the serial device in Linux, it was causing the USB controller to trigger a DTR---the subsequently-sent bytes were then either being consumed or dropped on the floor. Fixing this was as simple as re-coding my service to open the device once and hold onto it for the entire life of the process. No sweat!

So this still leaves us the question: why is DTR wired to RESET? Here's the magic I was missing: The Arduino UNO uses the USB connection for two things: sending and receiving serial communications, and programming the Arduino itself. How does it know that the signal coming in is supposed to be new program logic and not serial commands for the program to consume? It turns out that the IDE reprograms the Arduino by starting a serial connection (causing DTR to pull the RESET pin). The first thing the Arduino's main processor does on reset is run firmware that listens for a few milliseconds for a sequence of bytes indicating new code incoming (these bytes are documented in the STK500 communication protocol v1)---if it receives those bytes, it writes them to flash memory, otherwise it gives up and begins running userland code. The Arduino Serial Monitor triggers the reset also, but doesn't send the new-code bytes. Previous to this feature being added, reprogramming the Arduino required the developer to time resetting the Arduino and starting the code upload somewhat precisely, which was annoying. Incidentally, if you own an Arduino and don't want this feature enabled, there are a couple of convenient ways to disable it so that your processor won't reboot every time it starts a serial connection.

So there you go: no magic, just a pair of processors with some interwoven activation logic that allows the same wire to serve as both communications and as the channel to download new program versions. Which, when it all operates smoothly, is pretty magic!

Playing with light: "Hologram" (Pepper's Ghost) projector

There's a neat trick you can do with angled plastic to give the impression of an image floating in space. While this is technically a "Pepper's Ghost" illusion, it commonly goes online by the name "Pyramid Hologram." For our friends at Iguanatron, we put together a top-down projector implementation with a 20" computer monitor.

So here's how it went down.

GoBlockly Interpreter: An Interpreter for Blockly in Go

I've just published GoBlockly: an interpreter for the Blockly visual editor output written in Go. This is something I wrote as part of a larger project (reimplementing the Belphanior Butler in Go), but it was a reasonable-sized chunk to break off and polish. It passes Blockly's own unit tests and allows for addition of custom blocks.

Example output of a server running GoBlockly library

If you program in go and want to play around with Blockly as a UI for simple code authoring, feel free to try it out! It doesn't come batteries-included (i.e. you'll need to set up your own Blockly web client and send the code back-and-forth to the server in go), but Go makes it easy enough to write a web server that this should be straightforward.

Have fun! :)