Building Your Own Smart Home Controller - Part 1, The Repository
Several months ago, I decided to build my own smart home controller. My reasons are many, and I’ll detail a few in a future post or two, but suffice to say that I wanted complete control over the devices in my home, and do not want to bring in an appliance rife with DRM that may one day turn into an expensive, dead plastic box.
I’m at the beginning of my journey, but my aspiration is to build something that others can also use and build upon, which is why I and a few others started the Rosie project on GitHub. More on Rosie in a future post as well.
Over the last few months, I’ve spent many, many, many hours getting my gateway and repository set-up with the right software and some basic flows. My hope is that these instructions save you some time if you decide to undertake this for yourself.
This post does duplicate much of the instructions I’ve provided in the Rosie-Repository repo, though I wil provide a bit more context and explanation for those of you who like a bit more prose with your software setup instructions.
This is a longer, but skimmable post. I’ve even included a handy TOC for the jump-arounders among you:
- Rosie Architecture
- Role of the Repository
- Installing InfluxDB
- Installing Telegraf
- Installing Grafana
- Auto-Starting Services
The Rosie system, as I’m calling it now, consists of two major pieces of hardware in your home:
- A Gateway, which facilitates all messages from and communication between connected devices
- A Repository, which handles data storage, events and vizualizations
Role of the Repository
The Repository is, as it sounds, the storage system for Rosie. It hosts the system databse, manages events and notifications, and provides dashboards of home data.
On the hardware side, the choice of hardware is largely up to you, though I recommend using a Raspberry Pi 3 with at least a 64GB Class 10 SD card.
On the software side, the Rosie Repository relies on three primary software packages to do its work:
- InfluxDB, the primary database for the system
- Telegraf, which collects metrics about my repository and saves those to InfluxDB
- Grafana, which hosts realtime dashboards for home data hosted in InfluxDB
The instructions below assume you’re installing this software on a Raspberry Pi 3 running Debian. If your hardware or OS is different, be sure to inspect the instructions below to note OS differences, which would likely manifest either in the package manager used or the platform architecture (ARM is assumed below).
One of the biggest challenges I had getting my repository up and running was getting InfluxDB and Telegraf installed on the system. I tried using the prepackaged binaries provided on the InfluxDB website, but these didn’t work for me, so I ended up building from source myself, which took a bit of doing. Hopefully these instructions will save you some time if you choose to do the same.
You’ll need to start by updating some fo the default software.
Install Node.js and NPM
Raspbian already has node 0.10.29, but not npm. You can save yourself some headaches by grabbing a newer version of both.
Start with fetching the latest package list from your manager.
sudo apt-get update
Install some essential build tools we need, then node and npm
sudo apt-get install build-essential sudo apt-get install nodejs sudo apt-get install npm
Now create a symlink to your new version of node
sudo ln -s /usr/bin/nodejs/usr/bin/node
To build InfluxDB (as well as Telegraf and Grafana) from source, you will need Go.
Start by installing bison and the Go Version Manager (gvm)
sudo apt-get install bison wget https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer chmod +x gvm-installer ./gvm-installer
Versions of Go installed will be saved to
You’ll want Go 1.6, but due to a bug in the 1.6 installer, will need to install version 1.4.3 first.
gvm install go1.4.3 gvm use go1.4.3 export GOROOT_BOOTSTRAP=$GOROOT
Once you have 1.4.3, install Go 1.6.2
gvm install go1.6.2 gvm use go1.6.2 --default
Prerequisites done, it’s now time for the good stuff. Almost…
Install a couple of additional dependencies before grabbing the Go source.
sudo apt-get install ruby-dev sudo gem install fpm
Download and build InfluxDB from source
go get github.com/influxdata/influxdb cd $GOPATH/src/github.com/influxdata/influxdb/ ./build.py --package --version=0.13.0 --arch=armhf
arch flag to
armhf is critical for installing InfluxDB on ARM devices like the Pi.
When you execute the Python build script, you might get a (non-fatal) error message. You can ignore this.
Finally, install InfluxDB. Note that the exact name of your
debpackage might differ from the one below. check the
builddirectory to get the exact name.
sudo dpkg -i build/influxdb_0.13.0~4254ad3_armhf.deb
To see if things worked, run the
influxdcommand and then
influx. You should jump into the InfluxDB cli once the latter command is issued.
Now that InfluxDB is up, lets jump to Telegraf…
Telegraf is a utility that can automatically capture and stream messages into InfluxDB (and other systems), which we’re using as the data storage system on the Repository.
I’m running Telegraf on both my Gateway and Repository to collect server metrics (disk, diskio, memory, etc) about each system and stream them into InfluxDB. This allows me to monitor the brain of my smart home, and even include it in Node RED based workflows for managing my setup.
As an ARM-based system, the installation process for Telegraf on the Pi is a bit different than other Debian-based systems.
Assuming you installed InfluxDB first, you can jump right ahead to pulling the source down building it with Go. If you’ve not yet installed Go, walk through those instructions in the InfluxDB section first.
Grab the telegraf source
go get github.com/influxdata/telegraf cd $GOPATH/src/github.com/influxdata/telegraf
makecommand and watch the magic happen
That was easy right? No idea why Influx isn’t as simple, considering both are from the folks at InfluxData.
Now that we have InfluxDB and Telegraf set-up, let’s get Grafana installed for some world-class dashboards!
Start by making sure that you have Node 4 or later installed.
wget https://nodejs.org/dist/v4.0.0/node-v4.0.0-linux-armv7l.tar.gz tar -xvf node-v4.0.0-linux-armv7l.tar.gz cd node-v4.0.0-linux-armv7l cp -R * /usr/local
As with before, if you’ve already installed InfluxDB and Telegraf, you should have Go and can grab the source. If you’ve not yet installed Go, walk through those instructions in the InfluxDB section first.
Grab the Grafana source
go get github.com/grafana/grafana cd $GOPATH/src/github.com/grafana/grafana
Run the setup script
go run build.go setup
Grafana uses node and has a ton of dependencies. Let’s install those now:
sudo npm install node-gyp sudo npm install -f
Make sure to use the
-fflag to bypass issues with installing PhantomJS on ARM-based systems.
sudo npm install -g grunt-cli
To bypass an issue with PhantomJS (again), you’ll need to make a change to the
sudo vi build.go Go to line 76 (`:76`) and modify the line to include a `force` flag grunt("--force", release);
Run the build script again. There will be warnings, but it should complete and place a
.debfile in the
go run build.go build pkg-deb
Now install the package. Note that the exact name of your Debian package might be different from mine:
sudo dpkg -i dist/grafana_4.0.0-1468683150pre1_armhf.deb
Open a browser and navigate to port
3000 on your Pi and you should see the Garfana login screen.
Once you’ve installed everything, you’ll no-doubt want to set up your services to auto-start on a reboot.
This will save you time each time you power-cycle your devices.
What’s more, having your services managed by
systemd will allow you to restart and shut them down without jumping through hoops.
After you install InfluxDB, you should have a file called
influxdb.service in the
/etc/systemd/system folder on your Pi.
If this file does exist, you simply need to modify the
Group settings on line 9 and 10 to the appropriate user and group (aka the user under which you installed InfluxDB).
If this file doesn’t exist, you’ll need to create it:
cd /etc/systemd/system/ touch influxdb.service vi influxdb.service
Then copy the service file from the InfluxDB repository here.
Groupsettings on line 9 and 10 to the appropriate user and group (aka the user under which you installed InfluxDB).
Now reload the systemd daemon and enable InfluxDB:
sudo systemctl daemon-reload sudo systemctl enable influxdb sudo systemctl start influxdb
Unlike InfluxDB, Telegraf didn’t put a
service file in my
/etc/systemd/system folder, so I added one myself:
Crete the file
cd /etc/systemd/system touch telegraf.service vi telegraf.service
Copy the service file from the Telegraf repository.
usersetting on line 8 to the appropriate user.
Determine where telegraf is installed on your system and copy the result.
For my installation, it was
Modify the path to the
telegrafcommand on line 11 to the correct location of your telegraf installer.
sudo touch /etc/default/telegraf
Reload the systemd daemon and enable InfluxDB
sudo systemctl daemon-reload sudo systemctl enable telegraf sudo systemctl start telegraf
Auto-starting Grafana is the simpliest of all
sudo systemctl daemon-reload sudo systemctl enable grafana-server sudo sysatemctl start grafana-server
And you’re done. You Repository and Gateway are ready for you to use to automate your home! In a future post, I’ll help you get started with node-red, setup dashboards and more!