Project 1: Home Energy Monitor with Arduino, Part 3/3: Data

In the two previous articles, a sensor box and related software were developed for Arduino microcontroller board, providing measurements of home energy usage by observing the rate of energy meter's LED blinks (digital meter) or disk revolutions (analog meter). In this article, we'll focus on the data itself, and learn how to use a PC for logging sensor streams to local files, how to create nice plots and statistics from sensor data captures, and how to upload data to the cloud (Cosm) for everyone to see. In the spirit of openness and open source, PC software will be written using Python programming language on Ubuntu Linux operating system.


LEVEL: Beginner

Motivation

So far we have developed a sensor box and sensor sampling software for Arduino, using home energy meter as a data source. Hence, we have a setup that provides measurements of current energy consumption once per second, ready to be plugged into a USB port of a PC. However, all this will be of little use without a piece of software that stores the data stream into a file. And again, these data files are pretty pointless without means for visualizing their content as graphs or scripts that calculate interesting statistics from the tens of thousands of individual samples.

We must start by making a few decisions. The first thing would be to decide which programming language to use on the PC side. For hobby projects - and why not for professional work as well - I recommend the Python programming language, because it is pretty easy to learn, comes with all kinds of great libraries, and in general allows you to do wonders with just a few lines of code. Over the years I have learned to value that last part, especially in prototyping and hobby projects: the less code lines you have, the easier it is to comprehend and memorize the whole thing, which makes at least me much more productive.

Python runs on many different platforms, so you can choose any one that you like - probably it is Windows, Mac or Linux. Personally I prefer open source solutions especially at home tinkering projects, so I choose Linux. Since this article is written with beginners in mind, we'll go with the Ubuntu flavor. However, to stay focused to sensors, I won't go through setting up the operating system nor installing Python to it - there's plenty of resources available for both and you know how to use Google. Besides, in Ubuntu it's already set up - just type python in console to start the interactive shell. If you prefer to use Windows or Mac, that won't be a problem as Python is easy to install.

We are fortunate in that for Python there are truly excellent tools available free of charge as add-on libraries: Via PySerial we get access to computer's serial ports for reading the data stream from Arduino. With the aid of the Matplotlib library, we can create incredibly useful plots with just a few lines of code. Statistics and general number crunching will be handled with ease via the Numpy library. These tools are really vast in features; we will barely scratch the surface. However, you should find that they are also rather simple to use and you can easily expand the examples given here to create more fancy stuff on your own - which is the goal of this article series. Now, let's get going.

Project 1: Home Energy Monitor with Arduino, Part 2/3: SW

In the previous article, a sensor box was developed for Arduino microcontroller board. Its purpose is to measure home energy usage by observing the rate of energy meter's LED blinks (digital meter) or disk revolutions (analog meter). In this article, I discuss writing sensor software for Arduino. We start from the basics: sampling an analog signal and analog-to-digital conversion. Next, simple looping technique is presented for capturing sensor data. Finally, Arduino software is finished by extracting the number of LED blinks per second from digitized signal, and reporting it to PC via serial connection.


LEVEL: Beginner

Motivation

If you have built the sensor box presented in the previous article, I bet you're already anxious to get the system streaming data to a PC and seeing your home energy consumption as a nice graph plotted over time. Or maybe you are here because you've built something else, and now have a data stream that you wish to get from Arduino to PC. In this article, I will go through the process using energy meter setup as an example. It should be trivial to modify the presented software to work with other sensor data sources.

The solution described in this article involves a PC. You might wonder whether a PC is actually needed at all, as Arduino can be expanded with various stackable boards to have PC-like features. That's a fair question, and the answer is that indeed PC is not necessarily needed: you can expand an Arduino with an SD shield to write data to a memory card, or buy an Ethernet shield or even an Arduino with built-in Ethernet to connect directly to the Internet. It is also possible to print from Arduino. While such options exist, PC makes all of these easier to accomplish - especially during development phase, when you need to be able to analyze the signal in order to develop algorithms for processing it. Later on, you may want to remove PC from the equation.

In general, the topics covered in this article are good to master if you are interested in using sensors. For example, understanding how an analog signal is sampled and converted to digital form is fundamental for all sensor projects, as almost all sensors are based on an analog signal source but typically use cases are realized using digital signal processing (DSP) methods. Likewise, streaming data via serial connection is a basic skill. But, let's start with the sensor component and sampling theory.

Project 1: Home Energy Monitor with Arduino, Part 1/3: HW

The first sensor project at SensorBay.com covers a simple introductory project, which utilizes a PC, an Arduino microcontroller board, and a custom made sensor box for monitoring home electricity usage. The idea is to use an optical sensor to read current energy consumption from home electricity meter, and send this information to a PC for post-processing.


LEVEL: Beginner

Motivation

Everyone wishes to save energy and money. Minimizing your monthly electricity bill is a good place to start. What makes it difficult, is that your bill only tells the total amount of electric energy that was consumed during a long time window, typically one month. Hence, testing the effect of a change in behavior as an energy consumer is not practical. For example, if you lower the temperature in your house by one degree to save energy, what is the effect - how much will you save? What about two degrees? Or, what about 3 degrees in bedrooms but only 1 degree in living room? Waiting a month to see the result each time is not an option, as other things like weather changes come in to play.

A working solution would be to frequently log the values from your energy meter to a notebook, and then draw a graph to reveal the before-after effect. But things like this need to be automated to become practical. So we will use a home PC, and build a measurement device that can be simply attached on top of the electricity meter. To keep it completely harmless and safe, we will use an optical sensor. The measurement device continuously observes energy consumption and logs it into a file on your PC's hard drive. Then, you can easily draw graphs with a spreadsheet app.

Introduction to Arduino

In the second article, I introduce a cheap open source electronics prototyping platform Arduino, which will be used in many upcoming projects here at SensorBay.com. The article covers motivation, selecting and purchasing the board, setting up the development environment, and creating a simple programming example - just enough to get you prepared for next articles.

LEVEL: Beginner

Now that we have build a nice working environment (a small home electronics laboratory was set up in the first article) it is about time to start tinkering with something. So we need to acquire some sensors and stuff to play with. Time to go shopping! But what should we buy?

Workshop and Tools

This is the very first article at SensorBay.com. As an introduction, I write about the working environment and tools that I'll be using in the following posts, and give some tips about what things you should have in your workshop and how to organize it.

LEVEL: Beginner

This is an overview of my workshop. I have used only a few square meters of a room that has low, inclined ceiling. Yet, when properly organized, its all I need. You don't need much room for this hobby, but it helps to have a place that is reserved just for this purpose.

I have a table with enough empty space for a single project's stuff. All components, tools and equipment are set around me so that they are easily accessible. The chair is a special one.

I try to keep this place clean from clutter, but have to admit that it is sometimes difficult. Here's a few simple things that I've learned:

• Each thing really should have its own place and stay there most of the time
• Invest in good storage boxes and mark them carefully (e.g. with a label writer)
• Try to have only one project going at a time and clean after its finished

Welcome to SensorBay.com!

SensorBay.com is a new site targeted for amateurs and professionals who share a common interest to all things related to sensors.

SensorBay.com starts as a blog that will contain different kinds of sensor-related articles, such as tutorials, project descriptions, and programming examples. Focus is usually placed on how to use a sensor in a particular application.

The site is maintained by an individual sensor expert as a hobby project. You can read more about me from here.