Ethernet-connected temperature sensor, part 1: Intro, planning

(My original plan was to make posts about my "backlog" of projects first, so that the posts end up in the same order as the projects did. I've reconsidered, so that I can write about at least part of my current project in "real time", rather than after the fact. I'm currently at the stage where I have a working prototype, and have a custom PCB order on the way (literally, it's in the mail) - my guess is that it'll be here next week!

In a series of a few posts, I'll go through the entire project, from the idea to the final result, along with the many issues and considerations I went through.)

Intro

Back in February (2012), when I had only been working with Arduino/electronics for a month (prior to that, I had no experience with electronics - I couldn't have told you the difference between voltage and current), I was thinking about what to build next. I had previously bought two Maxim DS18S20 1-wire bus temperature sensors, so something utilizing them felt like a good choice.

I built up a circuit using one of the DS18S20 sensors and a regular parallel character display (16x2, HD44780 compatible) to show the current temperature. When I had all the code for that working, I added a 1024 kbit EEPROM to the mix. More coding to get the EEPROM working, but soon enough I had a program that read the temperature, and wrote the current time and temperature to the EEPROM, every 10 minutes.

(Actually, it didn't write the current time, but the number of seconds since starting up, since I didn't have an real-time clock chip handy.)

In either case, it worked! I then wrote a small utility to dump the EEPROM contents via the serial link, left the thing to log for a day, and then graphed the data using gnuplot - after calculating the log timestamps based on the boot time + the offset I'd stored.

Many months later, in late August, when I'd gained a ton of knowledge (in part from taking MIT's online 6.002x "Circuits and Electronics" course the first time it was offered), I was yet again looking for a project of appropriate difficulty.

The temperature logger idea came back to me, except that the goal this time around was to make something a lot more custom and permanent than a breadboarded circuit with ~25 jumper wires in a huge mess.

The entire goal was to design a product from scratch - begin with an idea, move on to writing down how things are supposed to work, then start working on a schematic, etc. - and when all that is done, design a PCB for it, and build the thing up!

Of course, having a custom PCB with parts on it doesn't make a final product. It also needed a enclosure of some sort, a power supply, cabling, programming, and more.

This first post will, apart from the long intro above, be about the early stages - the goals and the early planning.

Setting up the goal

I started fooling around a bit in a schematic editor, to try to get a feel for what I'd end up with. I had already decided to base the entire project around the Atmel Atmega328 microcontroller - simply because I wanted Arduino compatibility, and the Atmega328P-PU was what I already had in my Arduino Uno. I figured that if there would be any problems with the in-systems programming of the chip, I could always physically remove the chip, put it in my Arduino, and program it that way.

At first, the idea was to use the microcontroller, add an EEPROM for data storage and a real-time clock for time-keeping, connect the sensors and pretty much be done with it. I wasn't very happy with how manual this process would be, though - since EEPROMs pretty much only come up to 1024 kbit (128 kiB) storage space, you can't log temperatures for very long before you have to empty it. In addition, doing so would of course require bringing it to a computer, or bringing a computer to it. Far from optimal.

Soon thereafter, I found out about the FTDI "Vinculum" devices - the VDIP1 is a fully integrated USB host addon, with everything you need - the USB port itself and the controller chip. All I'd have to do is plug one in, write some code to interface with it, and I would have access to a whole USB memory stick worth of storage - currently available up to a ridiculous 256 GB worth of storage. With such plentiful storage space available, it could log for literally ages before running out of space.

(At one sample every 10 seconds, storing a 32-bit timestamp and two 16-bit temperature readings, it could go on for just over 10000 years before 256 GB is used up!)

It didn't take long to ditch that idea as well, though. Sure, the storage would last forever, but you still had to remove the USB stick, bring it to a computer and import the data before you could see what it had logged.

I started thinking about a solution that would work better in real-time - some sort of network connection.

After looking around a bit, I ended up going with Ethernet. Wi-Fi was another candidate, but I figured that in either case, you would probably have to either set up an access station, or drag an Ethernet cable (or both!) to use it in any sort of "remote" area, such as in a greenhouse.

After some time spent looking at the various Ethernet modules available, I ended up with the WIZnet WIZ820io. It was small(!), relatively cheap at 18 euros, fully integrated, and had a chip compatible with the Arduino Ethernet library. It would also be easy to mount, as it's essentially a 12-pin DIP package. Awesome!

At this stage, I'd decided to use the Atmega328 microcontroller, the WIZ820io Ethernet module, DS18S20 temperature sensors, and to interface it all with a computer (via Ethernet, of course) that does the slightly more heavy lifting of actually logging, processing and graphing the data, and finally present it to the user.

Here's a block diagram of the project in its final state:

Block diagram for the project. Filled arrow: data, unfilled arrow: data request.

The data acquisition board (DAQ board), i.e. the physical part of the project, runs the sampling process every 10 seconds. After reading the temperatures, it sends the readings via Ethernet to the computer, where a small Python program receives them, and stores them to an RRDtool database.

I chose RRDtool for several reasons: I have experience using it, the database format is well-suited for the task (the database never grows in size; it stores data "in a circle", so the oldest samples are eventually overwritten), and it can create rather pretty graphs - especially with some tweaking.

The database stores the data for a very long time - currently 20 years - before it is overwritten, though at different resolutions. If you want to look at data older than a year, for example, there will be one reading every 15 minutes. For more recent data, the resolution is 1 minute (past year) or 10 seconds (past two weeks).

At this point, I had the basic idea for the project down: sampling the temperature, and storing the readings in a database. What remains is to display the data to a user.

It was pretty clear from the beginning that the best way to present the data would be via a graph, on a simple web page. It'd accept user input regarding what to graph (i.e. which time span to display), and generate the graph.

To make it a bit more modern than to manually enter the date/time strings into text fields, I decided to use jQuery to create the UI, to allow for interactive date picker widgets, zooming in on the data by selecting areas of the graph, etc.

Despite having all this decided, essentially everything remained left undone. I hadn't created the circuit schematic, made the PCB layout, ordered a single component, or written a single line of code. All of that remained - and will be discussed in the coming blog posts.

The next post will focus on the schematic/circuit design. See you then!

As this post is lacking in images, here's one of the current UI:

The datepicker widgets show up when you click inside the text input boxes.