Django South raw data migration

At $dayjob one of our Django application used to have PickleFields as model attributes. They where mostly used to store metadata in the form of dicts. But as Pickling is considered a security risk we wanted to get rid of them. Since all information would have to be translated to JSON for the API anyway it made sense to store it in JSON as well. This is easy using the implementation provided by the JSONField.

Read More

Recom over ethernet

After buying and installing a Remeha Calenta boiler last year I started digging into its hackability. After some googling I found that Robert Hekkers already did a lot of research on this http://blog.hekkers.net/tag/remeha/.

Using this information I was able to write a simple python script which could request data from the boiler using a PL2303 USB to serial TTL adapter (dx) using this wiring layout. This together with a raspberry pi near the boiler is now reliably logging metrics to to Statsd which in turn send it to graphite. Metric include but are not limited to:

  • in/out temperatures
  • pump/fan speed
  • setting values (eg. hot water temperature, pump speed limits)
  • status/substatus
  • valve states (eg. three-way-valve, gas valve)
  • external inputs (eg. thermostat heat request)
  • etc

But of course I want more. My goal is to build a controller to manage the entire heating system in my house with a good balance between energy efficiency and comfort. For example: when heating the living room the boiler needs to pump a lot of heat in the floor heating to have it heat up quickly. But after that the amount of additional heat needed is low. A hr boiler will never reach its maximum efficiency if it has to operate at full capacity. The on/off thermostat I have currently doesn’t help either. The boiler does implement some modulation based on the temperature of the water returned. But this is slow an not anticipating. So to make this more efficient I need to tweak some settings in the boiler. The most interesting in this respect are:

  • output water temperature
  • min/max pump speed

These together determine how much heat is pumped into the house.

Both of these settings can be adjusted trough the panel on the boiler or using the Recom tool provided by Remeha. (the pump speed requiring a ‘maintenance’ password, which is in the manual).

Recom however is Windows only, so I had to run it in Virtualbox. After installing the PL2303 driver the Recom tool was able to find a boiler on COM3, success :). But any attempt of Recom to read or write settings resulted in error :(. I probably really needed the €€€€ Remeha USB interface to have it work properly or at least a FTDI usb serial converter (which I didn’t happen to have laying around). I plugged the converter back into the raspberry and retreated from the cold attic to the warm living room and ordered some FTDI cables online. Not yet defeated I downloaded a older version of the Recom tool hoping that would maybe possibly work. Never hurts to try right? But then I would have to go back up to the cold again, sitting uncomfortably with my laptop next to the boiler.

Socat to the rescue!

I remembered a tool I heard of a while ago but never had the chance to use, socat. This tool basically allows you to hook any data stream to any other data stream. Things like a pipe to network socket back to a pipe again. Or in my case:

Screen Shot 2014-01-08 at 00.42.27

A virtualbox virtual COM port, mapped to a pipe file, mapped to

1
~ $ socat /tmp/port TCP4-LISTEN:1234

a ipv4 socket on my laptop downstairs, connected to a raspberry pi upstairs, mapping it to

1
root@raspberrypi:~# socat TCP4:192.168.2.20:1234 /dev/ttyUSB0

a PL2303 usb to serial converter, connected to the boiler

Raspberry pi boiler

Success!

Untitled

How could a direct connection not work but one using a detour over ethernet work? Lets take a long shot, would this work with the newer Recom version as well?? Yes :)

recom

Device identification:

id

Reading metrics:

metrics

Setting parameters:

parameters

Everything works as expected. And using some more Socat magic I will be able to tap into the datastream and log all packets going from Recom to the boiler and back giving me enough data to reverse engineer the protocol and build my own implementation. More on that later.

Some of the metric in graphite:

thaddis