Internet of Things (IOT) for a Java developer

Nowadays there is no possibility not to read about the new next thing: Internet of Thins (IOT). As a Java developer who mainly develops web applications and the backends IOT is not the daily business (at least in my case) but if you search hard enough you’ll find a usecase.

Usecase

In my case the usecase was my new office. In winter it was very cold, in summer it was really (really!) hot so I had the idea it would be a good idea to have a temperature logger to let my bosses see how hot it was in the office. Additionally I wanted to log the outer temperature to correlate it with the temperature inside the office.

Of course I did not want to buy a completed solution because:

  • I wanted to try out IOT on my own
  • I wanted my data not to be pushed somewhere into the cloud
  • I’m stingy

So I decided to build up my own solution.

Technics

As this should be a learning project I wanted to use

  • spring-boot
  • a MQTT message broker
  • ELK stack
  • my old Raspberry Pi model B
  • Docker

and it was clear, the technical solution would not be trivial but complex and excessive. In the end the technical draft looked like this:

technical_schema.png

Docker was removed from my requirements list – took too much time to learn.

Implementation

Upgrade of my Virtual Private Server (VPS)

The first thing (yes, I know, should have been the last step…) was to upgrade my VPS. All the things I wanted to install needed more HDD space and RAM I had, so I upgraded to a VPS providing

  • 1 virtual CPU core
  • 2 GB of RAM
  • 40 GB of HDD

This small machine seemed to be enough to run all pieces of my solution plus some additional things. My suggestion were correct, the final version runs smoothly with this hardware.

Raspberry Pi as temperature sensor

Using the Raspberry Pi model B (512MB RAM, 1x 700 MHz CPU) was easier I thought. I just had to buy a DS18S20 temperature sensor, connect it to the Pi and do some bash magic.

Thanks to a very good documenation (in german which can be found here) my Pi was able to do temperature measurements in the first try.

Grabbing the temperature information using Java was not very hard, I used the Apache Commons Exec project to fire some bash commands and parse the results.

MQTT Message Broker

Choosing a MQTT Message Broker was not that easy, there are many implementations with pro and cons. I decided to use RabbitMQ with the MQTT plugin because it also provides AMQP which I may use in later projects.

Java and MQTT

When I started the Java implementation I just knew I wanted to use spring-boot, but I had no idea how to send and receive MQTT messages. There is a client called paho provided by Eclipse, but I wanted to stay in the Spring universe so I decided to give the MQTT support provided by spring-integration a try.

After some testing (unfortunately the spring-integration documentation not always is that clear) I was able to send and receive my temperature information via MQTT.

Using the ELK stack

Installing the ELK stack is not that hard, there is plenty of documentation in the web. But writing the Logstash configuration matching my temperature logfiles was hard (at least for me). I ended up using much trial and error in this script:

input {

file{
path => "temperature.recipient\logs\received-temperatures.log"
}
}
filter{
grok {
match => { "message" => "%{DATESTAMP:logtimestamp} \[MQTT Call: %{DATA:mqttCaller}\] %{LOGLEVEL}  %{JAVACLASS} - {measureTimestamp=%{DATESTAMP:measureTimestamp:timestamp}, room=%{DATA:room}, degreesCelsius=%{NUMBER:degreesCelsius:float}, degreesFahrenheit=%{NUMBER:degreesFahrenheit:float}}"}
}
date{
match => ["measureTimestamp", "dd.MM.yyyy HH:mm:ss"]
target => "measureTimestamp"
}
mutate {
convert => ["degreesCelsius", "float"]
convert => ["degreesFahrenheit", "float"]
}
}

output {
elasticsearch{
protocol => "http"
}
}

Pitfalls

Of course I had some problems during implementation and installation of my temperature logging project.

Using HTTPS on the server

In my local development environment there was no need to think about HTTPS, everything worked fine using HTTP.

Configuring RabbitMQ to use HTTPS was a hard time for me (remember, I’m a Java developer, no Admin!). I did everything I found in the documentation (see here and here), but it did not work.

The simple solution was to enable both listener ports 1883 AND 8883 and block port 1883 by the firewall.

While running your Java application you normally do not see details when a SSL connection is established. If it fails, you have no idea why. In this case the JVM argument


-Djavax.net.debug=SSL

helps a lot.

Kibana RAM consumption

My Kibana 4.3.0 consumed lots of RAM which my VPS did not have which resulted in very high system load due to swapping.

The reason for that is a memory leak in the Kibana server component. The complete error description and a workaround can be found here.

Results

In the end I was successful and I have a well working temperature logger and a big analysis toolset provided by kibana.2015-12-31_20h52_19.png

Unfortunately I moved to a new well temperatured office in the meanwhile so there is no more need for a temperature logger there. But, as you can see, the temperature logger also works at home 😉

The Java sourcecode can be found in my BitBucket Git repository. Feel free to have a look at the sourcecode, create issues at BitBucket or post some comments.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s