Cigars and Serverless IoT

Say what? Cigars and Serverless? What on earth could those two have in common? Maybe not much but bare with me and I’ll let you know.

The background

Some time ago I happened to run into a new Finnish open-source sensor beacon platform. I really wanted to give it a try. What could I do with it? Enter cigars! I came up with a requirement and created a user story that I wanted to implement: “As a cigar owner I want to be able to monitor the temperature and humidity of my humidor regardless of my location so that I know when to add purified water into the humidifier“. My current solution required me to open the humidor and check the meter inside it.

Screen Shot 2017-10-17 at 10.04.16

The brainstorm

Two important non-functional requirements were wireless communication from the humidor so I wouldn’t have to do any physical modifications to the humidor and a long battery life so I wouldn’t have to replace it too often. Both were met by the chosen sensor.

So what else would I need to make it happen? A place where I could process and save the sensor data and visualize it for devices regardless of their (or my) location. Enter cloud! I also had a couple of Raspberry Pi boards lying around so when the sensors arrived I was good to go.

AWS has an IoT service that can listen to messages from things (as in Internet of Things) you have registered to it. You can then do whatever you wish with those messages: save them into DynamoDB, process them with Lambda, forward them to Kinesis etc. Just what I needed. Enter serverless! Have to say I was very exited. I had never done any IoT stuff before so this was going to be a learning experience for me as well.

The solution

First thing I wanted to do was to read the sensor from the RasPi. A little bit of web surfing revealed a small but enthusiastic community around the sensor and I found a python script doing exactly what I wanted. The communication technology would be BLE.

Next thing was to connect the RasPi to AWS IoT service. That was also a no-brainer thanks to AWS documentation. AWS creates certificates and keys for the thing to be authenticated with and an endpoint for the messages. AWS processes the incoming messages with Rules. A rule defines a query that parses the incoming message and action(s) to be performed. The thing publishes it’s messages into a named MQTT topic via the given endpoint and the rule is a subscriber to the same topic.

I chose to save the data into DynamoDB by my IoT rule and implement a serveless website using S3 and Lambda. S3 is an object storage that is perfect for hosting static html files and Lambda is a compute service to run code without having to worry about any infrastructure. My Lambda function fetches the data from DynamoDB table and is called by ajax from html through API gateway.

Finally I wanted the lightest and the simpliest javascript graph library to visualize the sensor data from my humidor. See the screenshot  above. A bit boring graph I know. But luckily it is not an EKG!

At the time of writing this the data is flowing once every 15 minutes from the sensor into DynamoDB and it is read from there by Lambda whenever the html page is loaded. Maybe I’ll implement some alarms next?

Screen Shot 2017-10-17 at 8.27.36

What did I learn?

First of all I learned once again that serverless services are extremely fast and easy to implement for example for prototyping. They enable individuals and businesses to do things that have been impossible or at least expensive in the past. They also make it easy to explore new ways of doing thing and doing business. My rough cost estimation for this solution is 1-2€ per month after the AWS free tier has been eaten. So it is fast, easy and cheap as well.

Secondly I learned a lot about how DynamoDB works. There was quite a few tricks on the way. For example is allows you to set a TTL attribute for a field containing epoch seconds as a string but it won’t do anything.

AWS Services: IoT, Lambda, S3, DynamoDB
Sensor: Ruuvitag
Hardware: Raspberry Pi
Source and more details: Github

How not to suck as an agile team member?

Do you think of agile development as an act of freestyling and cappuccino drinking with no plans attached? Let’s just do something and deliver it to the customer, they will surely appreciate it?

I have news for you: that could not be further from the truth. My experience says that agile team and agile team member needs two thing above everything else. Those things are discipline and communication. The latter is quite self explaining (I’ll come to that later) but discipline? Is that like waterfall?

Agile is about reacting to changes and delivering small deliverables often. And to be able to do that a team must have a structured process and agreed ways of working.

Let’s take for example DoD which stands for Definition of Done. DoD tells the team what are the conditions a requirement has to fullfill in order to be called “done”. Like unit tests and code review. What happens if somebody is not disciplined and didn’t write unit tests for a task? The rest of the team thinks that they are done! And yes, I do know that there are ways to make sure that no untested code get’s into master branch in version control. Another example could be a scrum board. What if a team member doesn’t update his/her progress in real time? The rest of the team doesn’t have a clue how the team is progressing. Discipline.

One attribute of a successful agile team is continuous improvement. Of their processes. It is impossible to have a perfect process in place from day one. And if you don’t improve you rot. Teams start with some process and work actively to improve it. One way to make improvement happen is to agree on an intermediate goal and actively work to reach it. Everybody in the team has to do their part. Discipline.

So discipline makes sure that the team is moving into right direction and the whole team is in the same boat. And communication is the way to make sure discipline is in place.

Serverless 101 and Siili CraftCon

The second official Siili CraftCon was held before summer holidays 2017. It is an internal craftmanship conference for all craftmen and -women in Siili. This time it was half days and three tracks worth of pure skillz with topics such as “how to be a tech lead“, “data driven design“, “RPA” and more.

I had the pleasure of speaking about serverless architecture to the whole crowd as a closing presentation. Since I am a keen agile/lean fan I am also totally in love in serverless architecture and everything it has to offer in terms of reacting to changes and feedback and the ease of implementing new features and trying out new things.

Serverless means that you only need to think of you business logic. Everything else is taken care of by your chosen vendor. All big name vendors have their own serverless platform and services. In this context I am talking about PaaS (Platform as a Service) and FaaS (Function as a Service) side of serverless.

Some may include SaaS and (m)BaaS solutions into serverless context. SaaS stands for Software as a Service and like the name implies it includes software you configure for your needs. Some examples are Google Apps, Dropbox and Slack. (m)BaaS is (mobile) Backend as a Service and it provides some backend services, such as authentication, mainly for mobile applications.

FaaS is a subset of PaaS and means you write your function in you chosen language (or in a language that is supported by your chosen vendor) and deploy it. You also have to configure how the function is called. It can listen to events or can be triggered by an http request via an API gateway among other ways. Your vendor takes care of scaling it to your needs and you pay only for execution time. Wikipedia explains FaaS as “A category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure“.

PaaS includes lot more than just FaaS. Widely available PaaS services include messaging, databases, big data, analytics, file storage etc. They all are services you launch and configure. You can insert your FaaS function in a PaaS workflow and use all other available services with it. Again your vendor takes care of your infrastructure needs like scaling and backups and you pay for what you use. Wikipedia explains PaaS as “A category of cloud computing services that provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure“. See how it differes from FaaS?

In short, serverless means you are responsible only for your code and your data.

In Siili we have a lot of internal serverless development also on top of all the fancy stuff we create for customer. We also have cloud sandboxes freely available for all Siilis for learning and trying out serverless stuff.