Swedwise Innovation Labs – Using Mulesoft to make weather data flow
Swedwise Labs is our new forum for ideas, innovation and education. For our first Lab, we decided to put Mulesoft to the test as a weather station. With open meteorological data and through available APIs, we set out to find out whether there would be any precipitation within next three hours at a specific location, and then visualize the data. Why, how and what did we do? Read on!
Why Swedwise Labs?
The goal of each lab-session is to have a working concept of a predetermined idea and should be developed in a single sprint. The Labs involve technologies within the range of our business areas but also cross borders to stimulate fresh ideas and exploration.
The Station
For this lab, the goal was to retrieve some sort of data from several remote sources, and if necessary, sort and/or filter out parts of the retrieved data, then publish the results based on the retrieved data. What data should we use? Well, it’s been raining a lot here in Stockholm, so we decided to use open meteorological data. Swedish SMHI and Norwegian YR both provide APIs.
http://opendata.smhi.se/apidocs/metfcst/
Our Lab Developers (right to left), John Hollingworth, Henrik Mörner, Pontus “Frank” Andersson.
The Plan
- On demand from a user or by hourly-triggered action, retrieve weather forecasts from SMHI and YR through their respective API’s
- Processing and compare the forecast data
- Answer the question ”Will there be any precipitation within next three hours at a specific location?”
- Visualize the probability of rain (or other precipitation) within the next three hours with coloured led lights
- Green = No precipitation
- Yellow = Slim chance/risk
- Red = Precipitation is a fact
The Arduino UNO
The solution
Mulesoft AnyPoint Platform acted as the main player to orchestrate the data flow. The Mulesoft Application listened, and reacted to user input from an Arduino client (which we had never programmed before). Upon request from the client, a ‘scatter gather’ process was initiated to retrieve the forecast data.
The results were combined into a unified JSON-object. A Python script processed the JSON-formatted data and calculated if precipitation was imminent and then returned one of the three colour codes back to the Arduino client via Mulesoft AnyPoint Platform, which then interpreted the response and lit up the corresponding led light.
Conclusion
Before we started, we only had the general idea of what to do. During a period of six hours, we decided upon a weather forecast service, and were then able to come up with the solution presented above, together with a working concept. In reflection, we needed a smaller client that got its coordinates from its current location. If we were to build a working physical prototype, then a well-designed object that could be placed say just outside a building’s entrance, which would tell you when to bring your umbrella – or not!
More info….
Detailed Ardunio implementation
The Arduino UNO was connected to the Internet via an Ethernet Shield and was programmed to make a web-call to the Mule web server once every hour. The web server returned a value that translated to one of three colours: red, yellow or green depending on how high the risk for rain was at the time of the web call. Depending on the value we got in our response the lamp with the corresponding colour on the Arduino board was lit for one hour, displaying how high the risk of rain was for the next hour. After this the Arduino performed the next web call to refresh the value and continue displaying a light of some color.
Detailed Mule implementation – The LederStation
The Mule application (RT 3.7), which main purpose was to receive a consumer call via REST-service, perform a weather-forecast-retrieval from YR and SMHI, process the information and return a ”color-response”, whereby the different colors reacted if precipitation is inbound for the upcoming three hours, back to the consumer.
Lederstation Flow
Main application flow created an http listener. Upon request from client (in this case, the Arduino client), a scatter gather process was initiated, which makes simultaneous requests to the YR- and SMHI-weather services (both being the most well-known forecast-suppliers in Sweden – no source needed). Once the scatter gather-process was successful, the result from both services was combined into a JSON-object. A Python script that calculated if precipitation was imminent returned a color, which was sent back to the client.
localhost: 8081
/api/lederstation
method GET
getYR Weather Forecast Flow
Subflow YR: get YR-weather forecast. Once a response was retrieved, the forecast for the three upcoming hours was selected and transformed the XML-response to a Java-object. This enabled us to combine the results from both sub flows into one object after the main scatter gather).
Host api.yr.no
path /weatherapi/locationforecast/1.9/
method GET
params_ lat=59.319478&lon=18.073891
getSMHI Weather Forecast Flow Subflow
SMHI: get SMHI-weather forecast. Once response was retrieved, the forecast for the three upcoming hours was selected and transformed the XML-response to a Java-object.
Host opendata-download-metfcst.smhi.se
path api/category/pmp2g/version/1/geopoint/lat/59.3194/lon/18.0738/data.json
method_ GET
Test Color Response Flow
Mock Service for Arduino-development enabled this side to test the Arduino solution at an early stage. Pass in color as parameter receives response from service with that color.
path /api/lederstation/test?color=yellow
Data processing with Python
The data manipulation was done with Python 3.4, which utilizes a built in JSON module. Python was chosen for learning purposes and ease of manipulating JSON data. Any scripting language or mapping engine compatible with Mule could have been used. The processing sorted out the precipitation values and calculated probability by comparing results from both data sets. A string with the resulting color was returned.
Wanted features
- Get either latitude and longitude or city name from the incoming parameters
- Exception strategy when YR or SMHI fails
- Set time span for precipitation logic (dynamic range, not bound to 3 hours)
Share this post
