The Aptitive team just recently hosted our very first (data) Science Fair – a data-science twist on the science fairs of yore where students would stand in the local school gym with the mysteries of the universe printed onto cardboard trifolds, only instead of baking soda volcanoes and tennis ball solar systems, we saw demos for data warehousing, integration tools, predictive analytics, real-time data analytics, and some impressive data visualization tools (I’m looking your way, Looker. That VR demo was pretty cool).
We connected an Android phone to Azure IoT Hub and streamed its accelerometer data to the cloud every half-second. That data was read by an Azure function that converted the acceleration value to a percent of the robot’s max power and stored the result in an Azure Queue Store. The Raspberry Pi acting as the brain of our robot would read from the queue and run it’s four little motors accordingly. The harder you shook the phone, the faster the robot would go.
Watch this video of the robot race in action>>
If you’re not the type of person who is immediately overcome with joy at the thought of controlling robots via frantic arm-flailing and would rather we just science some data, we didn’t forget you.
We also performed real-time ETL on the acceleration data for data warehousing and visualization using Azure Stream Analytics. If you’re unfamiliar, Stream Analytics is a Platform as a Service tool designed to query, transform and load streaming data using a Microsoft brand of SQL that contains support for temporal logic. The robot may have been shiny, but the Azure powered real-time data analytics were the real star of the show.
While we were collecting two messages per second in IoT Hub, Stream Analytics was executing a series of saved queries on that data that averaged the acceleration values and sent the results to Azure Blob Storage and Microsoft PowerBI simultaneously. While participants were shaking their arms into a knotted mess, we were watching their acceleration dance up and down on a PowerBI dashboard while it was also being stored away for integration into a data warehouse (which, in fact, our data team was doing using a combination of Alooma and Snowflake).
The business application of our project was the power of real-time data analytics. Connecting to a single source of data and passing it on is just scratching the surface of what Stream Analytics is capable of. Combine a stream of how fast your delivery trucks are eating diesel with your maintenance records and now you have real-time data analytics reports of when you need to bring the next one into the shop. Add in your mileage reports and expected costs of maintenance and now you have a live feed of the profit margin you’re earning for each truck per mile.
Whether this is relevant to you is entirely dependent on your business needs, but this is by no means the only possible scenario. Finding that sweet spot where the dreams of business users meet technology is exactly where real-time ETL shines, and in a world where IoT is commonplace and data is collected in massive volumes, being able to transform your data for analysis on the spot is the next step in knowing what’s happening in your business the moment it happens.
You can also control robots with it too. So there’s that.
Click here to learn more about the (data) Science Fair and see more photos from the event.
Phil Howard is a Data Management and Analytics Consultant at Aptitive. In his role, Phil enables companies to discover insights and access the information held by their data.