S3A7 with Smart Garage Monitoring Demo

 

This tutorial uses the Renesas S3A7 IoT Fast Prototyping Kit with the Renesas IoT Sandbox to build a Smart Garage Monitoring solution.  This tutorial will show you how to build workflows starting from a S3A7 template project.

In this Smart Garage Monitoring Guide, you will:

  • Print alerts onto the screen and receive push notifications on your phone
  • Sense if a garage door is opening or closing using the vibration sensor
  • Sense if a garage door is open or closed using the proximity sensor
  • Sense if a car is running in the garage based on emissions using the air quality sensor
  • Touch screen to trigger AMS sensor sampling for debugging

Here’s a diagram of 9 workflows we will create in this tutorial

1

Here’s a high level diagram of how the S3A7 board is configured to send and receive data to the cloud.screen-shot-2016-11-07-at-7-18-40-pm

Here’s what you need to get started:

  1. Synergy S3A7 IoT Fast Prototyping Kit
  2. WiFi Internet Access
  3. iOS or Android phone mobile app (Download here)

Estimated Time to Complete: 1.5-2 hrs

Prerequisites:

STEP 1: ACTIVATE NEW S3A7 TEMPLATE PROJECT IN RENESAS IOT SANDBOX

Click here to activate a new S3A7 template project.  NOTE: We will use this activation link because it has all the necessary tags and streams automatically generated.  This is faster than starting from a blank project from scratch.  Note: You can use the same email address that you used in prior activations. Specify a unique project name for your Smart Garage Monitoring project.

Once activation is completed, you will receive an email with your account credentials needed to provision the kit and access the cloud portal.

This email will include your:

  • API Key
  • Device username and password

We will connect your board to this project in a later step.

STEP 2: SWITCH TO PROJECT ON RENESAS IOT SANDBOX

Login to Renesas IoT Sandbox with your web credentials. Select your newly created project in the top right drop down.

2

In order to switch between projects, click on “Your Project Name” in the top right.  Note, if you have other browser tabs open pointing to other projects, those sessions will be invalidated.

STEP 3: CONNECT TO MOBILE

You should already have the app installed from going through the Smart Chef tutorial. If you uninstalled the app, you can find links to the mobile app on renesas-docs.mediumone.com/?mobile or search for “IoT Controller” on the Apple or Google app store.

Make sure you enable push notifications when asked. If you have the app installed previously and you disabled push notifications, you will need to enable them in your Settings. Go to your phone Settings

Scroll down until you find the “IOT Control” app, select the app and select “Notifications”. From the notifications screen, make sure that the “Allow Notifications” switch is enabled.

6     7

Create a new profile in the app for this Smart Garage Monitoring project and input the API Settings from the welcome email you received, just like you did for the Smart Chef tutorial.  You can use whatever profile name you like.  Be sure to use the Mobile App credentials provided in the activation email for this project.  

Note: these instructions are for the iOS App but Android is similar.

Open the app, click + to add a new profile.   

8

Then click on API Login and specify a profile name, API keys, User, Password.

9

Be sure to click on “Renesas” to select the right environment.  Click Done.  Click Save.

img_3490

You can close the app, we will come back to this app later.

Checkpoint

In order to make sure the app is connected to the project, go back to your Renesas IoT Cloud web portal.  Click on Data Viewer – > Data Streams on the left side of the Renesas IoT Sandbox and select raw datastream. Select Configure on the top right and look for the “FCM_token” tag and check it and click Save.

11

You should see at least 1 event with the token like so.  If not, the mobile app has not authorized push notification or you did not enter the correct API credentials for this project.

12

Congrats! You’ve just connected the mobile app to this tenant and you can now receive push notifications.

STEP 4: INITIALIZE BOARD DISPLAY

In this step, you will create your first workflow in order to print “Smart Garage Monitoring” on the board screen when it connects.

Click on Workflow Studio on the left side of the Renesas IoT Sandbox and click the “Create +” button.

First, name the workflow “Initialize Display” by typing this in the textbox at the top left of the workflow then click anywhere on the empty canvas to save it.

13

Next, select the “Tags & Triggers” icon from the right toolbar.

14

From the Tags & Triggers menu, select “raw” and click and drag “connected” from the menu onto the empty workflow space.

15

Dragging the “connected” tag

16

“Connected” tag after it is released into the workflow

Now you need to select the “Modules” icon.

screen-shot-2016-11-07-at-3-32-12-pm

From the Modules menu, select “Foundation” and click and drag “Base Python” from the menu onto the workflow space.

18

Drag the “in1” circle on the Base Python module to the circle on the bottom of the connected tag. This will give the Python code access to the information in the connected tag.

19

Double click on the Base Python module.  Delete the code that is already there and replace it with the following code:

import MQTT
# print message to screen using MQTT
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(1, "SMART GARAGE MONITORING"), 'latin1')

Then click the “Save and Activate” button to finish setting up the workflow.  This will save the workflow and immediately activate it, whereas “Save” will not activate the workflow.

Checkpoint

Now it is time to test your first workflow. Power up or restart the board and touch the screen within 10 seconds to boot it up in provisioning mode. Connect to the given SSID and open http://192.168.3.1 on your browser. On the provisioning page, enter your WiFi information and this project information, like you did for the Smart Chef project. After clicking connect, restart your board by pressing the white reset button on the S3A7 board.  

Note: Refer to the Smart Chef Quick Start Guide for more information on how the board works.

Once your board connects to the WiFi, the screen should have a “Smart Garage Monitoring” message near the top of the screen.

Note: The depending on the image programmed on your board, the font size may appear different from the images shown in this tutorial.  However, the functionality will remain the same.  

If your board display has the “Smart Chef” logo, is part of the firmware and can not be removed via the cloud and you have an older image.  You may considering updating to the latest Smart Chef image as described in the Smart Chef Quick Start Guide.

 

Congrats! You’ve just built your first workflow!

STEP 5: PROCESS TOUCH SCREEN INTERACTION

In this step, you will be creating a workflow that prints how many times the screen has been pressed.

Create a new workflow and name it “Touch Screen AMS Sample Count” by clicking on Workflow Studio and “Create”.   

screen-shot-2016-11-07-at-7-03-35-pm

Drag the “touched” tag from the “raw” stream into the workflow, just like you did in the previous step. Then drag a Base Python module and connect its “in1” input with the touched tag.  

screen-shot-2016-11-07-at-7-04-17-pm

Double click on Base Python, then delete and replace the Base Python script with the following code:

 

import MQTT
import Store
# get last sample count from data store
sample_count = Store.get("sample_count")

# initialize to 0 if this is the first time
if sample_count == None:
    sample_count = 0

# save increment back to data store
sample_count = int(sample_count)+1
Store.set_data("sample_count",str(sample_count))

# print the count to screen at row 3
MQTT.publish_event_to_client('s3a7','D{};{}'.format(3, "Touch Count: "+str(sample_count)),'latin1')

Then click the “Save and Activate” button to finish setting up the workflow.

Checkpoint

Touch the screen.  You should see the value increase by 1 each time.

Note, you don’t have to restart the board for this workflow to work.  This is a powerful capability worth highlighting in this kit.  Just tap the screen!  You will see the Touch Count message increment every time you tap..

Congrats! You’ve just learned how to trigger from a screen touch and print to the display.

STEP 6: SAMPLE AIR QUALITY AND PROXIMITY DATA

In this step, you will be creating a workflow that samples the air quality and proximity sensor data on the screen.  Make sure the sensor is attached to the board. Note, in this step, the data is still encoded and we will decode it in subsequent steps.

Create a new workflow and name it “Sample Air Quality and Proximity”. Drag the touched tag and the connected tag into the workflow. Select “Scheduler” from the Tags & Triggers menu and drag the “Custom” trigger into the workflow.

screen-shot-2016-11-07-at-4-01-08-pm

Double click on the custom trigger and check the box next to “All Users” before clicking Save. Leave Minutely Interval to 1 (default).  

screen-shot-2016-11-07-at-4-05-07-pm

Finally, drag a Base Python module into the workflow.

In the previous workflows, the Base Python module only used one input, but we need to use three for this workflow. In order to allow for more inputs, double click the Base Python module, click “Inputs/Outputs” and press the “Add Input” button twice so that there are three total inputs.

22

Click save and connect the custom trigger to in1, the touched tag to in2, and the connected tag to in3.   Note, in this workflow, the ordering on the tag to input does not matter because we don’t use the data from these tags.  However, in some workflows below, the ordering will matter.

Also, this workflow will trigger when one of these 3 things happen:

  • Connect event is received
  • Touched event is received
  • Once per minute

23

Once the Base Python is connected with its inputs, replace the script with the following code:

# This workflow sense registers commands to the board to request sensor data.  The exact register values
# are based on the sensor data sheet

import cloud_driver_ams

cloud_driver_ams.request_ams_proximity(3, 'proximity', 's3a7')
cloud_driver_ams.request_ams_air_quality(3, 'air_quality', 's3a7')
    

Then click the “Save and Activate” button to finish setting up the workflow.

Checkpoint

Power up the board if it’s off, resetting the board is not necessary. Click on Dashboard on the left side of the Renesas IoT Sandbox.  Touch the screen. In the Real Time Events Log widget, you should see raw air quality and proximity data every time you press the screen. If you don’t press the screen, the custom trigger will make the data appear once per minute.  You will also see the count increase on the screen with each touch. This is raw data – in the next step, we will process the data.

24

Congrats! You’ve just learned how to create a timer workflow and sample vibration and air quality data from the board.  Note: see the Smart Chef workflows for examples on how to sample the other types of sensors.

STEP 7: PROCESS AIR QUALITY

In this step, we will take the raw air quality data that our previous workflow produced and process it to display it on the board.

Create a new workflow and name it “Air Quality Processing”. Drag the rawair_quality” tag into the workflow. Make sure you do not accidentally use the processed “air_quality” tag. Drag a Base Python module into the workflow, connect it to the air_quality tag, and replace the script with the following code.  Note, this workflow uses the data from the trigger using the IONode workflow library.

# This workflow converts the encoded air_quality data to human readable format
import MQTT 
import cloud_driver_ams

# get input data from the connected trigger.  Note "in1"
buf = IONode.get_input('in1')['event_data']['value']

status = cloud_driver_ams.decode_ams_status(buf)
air_quality_value = cloud_driver_ams.decode_ams_air_quality(buf)
resistance = cloud_driver_ams.decode_ams_resistance(buf)

# generate output event
IONode.set_output('out1', {
        "air_quality": air_quality_value,
        "resistance": resistance,
        "status": status
    })

# print to display at row 5 & 6
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(5, "Air Quality: "+str(air_quality_value)+" ppm."), 'latin1')
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(6, " Status: "+str(status)), 'latin1')


Unlike the previous workflows, we still have more to add to this workflow. Click save and then select the “Outputs” icon on the right toolbar.

screen-shot-2016-11-07-at-4-11-01-pm

Drag the “Processed Stream – Single” output onto the workflow and connect its “in1” to the Base Python module’s “out1”.

26

Double click the Processed Stream and make sure the “processed” stream is selected (should be by default).  Don’t change any other values.  Note: this workflow creates a new processed air_quality event with each encoded raw air_quality event.

Then click “Save and Activate” to finish setting up the workflow.

Checkpoint

Touch the screen again.  You should see a line that says “Air Quality: [number] ppm. Status: [status]”.

Note, if status is “RUNNING” the sensor is calibrating and the air quality value is not determined.  Wait until status is “OK”, then test the sensor.  Exhale onto the white circle on the AMS sensor and press the screen again. You should see the air quality value increase.  

28     

Congrats, you’ve just converted the air quality sensors values and printed it on the screen.  You’ve also learned how to sample the sensor data by touching the screen.

STEP 8: ALERT USER IF EXHAUST DETECTED

In this step, we will use the air quality value to determine if there is car exhaust in the air. If there is exhaust, we will alert the user.  This workflow will determine is the exhaust is on if the air quality is above 900 ppm.

Create a new workflow and name it “Garage Detect Exhaust”. Drag the processedair_quality” tag into the workflow. Make sure you do not accidentally use the raw “air_quality” tag. Drag the rawFCM_token” tag and a Base Python module into the workflow. Increase the number of inputs for the Base Python to two, connect both tags to the module such that air_quality goes to in1 and FCM_token goes to in2.  This ordering is important.

Note, this workflow uses the FCM (Firebase Cloud Messaging) token from the earlier step when connecting the mobile app in order to send push notifications.

30

Replace the script with the following code in the python module:

# This workflow sends a push notification when the air quality exceeds a threshold
import FCM
import Store
import MQTT

# set air quality alert threshold
threshold = 900

# get last push state from data store
last_push = Store.get("garage_air_quality_last_push")

# get air quality data from input
air_quality=IONode.get_input('in1')['event_data']['value']

# get FCM token for push notification from input
FCM_token = IONode.get_input('in2')['event_data']['value']

if air_quality > threshold:
    # print to screen
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(7, "Exhaust DETECTED"), 'latin1')
    # send notification
    if last_push != "high":
        FCM.send_fcm_notification_to_m1_application([FCM_token],"Exhaust DETECTED", sound="chime")

        # save to data store
        Store.set_data("garage_air_quality_last_push","high")
else: 
    # print to screen
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(7, "Exhaust NOT DETECTED"), 'latin1')

    # send notification

    if last_push != "low":
        FCM.send_fcm_notification_to_m1_application([FCM_token],"Air Quality back to normal", sound="chime")
        Store.set_data("garage_air_quality_last_push","low")

Then click “Save and Activate” to finish setting up the workflow.

Checkpoint

Power up the board or reset the board if it is already on. You should see a line that says “Exhaust NOT DETECTED” or “Exhaust DETECTED” depending on your air quality level.

To test this feature, exhale onto the white circle on the AMS sensor and press the screen again. You should see the air quality value increase and the message should now say “Exhaust DETECTED”. You should also receive a push notification from the mobile app.

33

Congrats! You’ve just created your first alert and push notification.

STEP 9: PROCESS PROXIMITY

In this step, we will take the raw proximity data that was produced earlier and process it to show it on the screen.

Create a new workflow and name it “Proximity Processing”. Drag the rawproximity” tag, a Base Python module, and an output Processed Stream – Single into the workflow.

screen-shot-2016-11-07-at-4-11-01-pm

Connect the tag to the base python module’s in1, and connect the output to the base python module’s out1.

screen-shot-2016-11-07-at-4-14-14-pm

Replace the Base Python script with the following code.  This workflow follows the similar concept as the Air Quality Processing workflow.

# This workflow converts the encoded proximity data to human readable format
import MQTT
import cloud_driver_ams

proximity = cloud_driver_ams.decode_ams_proximity(IONode.get_input('in1')['event_data']['value'])

# generate output event
IONode.set_output('out1', {'proximity': proximity})

# print to display
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(9, "Proximity: "+str(proximity)), 'latin1')

Then click “Save and Activate” to finish setting up the workflow.

Checkpoint

Make sure the AMS board is flat on the table facing up.  Touch the screen. You should see a line that says “Proximity: [value]”.

28

To test this feature, put your hand directly over the white circle of the AMS sensor (about 2 inches away) and press the screen. You should see the proximity value increase.

Congrats! You’ve learned how to sample the proximity data from the AMS sensor.  

STEP 10: CHECK IF GARAGE DOOR IS OPEN OR CLOSED

In this step, we will use the proximity value to determine if the garage door is open or closed.

Create a new workflow and name it “Garage Door Open or Closed”. Drag the processedproximity” tag, the rawFCM_token” tag, and a Base Python module into the workflow. Increase the number of inputs for the Base Python to two and connect both tags to the module. Ordering is important here so connect proximity to in1 and FCM_token to in2.

Finally, delete and replace the script with the following code.  

# This workflow will monitor proximity to determine if the garage is open or closed and sends a push notification
import FCM
import Store
import MQTT

# set proximity threshold
threshold = 1000

# get last state from store
last_push = Store.get("garage_open_last_push")

# get proximity data
proximity=IONode.get_input('in1')['event_data']['value']

# get FCM token for push notification
FCM_token = IONode.get_input('in2')['event_data']['value']

if proximity > threshold:
    #print to display
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(10, "Garage Door: CLOSED"), 'latin1')

    # send notification
    if last_push != "open":
        FCM.send_fcm_notification_to_m1_application([FCM_token],"Garage Door Closed", sound="chime")
        Store.set_data("garage_open_last_push","open")
else: 
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(10, "Garage Door: OPEN"), 'latin1')
    if last_push != "closed":
        FCM.send_fcm_notification_to_m1_application([FCM_token],"Garage Door Open", sound="chime")
        Store.set_data("garage_open_last_push","closed")

Click “Save and Activate” to finish setting up the workflow.

Checkpoint

Touch the screen on the board.  You should see a line that says “Garage Door: OPEN” or “Garage Door: CLOSED” depending of there’s something in front of the AMS sensor.

To test this feature, put your hand directly over the white circle of the AMS sensor and press the screen. You should see the proximity value increase and the message should now say “Garage Door: CLOSED”. You should also receive a push notification from the mobile app.

38

Note: You can wait up to one minute to see the changes on the screen instead of touching the screen when you move your hand away.  Again, this is because our earlier workflow samples the proximity data once per minute.

STEP 11: SET VIBRATION AGGREGATION WINDOW

In this step, we will set the board to send motion data to the Renesas IoT Sandbox every 10 seconds. You may have noticed the vibration data being sent earlier to the Real Time widget.  By default, the board will send vibration data once per 10 minutes.  WARNING: This step will increase the amount of free Renesas IoT credits consumed so you can increase the 10 seconds window to a high sampling period later on.

Note: The vibration sensor driver is built into the board firmware as opposed to the AMS Sensor.  The vibration data will transmitted in human readable format and not encoded. This is why the vibration and AMS sensor follow different techniques to acquire data.  Potentially, using workflows you can connect different types of sensors and read the registers without needing to change the firmware.

Create a new workflow and name it “Set Vibration Aggregate Window Garage”. Drag the rawconnected” tag, the rawtouched” tag, and a Base Python module into the workflow. Increase the number of inputs for the Base Python to two, connect both tags to the module, and replace the script with the following code:

# This workflow will send a message to the device to increase the vibration period to 10 seconds
import MQTT

# set to 10 seconds
aggregation_window = 10000 
MQTT.publish_event_to_client('s3a7', 'Svibration_window{}'.format(aggregation_window), 'latin1')
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(12, "Vibration Freq: "+str(aggregation_window)+" ms"), 'latin1')

Click “Save and Activate” to finish setting up the workflow.

Checkpoint

Touch the screen on the board. You should see a line that says “Vibration Window: 10000 ms”.  You have just trigger this workflow to set the new vibration aggregation sampling frequency.

To verify it is sending every 10 seconds.  Click on Dashboard on the left side of the Renesas IoT Sandbox. In the Real Time Events Log widget, you should see motion data once every ten seconds

screen-shot-2016-11-07-at-7-09-06-pm

Congrats! You’ve just learned how to use a workflow to set the vibration aggregate window on the board.  

STEP 12: DETECT GARAGE DOOR MOVEMENT

In this step, we will use the vibration information to detect garage door movement.

Create a new workflow and name it “Detect Garage Movement”. Drag the rawz_max”, “y_max”, “x_max”, and “FCM_token” tags from the raw stream into the workflow. Create a Base Python module with 4 inputs and connect all the tags with the module inputs. Note and preserve the ordering.

41

Replace the Base Python Module script with the following code.  

Note: In this workflow we calculate the total movement across each axis on the accelerometer.  We also count the number times we see movement and only trigger the alert when we see it after 2 cycles.  This eliminates false alerts when there a sudden vibration.   When a garage door is opening or closing, it will vibration for a minimum duration of time, so we are basing this workflow around that.

There are other vibration data sent such as x, y, z min and average, and, the vibration frequency for each axis.

# This workflow will calculate a total movement by adding up all the max values from x,y,z axis on the accelerometer
# An alert is generated when the total movement is above a threshold 
# and has been seen for at least 2 consecutive periods

import FCM
import Store
import MQTT

# set movement threshold
movement_threshold = 1.5

# get movement cycle count from Store library
cycle_count = Store.get("cycle_count")

if cycle_count == None:
    cycle_count = 0
cycle_count = int(cycle_count)+1

# get FCM token for push notification
FCM_token = IONode.get_input('in4')['event_data']['value']

# get last Push Notification state
last_push = Store.get("garage_movement_last_push")

# Add all the max values to get total movement
total_movement=abs(IONode.get_input('in1')['event_data']['value'])
+abs(IONode.get_input('in2')['event_data']['value'])
+abs(IONode.get_input('in3')['event_data']['value'])

# Send total movement to device
MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(13, "Total Movement: "+str(total_movement)), 'latin1')
if total_movement > movement_threshold:
    # print to screen
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(14, "Movement DETECTED"), 'latin1')
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(15, " Cycles: "+str(cycle_count)), 'latin1')

    # condition to send push
    if last_push != "true" and cycle_count > 1:
        FCM.send_fcm_notification_to_m1_application([FCM_token],"Movement Detected", sound="chime")

        # save to data store
        Store.set_data("garage_movement_last_push","true")
else: 
    cycle_count = 0
    # save to data store

    Store.set_data("garage_movement_last_push","false")
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(14, "Movement NOT DETECTED"), 'latin1')
    MQTT.publish_event_to_client('s3a7', 'D{};{}'.format(15, ""), 'latin1')

# save cycle count back to store  
Store.set_data("cycle_count",str(cycle_count))

Click “Save and Activate” to finish setting up the workflow.

Checkpoint

Power up the board if it’s off.  Within 10 seconds after boot up or workflow activation, you should see two new lines that say “Total Movement: [value]” and “Garage Movement: NOT DETECTED”.

To test this feature, shake the board. You should see the total movement value increase within 10 seconds. If the total movement is greater than 1.5, the Garage Movement message will change to “DETECTED. Cycles: 1”.

Once the Cycles value increases to 2, you should receive a push notification.

44

Congrats! You’ve just learned how to use the vibration data generate a push notification.

WHAT NEXT?

Congrats, you’ve just prototyped a Smart Garage Monitoring solution using the S3A7 Fast Prototyping kit and Renesas IoT Sandbox.  You can now use this kit to mount it on an actual garage opener and power it over wired USB.  This just scratching the surface of the Renesas IoT Sandbox capabilities.  Please check out the other tutorials and documentation.  Happy building!