AE-CLOUD1 Diagnostics Intelligence with Synergy SSP 1.3.2 and WiFi

LEVEL 2: INTERMEDIATE |

This guide is an updated version of the S5D9 Diagnostics Intelligence Tutorial.

In this guide, you will learn how to set up an AE-CLOUD1 kit with Renesas IoT Sandbox with either Ethernet or WiFi. Then, you’ll login and view temperature, pressure, humidity, vibration, and noise level data on the Dashboard.  You will also receive daily email reports at midnight with the sensor data and other statistics.

HERE’S WHAT YOU NEED TO GET STARTED:

  1. Renesas AE-CLOUD1 Kit (Order Here)
  2. Qualcomm GT202 WiFi module
  3. Micro USB cable to power up the board
  4. Windows computer
  5. Ethernet Cable with Internet access or WiFi

PREREQUISITES:

 

OPTIONAL: PROGRAM THE S5D9 BOARD

This step is required if you have changed the firmware from the factory image.  Program the S5D9 image to the S5D9 board following the instructions in the “HOW TO FLASH THE AE-CLOUD1 BOARD TO FACTORY DEFAULT” tutorial. Be sure to download the latest release .srec in the prerequisites above.

STEP 1: ACTIVATE KIT IN RENESAS IOT SANDBOX

Click here to activate the kit and complete the registration process to create a Renesas IoT Sandbox account preloaded with the workflows. Once activation is completed, you will receive an email with your account credentials needed to provision the kit and access the cloud portal.

This email will include your:

  • Renesas IoT Sandbox web login
  • API Key
  • MQTT Project ID
  • MQTT Registration User ID
  • Registration User password

The email will also include a .txt attachment that you will drag into your board to auto-enroll in a later next step. To learn more about how auto-enrollment works, view Appendix C.

STEP 2: CONNECTING TO WIFI OR ETHERNET

This kit allows you to choose to connect the board to the internet via WiFi or Ethernet. First, check the board whether there is a connection via Ethernet cable. If you want to connect via Ethernet, connect the Ethernet cable then skip to the next step.

Alternatively if there is no Ethernet connection, the board will try to connect via WiFi. To be able to connect to WiFi, you’ll need a Qualcomm GT202 WiFi module attached to the board. The following steps will instruct you how to set this up and specify the WiFi credentials.

  1. Make sure the jumper for the PMOD is on 3.3V like this:
  2. Connect the Wi-Fi module to the AE-CLOUD1 board

  3. Plug your AE-CLOUD1 board into your Windows computer and it should show up as a USB Drive. Open the folder.
    Note: There may be other files also in the directory.


  4. Create a new text file ‘wifi.txt.’ Open the file.
  5. On the first line, type in the WiFi SSID you want the board to connect to. On the second line, type in the WiFi password, then Save.

Your board should now be configured to your WiFi settings. Next, you will connect your board to the Renesas IoT Sandbox project you’ve created earlier in step 1.

STEP 3: CONNECT THE AE-CLOUD1 KIT TO THE CLOUD

This step is to connect your AE-CLOUD1 board to the cloud and auto-enroll an API Basic User in your project. 

  1. If you have not already done so, plug your AE-CLOUD1 board into your Windows computer and it should show up as a USB Drive. Open the folder.
  2. Download the file named “m1config.txt” from your welcome email that you’ve received from Step 2. Do not change the file name. The email should have these credentials on each line:
    1. On the first line, the API Key
    2. On the second line, the MQTT Project ID
    3. On the third line, the MQTT Registration User ID
    4. On the last line, the Registration User Password

Sample m1config.txt:

  1. Drag this file into the board directory. Wait several seconds, then unplug the board and plug it back in. Make sure it is still connected to the Ethernet cable or WiFi module.
  2. After a few moments, both the green and yellow LED will turn on. This signifies that the board is trying to connect to the internet and your Renesas IoT Sandbox credentials (via m1config.txt file). Once connected, only the green LED light on the board should be on. If the board can’t connect for some reason, all 3 lights will simultaneously blink on and off. If this happens, verify the “m1config.txt” file is correct and your internet source is working.

Note: Once your board is properly connected, the m1config.txt file will be automatically deleted for security purposes. However, your board will still store its credentials until a new m1config.txt is created.

STEP 4: VISUALIZE DATA ON DASHBOARD

Now that your board is connected to Renesas IoT Sandbox, you can view data being sent from the board to the cloud. Log in to Renesas IoT Sandbox. Once logged in, make sure you are in the correct project on the upper right hand corner.

If you are not in the correct project, click on the drop-down list next to the symbol and select the project name that you just activated in Step 2.

Click on the Dashboard and you should see a preset Dashboard with five widgets. They each have a red-bordered box with ‘Select User’ as pictured:

Click on the drop-down and select your user on all five of the widgets. Your user is generated by each board’s chip unique ID fused onto the silicone. For any given board, the username is always the same. If you do not see any user listed, there was a problem with auto-enrollment. To debug this, view Appendix C.

Now scroll to the top of the Dashboard and click the save icon:

This is so you can save your Dashboard ‘View’ and not have to re-select your board’s user each time.

There are two types of events sent from the board to Renesas IoT Sandbox:

Summary Events

These events are always sent every 15 minutes by default. A summary event contains the average values from each of the sensor readings within the past 15 minutes. The sensors included are temperature, pressure, humidity, noise level, and vibration stats. To change the frequency of these summary events, view Appendix A.

Threshold Events

These events are sent whenever a sensor reading passes a certain threshold.  To change these thresholds, view the Appendix A

At the top of the Dashboard, you will see a Last Value Table widget. This widget only records the last entered value of that tag, along with the observed at time-stamp (in UTC). You can use this widget to see the last time the board status was ‘on’ and the last values for city, country, firmware version, WAN IP, LAN IP, and MAC address. For this tutorial, first, make sure that your board is connected by seeing the rows filled with the value and time-stamps. If there is nothing in the ‘value’ and ‘last observed at’ columns,  your board has had problems connecting to the cloud. You may need to click the refresh icon to update the widget with the latest data.

After your board has been connected for at least 15 minutes, it’s first summary event should be sent in. When this happens, the Real Time Gauge widget on your Dashboard should be updated with values like below. These values are the averages of the sensor for the past 15 minutes. It will continue to be updated every time a summary event is received.

Next, you will see a Table widget. This widget presents all the data from the Humidity, Noise, Pressure, and Temperature sensors sorted by most recent. Using this widget, you can see the sensor value changes over time as well as the last time the board sent an ‘online event.’ Click the refresh button to  update the table if any new data is received.

Under the table widget is located the Cross Filter Chart widget. This widget displays board vibration data for the past day. Furthermore, you can add filters based on time range and vibration levels. To see refresh the widget and see updated data, click the refresh button.To customize this widget, view the diagram below and the captions that correspond to the numbers.

  1. User selector. Here, you select the randomly generated username for your board. The red box next to it is the Tag selector. By default, it is ‘processed.vibration_status’.
  2. Date selector. By default, it is the past 24 hours. You can customize the date range.
  3. Filter charts. These charts are used to filter your data in the main chart (#4).
  4. Main chart. Your values for the selected tag are displayed in this chart. You can filter the data by time of date, date range, and values by the filter charts (#3)

The last pre-set widget on the Dashboard is the User Input widget. This widget does not display data but instead allows you to send data. For this tutorial, you will use this widget to control the LEDs in the next step. You will also use this widget to send a sample report later on.

To learn more about widgets, check out the Widgets Documentation.

STEP 5: LED TOGGLE AND BLINK

On the Dashboard, scroll down until you see the User Input widget. Your board’s username on the drop-down should already be selected.

Now we will send events from the cloud to the board to toggle and blink the LED lights.

Blink

On the User Input widget, go to the box that says ‘Blink.’ Type in the number of times you wish to make the LED blink; for demo purposes, pick a number under 10, then click ‘Send Event.’

The red LED on the board will blink the number of times you specified. You should also see this event appear in the Real Time Events Stream widget.

Toggle

On the User Input widget, go to the box that says ‘Toggle.’ To turn on the yellow LED, type in ‘1’.  Click ‘Send Event.’ You should see this event appear in the Real Time Events Stream widget. The yellow LED on the board will also turn on. To turn it off, type in ‘0’ and click ‘Send Event.’ If any number other than 0 or 1 is used, it will change states from off to on or vice versa.

To learn more about how toggle and blink works, view the ‘LED Toggle and Blink’ Workflow in Workflow Studio or view Appendix B .

STEP 6: NOISE DETECTION WORKFLOW

In this step, you will create a workflow from scratch to be able to detect noise.

First, you’ll need to activate a tag that you will be using in the workflow. Navigate to Config > Data Streams on the left sidebar.

Hovering over the ‘raw’ stream, click on ‘Edit’. This will open up a page that shows all the tags transmitted from the board to the cloud.

Scroll down the list and look for the tag “raw.mic_fft_max.”

Check the checkbox. Then, scroll to the bottom and click ‘Save Data Stream.’

You will now have access to use that tag throughout Renesas IoT Sandbox. Now you can build your workflow that will be triggered by this tag.

Navigate to “Workflow Studio” on the left sidebar.

Click on ‘Create.’

This will open a new blank workflow canvas.

In the name text area, type “Voice Detection” and click anywhere else on the canvas to save the name.

Click on the “Tags & Triggers” icon on the right sidebar. This is where you will specify when the workflow will run.

Click on “raw” and then look for “mic_fft_max”. Click and drag “mic_fft_max” onto the white canvas.

Next, click on “Modules.” Under “Foundation”, click and drag “Base Python” into the canvas.

Connect the nodes between the two boxes.

Next, click on Outputs on the right sidebar.

 

Click on the “Output” drop-down and click and drag “Processed Stream – Single” onto the canvas. Connect the node to the “Base Python” module.

 

Double click on the “Base Python” blue box. A popup should appear.

Copy and paste this script into the “script” text area:

import Store
import datetime
import json

def get_current_iso_time():
    dtnow = datetime.datetime.now()
    dtutcnow = datetime.datetime.utcnow()
    delta = dtnow - dtutcnow
    hh,mm = divmod((delta.days * 24*60*60 + delta.seconds + 30) // 60, 60)
    return "%s%+03d:%02d" % (dtnow.isoformat(), hh, mm)

histogram = IONode.get_input('in1')['event_data']['value']

max = 0
count = 0
total_energy = 0

for bin in histogram:
    if bin > max:
        max = bin
    total_energy = total_energy + bin
    count+=1
    average = total_energy/count
 
log("max:"+str(max))
log("total_energy:"+str(total_energy))
ratio = total_energy/max
log("ratio:"+str(ratio))

voice_detected = 0
if max > 200 and ratio < 120:
    voice_detected = 1
 
if voice_detected:
    IONode.set_output("out1", {"last_voice_detected": get_current_iso_time()})

Click ‘Save and Activate.’

You’ve just created a workflow that will output an event with the tag “last_voice_detected” with the value of the time-stamp whenever your board detects a voice.

Note: It will take 15 minutes to populate the ‘last_voice_detected’ processed tag; the time interval is set in your ‘Update Sample Period and Thresholds’ workflow. If you want to reduce the sampling rate, view appendix section. Note that lowering sampling period increase the usage of free credits. Alternatively, you can skip the following step and come back to after the specified time period.

To test this workflow, navigate back to the Dashboard. The first widget should be the Last Value Table. Click on the “Tag Config” settings gear icon. Locate “processed:last_voice_detected” and click on the checkbox next to it. Save. If you cannot find the tag, you may need to refresh the page.

A new row should have been added to your Last Value Table widget with the time-stamp of the last time voice was detected.

Trigger this workflow by speaking to the board. The mic on the board may not be as sensitive so you may need to hold it up to speak into it.

After a moment, the “processed:last_voice_detected” value should have been updated with a new time-stamp.

Remember to click on the ‘Save’ icon to keep this new addition to your Dashboard.

 

STEP 7: DAILY REPORT

The first step to set up your daily report is to add your email to the workflow. To do this on Renesas IoT Sandbox, click on ‘Workflow Studio’ on the sidebar. Next, click on the ‘Daily Report’ workflow:

Inside of the workflow:

Double-click on ‘Base Python’ box to edit this workflow’s code. After the import lines, you should see a line:

“recipients = ‘email@mediumone.com’”

Replace email@mediumone.com with your email. Click ‘Save and Activate.’

Now that you’ve got your email in the workflow, you can trigger the workflow to get a report. By default, the workflow is triggered daily at midnight (PT). If you wish to change this time, double-click the box labeled ‘Daily’ and change the settings there. For the tutorial, we will trigger this workflow now by sending the event {“sample_report”: 0} through the Dashboard.

Back on the Dashboard, scroll down until you see the User Input widget like below. Your board’s username should already be selected.

To trigger the Daily Report Workflow, click the ‘Send Event’ button that is under ‘Send Sample Report’.

After a few seconds, you should have received a Daily Report in your email with statistics of the past 24 hours. This report shows Temperature, Pressure, Humidity, Noise, and Vibration levels in Bar Charts. The sensor values are averaged and bucketed per hour.  There are also two pie charts that show the vibration status and board online status percentage. Lastly, there is a Geo map that provides your location based off of your IP address. Here is a sample report:

Note: Your graphs may look more blank if your board hasn’t been connected for the full past 24 hours.

WHAT’S NEXT?

Congratulations! You’ve successfully connected your AE-CLOUD1 board and sent data to the Renesas IoT Sandbox. You are now familiar with AE-CLOUD1 Fast Prototyping Kit, Renesas IoT Sandbox platform, and just tapped into the possibility of exciting new projects.  To add over-the-air (OTA) firmware update capability, continue on to the AE-CLOUD1 OTA Tutorial

APPENDIX A:

Changing the Sample Period and Thresholds

Navigate to the ‘Update Sample Period and Thresholds’ Workflow in the Workflow Studio.

Double-click on the ‘Base Python’ box to view the workflow’s code. Follow the directions in the workflow comments to change either the sample period or threshold.

SAMPLE PERIOD

This is the time before a ‘summary event’ is sent from the board to the cloud. This ‘summary event’  includes the average values across all sensors on the board and will be sent as long as the board is connected. If you change the sample period, you must also change these in the ‘Check if Board is Online’ workflow to ensure that the board online time and vibration on time are accurate:

  • Double-click the ‘Custom’ trigger green box and change the minute interval to the new sample period.
  • Double click the ‘Base Python’ box and look for ‘minutes=15’ in the code. Change this value to mimic the new sample period.

THRESHOLDS

This is the threshold a certain sensor much reach before a ‘threshold event’ is sent.

You must reset the board for any of these changes to take effect.

APPENDIX B:

Changing LED Settings

Navigate to the ‘LED Toggle and Blink’ Workflow

Double-click on the ‘Base Python’ box to view the workflow’s code. In this workflow, an event is sent to the board in the format of ‘L2:’ for toggle or ‘B4:’ for blink. The ‘2’ and ‘4’ specify which LED to toggle/blink. Change this number to

the following values to change the LEDs controlled:

These numbers correspond to the following lights:

  • Green: 1
  • Yellow: 2
  • Green + Yellow: 3
  • Red: 4
  • Green + Red: 5
  • Yellow + Red: 6
  • Green + Yellow + Red: 7

Once changed, you will need to ‘Save and Activate’ and your changes should take place immediately.

APPENDIX C:

Auto-Enrollment

This kit introduces an auto-enrollment process that connects your board with Renesas IoT Sandbox and automatically creates an API user specifically for your board. The username is generated by the unique ID on the board’s chip. The auto-enrollment process takes place in Step 3 of this tutorial when you put the ‘m1config.txt’ file with your credentials onto the board through USB. The new credentials are stored in the data flash. If erased, auto-provisioning it will reset the password.

What if I lost my welcome email with the ‘m1config.txt’?

You will need to re-create it by creating a new .txt file and pasting the following onto each line. The m1config.txt file is in the format:

  1. On the first line, the API Key
  2. On the second line, the MQTT Project ID
    • The MQTT Project ID can be found by clicking on Setup -> MQTT.
  3. On the third line, the MQTT Registration User ID
  4. On the last line, the Registration User Password
    • The Registration User Password was auto-generated so if you lose the email, you will need to reset that user’s password by going to Setup -> Manage Administrators and clicking ‘Edit’ under the ‘Actions’ column.

What if I accidentally delete my user?

If you accidentally delete your user, you can recreate it by auto-enrolling the user again. It will re-create the user automatically although you will lose all the previous data generated.

What if I accidentally delete the registration user?

If you accidentally delete the registered user, you will need to re-create it again if you want to add a new board to your project. To do this, go to Setup -> Manage Administrators. Here, you can click ‘Add New User’ and specify the registration user’s username and password. Only the MQTT ID and password from this user are needed. Repeat step 3 with these new credentials to auto-enroll a new board.

How do I manually provision my board instead of auto-enrolment?

To manually provision your board, complete Step 3 in addition to these next steps. When connecting the board to your Windows computer, it should show up as a USB. Click on the USB folder and create a new .txt file called ‘m1user.txt’. Open this text file. This text file will have a total of two lines:

Line 1 will be the User MQTT ID

Line 2 will be the User Password

See below on how to get these two values.

Sign onto Renesas IoT Sandbox and click on Setup > Manage Users. Click on ‘Add new user’ and type in a username and password then click ‘Save’. The password that you specified to create this user will be Line 2 of ‘m1user.txt’.

Back on the Manage Users page, a new row should’ve shown up. Copy the text highlighted where the red box is. This is the User MQTT ID. This will be on Line 2 of ‘m1user.txt’.

Save the file and wait 2 seconds. Unplug the board and plug it back in. It should now connect. Be sure to select this user’s username in the widgets for the Dashboard.

APPENDIX D:

GPIO

Interrupt driven GPIO input will be transmitted to the cloud to enable external peripherals such as buttons to be easily connected to the board GPIO output: The cloud can control a GPIO pin. On both grove connectors, pin 1 is an interrupt-input, and pin 2 is a controllable output.

#G = GPIO

#G<port>:<pin>:<level>

# for example:

MQTT.publish_event_to_client(‘s5d9’, ‘G4:11:1;G1:1:1’) # sets pin 4_11 high and pin 1_01 high

MQTT.publish_event_to_client(‘s5d9’, ‘G4:11:0;G1:1:0’) # sets pin 4_11 low and pin 1_01 low

APPENDIX E:

Restoring Factory Image to AE-CLOUD1

Please view this tutorial to restore the factory image to your AE-CLOUD1.