Alexa Dust Sensor Tutorial

Author:  Spindance

Part 1:  Embedded Instructions

Overview

This is an air quality detection application that runs on the Renesas Synergy™ S3A7 Fast Prototyping Kit built in the Renesas e2Studio ISDE utilizing the Synergy™ Software Package (SSP). The application calculates dust particle concentration using the Seed Studio PPD42NS Grove Dust Sensor, and reports low pulse occupancy (LPO) data to Renesas IoT Sandbox.

Upon resetting, the application flashes the S3A7 Fast Prototyping Kit’s blue LED while waiting for a one minute sensor warmup period to pass. Thereafter, the green LED will flash to indicate that dust particle concentration readings are taking place and being periodically reported to Renesas IoT Sandbox in units of pcs/0.01 cubic feet. The yellow LED is turned on after startup to indicate that initialization succeeded, whereas the red LED indicates that initialization failed.

Currently, the green and blue LEDs flash on and off every second. Dust concentration is calculated by default over a 15 second window and reported to the cloud at the end of each window. The dust calculation window is able to be modified remotely via Renesas IoT Sandbox. The Grove Dust Sensor outputs low when dust particles are detected, and once the warmup period is over, the blue LED is turned on during low pulses from the sensor.

The board can be provisioned with Wi-Fi and Renesas IoT Sandbox connections by tapping the LCD immediately after the board is powered up, connecting to the board as a Wi-Fi access point, then entering the network SSID and password of the network the board should connect to, and also entering the project and user MQTT IDs and API key and password that were provided to you when registering with Renesas IoT Sandbox.

The Grove Dust Sensor must be attached to Grove A (UART3); Grove B (UART2) does not have the necessary interrupt pin and therefore is not compatible with this implementation.

The application contains 4 ThreadX threads and drivers.

  • The HAL/Common thread provides hardware abstraction and board support and contains the following components in its thread stack:
    • g_cgc CGC Driver on r_cgc
    • g_ioport I/O Port Driver on r_ioport
    • g_elc ELC Driver on r_elc
  • The MQTT Thread provides Renesas IoT Sandbox cloud communications and contains the following components in its thread stack:
    • g_flash0 Flash Driver on r_flash_lp
    • g_sf_wifi_nsal_nx0 NetX Port using Wi-Fi Framework on sf_wifi_nsal_nx
    • g_sf_wifi0 GT202 Wi-Fi Device Driver on st_wifi_gt202
    • g_spi0 SPI Driver on r_rspi
    • g_external_irq0 External IRQ Driver on r_icu
  • The GUI Thread provides LCD support and contains the following components in its thread stack:
    • g_Backlight_PWM Timer Driver on r_gpt
    • g_sf_touch_panel_i2c0 Touch Panel Framework on sf_touch_panel_i2c
    • g_transfer_dma Transfer Driver on r_dmac Software Activation
    • g_sf_message0 Messaging Framework on sf_message
    • g_i2c0 I2C Master Driver on r_riic
    • g_sf_external_irq0 External IRQ Framework on sf_external_irq
    • Touch Panel Driver on touch_panel_sx8654
    • g_transfer0 Transfer Driver on r_dtc Event IIC0 TXI
    • g_transfer1 Transfer Driver on r_dtc Event IIC0 RXI
    • g_external_irq1 External IRQ Driver on r_icu
  • The Sensor Thread provides Grove Dust Sensor support, including LPO calculations and reporting, and contains the following components in its thread stack:
    • g_fmio0 Factory Microcontroller Information (FMI) driver on r_fmi
    • g_toggle_timer Timer Driver on r_gpt. Provides a 1 Hz heartbeat that is used to flash LEDs to indicate status, and to determine if the dust concentration window has elapsed.
    • g_sensor_input_irq External IRQ Driver on r_icu. Used to receive callbacks when the sensor output changes from high to low or vice versa. This IRQ is used to record the start and accumulative duration of low pulses over the dust calculation time window.
    • g_elapsed_time_timer Timer Driver on r_gpt. Provides a microsecond resolution timer/counter for calculating elapsed time.

Windows Installation

Note that the following steps may require creating Renesas and SSP accounts.

  1. Install e2studio, selecting Synergy Software Package during installation.
  2. Install SSP v1.2.0 “SSP_Distribution_1.2.0.zip”
  3. Install WiFi Addon —> SSP Utilities —> “Wi-Fi Framework (with GT202 Drivers)” (Click down pointing arrow)
  4. Copy Renesas.S3_IOT_BOARD.1.2.0.pack to: \Renesas\e2_studio\internal\projectgen\arm\Packs
  5. Clone or download this repo to, for example, c:\Users\<username>\e2studio\. The root directory is an e2studio workspace.
  6. Open e2Studio. Open the workspace in e2Studio, for example, via File -> Switch Workspace -> <navigate to the workspace directory>
  7. Install e2Studio License – Navigate to license file when prompted: \Renesas\e2_studio\internal\projectgen\arm\Licenses Select SSP_License_Example_EvalLicense_20160629.
  8. Double click the project’s configuration.xml, and click Generate Project Content. e2studio may need to be restarted once this is complete.
  9. Attempt to build by selecting project GroveDustSensor in the project explorer, then clicking Project -> Build Project, by right clicking the project in the project explorer and selecting Build Project.

 

Part 2:  Cloud Instructions

Walkthrough

This walkthrough will guide you through the development of an Alexa skill. You will also learn how to integrate the Alexa skill with the Renesas IoT Sandbox. By the end of the walkthrough you will be able to query Alexa, “Alexa, ask Brainy Office, what is the current air quality?” to which Alexa will reply with “The current air quality is X”, where X is the most recent air quality measurement.

Contents

  • Prerequisites
  • Creating Alexa Skill
  • Integrating Alexa with Renesas IoT Sandbox
  • Enable and link the Alexa skill
  • Testing the Alexa skill
  • Creating the workflows
  • The Finish Line

Prerequisites

A smartphone with the Amazon Alexa app, or the web based alexa app

 

Creating Alexa Skill

Log in to your Amazon Developer account at https://developer.amazon.com (Note that this is not the same thing as an AWS account). Navigate to the Alexa tab in the developer account.

This walkthrough covers the Alexa Skills Kit, so click on the Get Started button under Alexa Skills Kit.

Click Add a New Skill.

For this walkthrough we will be creating a skill type with a Custom Interaction Model. Refer to the image below for guidance. The name of the skill is displayed in the Alexa mobile application. The invocation name is the name the user will speak to when requesting information. In our case we want to talk with Brainy Office.

On the Interaction Model configuration panel, configure the intent schema, custom types, and sample utterances.

  • Intent Schema – The intent schema defines the interface between Alexa and alexa request handlers.
  • Custom Slot Types – Specifies the custom types being used by the intent schema. The file is in yaml format for readability and organizational purpose. Refer to the Alexa Skill Configuration UI for the proper way to insert the data.
  • Sample Utterances – A list of specific examples that help Alexa understand what the user is attempting to do.

After configuration, the interaction model should look like the following:

Related Useful Links

https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/overviews/understanding-custom-skills
https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/defining-the-voice-interface

 

Integrating Alexa with Renesas IoT Sandbox

This section covers how to integrate the newly created Alexa skill with the Renesas IoT Sandbox. In order to do this we first need to create a stream on the Renesas IoT Sandbox to receive the Alexa requests from Amazon. After we have created a stream, we then need to create an OAuth client that our Alexa skill can link with. This is required to authenticate and authorize our Alexa skill with a particular account and configuration in the Renesas IoT Sandbox. Once this configuration is completed we then need to finish configuring our Alexa skill on Amazon.

Creating the Alexa Request Stream

Streams are configured by clicking on Config -> Data Streams in the Renesas IoT Sandbox. You will likely have several streams already defined. Scroll to the bottom of the list and click Create New Stream

Name the new stream, alexa_requests, and click Save Data Stream. All requests sent by the Alexa service will be delivered to this stream.

Creating the OAuth client

The OAuth client forms the link between our Alexa. This link is between a user’s Amazon Alexa account and a particular user account on the Renesas IoT Sandbox. To begin, navigate to Setup -> Manage OAuth Clients configuration page in the Renesas IoT Sandbox. Click the Add New OAuth Client button.

Name the client something meaningful such as Alexa OAuth. The Additional login message is displayed to the user on the login page, during the account linking procedure, within the Alexa mobile phone application.

We need to setup permission for Alexa to publish requests to the alexa_requests stream configured previously. Click the plus button next to permissions and select the alexa_requests stream from the drop down. Be sure to check both Read Permission and Write Permission. At this point your configuration should look like the image below.

Click Save when you are ready to move on.

Finishing the Configuration

Now that we have the OAuth client created on the Renesas IoT Sandbox we now have to share the Client ID and Client Secret with the Alexa skill configuration. Similarly, the Alexa skill configuration needs to share some configuration details with the Renesas IoT Sandbox.

The Client Id and Client Secret can be found in the Manage OAuth Clients configuration of the Renesas IoT Sandbox. See the image below.

Record the Client Id and Client Secret for the following steps.

Return to the Alexa skill configuration tool and navigate to the Configuration panel. For this walkthrough we are going to use the HTTPS Service Endpoint Type. Select the geographical region that is appropriate for your usage and paste https://assistant-rna.mediumone.com/alexa_request?stream=alexa_requests into the text window. The stream=alexa_requests portion of the query string should match the name of the stream that was created previously.

Next, select Yes for the account linking option and enter in the following information:

  • Authorization URL – https://auth-rna.mediumone.com/oauth2/authorize
  • Client Id – The Client Id recorded above, found in the Manage OAuth Clients configuration of the Renesas IoT Sandbox
  • Domain List – N/A
  • Scope – N/A
  • Redirect URLs – Record these for following steps. These will need to be added back into the Renesas IoT Sandbox
  • Authorization Grant Type – Select Auth Code Grant
    • Access Token URI – https://auth-rna.mediumone.com/oauth2/token
    • Client Secret – The Client Secret recorded above, found in the Manage OAuth Clients configuration of the Renesas IoT Sandbox
    • Client Authentication Scheme – HTTP Basic
  • Permissions – All unchecked

Fill in the Privacy Policy with https://mediumone.com/privacy

Now, back to the Renesas IoT Sandbox Manage OAuth Clients. Edit the newly created OAuth client from above.

Use the plus button next to Redirect URIs and copy/paste the Redirect URLs from the Alexa skill configuration.

Select Save in the Manage OAuth Clients tool and the Alexa skill Configuration tool.

As a final step in the Alexa skill configuration, on the SSL Certificate panel select “My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority” and click Save

This completes the configuration between Renesas IoT Sandbox and the Alexa skill. We are now ready to start handling requests, but first, let’s enable and link our Alexa skill.

Related Useful Links

https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/linking-an-alexa-user-with-a-user-in-your-system

 

Enable and link the Alexa skill

Using the Alexa mobile app or the web based version available at http://alexa.amazon.com/spa/index.html navigate to the Skills panel.

In the upper left hand corner select Your Skills.

The skill we created above should be in the list.

If the skill is not listed, return to the Alexa Skill configuration Test page and Enable the skill.

In order for the skill to communicate with the Renesas IoT Sandbox it first must be linked to your Renesas IoT Sandbox account. Select the skill then select the Link Account option.

Login to the portal using the username and password of the device you received in your activation email for the project. For example, the login might be device and the password might be A1g4KY1E90

If everything is successful you should be displayed a screen similar to the following.

We are now ready to being testing.

 

Testing the Alexa skill

Before we start handling Alexa requests let’s check to make sure our skill is functioning and requests are being sent to our alexa_requests stream. Open the Test panel in the Alexa skill configuration tool. Scroll to the bottom and enter in the utterance of “What is the current air quality?”. Click Ask Renesas Alexa Demo to send the request. The Service Request panel displays the JSON document that is posted to the https://assistant-rna.mediumone.com/alexa_request endpoint. The Service Response in this case is “There was an error calling the remote endpoint, which returned HTTP 503 : Service Unavailable”. This is ok for right now because the Renesas IoT Sandbox is not handling the alexa requests yet.

Return to the Renesas IoT Sandbox and view the alexa_requests data stream.

The data stream should contain the request that was just sent by the Alexa skill Test tool.

Now, let’s move on to handling those Alexa requests by creating some workflows.

 

Creating the workflows

Two workflows need to be created to handle the air quality measurements. To make the air quality measurement more friendly to the user we are first going to process the raw pcs/0.01ft^3 value into a qualitative description. Second we are then going to respond to the Alexa requests by returning these qualitative descriptions.

Qualitative air quality workflow

The raw air quality measurements are published the raw event stream with the identifier pcs. To being processing the raw:pcs values first enter the Workflow Studio in the Renesas IoT Sandbox. Click Create to beging creating a workflow. Give the workflow a name such as “Grove Dust Sensor Processing”.

From the right hand side of the workflow studio find Tags & Triggers -> raw -> pcs. Drag the box onto the screen.

If the pcs value is not available in the Tags & Triggers -> raw selection you may have to enable it from Config -> Data Streams -> raw by checking Active for the raw:pcs tag.

Now, find Modules -> Foundation -> Base Python and drag it onto the screen.

Now, find Output -> Processed Stream – Single and drag it onto the screen.

Connect raw:pcs to in1 of the Base Python block. Connect out1 of the Base Python block to in1 of the Processed Stream – Single block. When you are finished the workflow should look similar to the image below.

Double click the Base Python block to begin editing the code. Every time a pcs value is published to the raw stream the code in this block will run. Copy/paste the code from grove-dust-processing-workflow.py. Click Save

Double click the Processed Stream – Single. In the Tag Name place the value qualitative_air_quality. We will use this processed stream value in the Alexa workflow. Click Save and Activate

Alexa workflow

We would like Alexa to response to our requests. The way to accomplish this within the Renesas IoT Sandbox is to create a workflow to handle the request and provide a response. Begin by entering the Workflow Studio in the Renesas IoT Sandbox. Click Create to begin creating a workflow. Give the workflow a name such as “Alexa Request Handler”.

From the right hand side of the workflow studio find Tags & Triggers -> alexa_requests -> request.intent.name. Drag the box onto the screen.

Now, find Modules -> Foundation -> Base Python and drag it onto the screen.

Connect the alexa_requests:request.intent.name block to the Base Python block. When you are finished the workflow should look similar to the image below.

Double click the Base Python block to begin editing the code. Every time an Alexa request is published to the alexa_requests stream the code in this block will run. Copy/paste the code from alexa-workflow.py. Click Save and Activate

Related Helpful Links

Debugging help – http://renesas-docs.mediumone.com//?workflowstudio#debugging
IONode Lib – http://renesas-docs.mediumone.com//?libraries/ionode
Analytics Lib – http://renesas-docs.mediumone.com//?libraries/analytics
Built-ins – http://renesas-docs.mediumone.com//?libraries/builtin

 

The Finish Line

Return back to the Alexa Skill Test tool. Try the question, “What is the current air quality?” again. This time you should get a response similar to the following:

{
  "version": "1.0",
  "response": {
    "outputSpeech": {
      "type": "PlainText",
      "text": "The most recent air quality is Very Good"
    },
    "card": {
      "content": "The most recent air quality is Very Good",
      "title": "Brainy Office",
      "type": "Simple"
    },
    "reprompt": {
      "outputSpeech": {
        "type": "PlainText",
        "text": ""
      }
    },
    "shouldEndSession": true
  },
  "sessionAttributes": {}
}

Now, you should be able to ask your Alexa enabled device, “Alexa, ask Brainy Office, what is the current air quality?”. The response should be “The most recent air quality is ____”.

Congratulations! You have successfully created an Alexa skill and integrated it with the Renesas IoT Sandbox.