Amazon Service

Preparation

Alexa service includes AVS (Alexa Voice Service) and ASK(Alexa Skill Kit). In order to make Ameba support Alexa, we need to set up and support AVS. In addition, Ameba needs an Audio codec with I2S, and a speaker to process the audio.

1. Ameba * 1
2. ALC5680 *1
3. Button *1
4. Speaker *1

• Set up Alexa Voice Service

Refer to the official document to set up in the first time
https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/getting-started-with-the-alexa-voice-service#register

Below are the 4 steps:

• Step 1: Register Your Product
  This part is for account setting.
• Step 2: Prototype
  Writing programs to support Alexa on Ameba and we can do some basic controlling in this step.
• Step 3: Design and Build with AVS
  This part is to control the details like speakers number, default distance between devices. We don’t execute this step in the example.
• Step 4: Launch Your Product
  This part is for launching products. We don’t execute this step in the example.

• Sign up to be Amazon developers

First step, you have to be a developer of Amazon. Fill in the registration form in the first time.

https://developer.amazon.com/registration/profile.html

Click “Save and Continue” on the bottom of the form.
Click “Accept and Continue” if you agree with the consent.
Then it shows up the information of APP advertisement. We skip this part first and click “Save and Continue”.

• Add AVS App

Enter to the Amazon developers control panel

Click “Alexa Voice Service”

Then click ”Get Started” to start Arduino Alexa setting

• Product Information

Fill in some devices related information in this page:
1. Product Name: This one should be the same with the name we register to be a developer.
2. Product Type ID: Use only alphabets, numbers and under line.
3. Product Type: choose Alexa-Enabled Device

• Security Profile

Set up Security profile. We have to add a new one if we haven’t set up before.

Two fields to write in General:
1. Security Profile Name: Security profile name
2. Security Profile Description: Security profile description
Click “Next” and output the Security Profile ID.

Get Client ID and Client Secret, and the contents will be used in the example. You can check these contents again later.

Fill in the information that which URL to get when users authorize Ameba to use Alexa and how to return the authorization messages. And we use mDNS location here:
1. Allowed Origins: https://localhost:3000
2. Allowed Return URLs: https://localhost:3000/authresponse
Click ”ADD” and ”UPDATE”.

• Enable security profile

Activate the Security profile https://developer.amazon.com/lwa/sp/overview.html
Choose the Security Profile we set and click “Confirm”.

Fill in the URL which display the privacy policy when sign in in the first time. We use the same mDNS location and click “Save”.

Finish the AVS setting

• Apply for Refresh Token

We apply for the Refresh Token before the example implementation to reduce time consumption.

Github provides application tool on Windows/Linux:
https://github.com/ambiot/amb1_arduino/tree/master/misc/amazon_alexa_avs_auth_tool

Use openssl to get certificate before the execution.

openssl genrsa -out ca.key 4096
openssl req -new -x509 -days 365 -key ca.key -out ca.crt -sha256

Output ca.key and ca.crt files, put this two files and the tool in the same directory. Need to compile the Refresh Token tool on Linux first:

gcc -o amz_oauth_linux amz_oauth_linux.c -lssl -lcrypto

and execute

./amz_oauth_linux

Open cmd to run Refresh Token tool on Windows:amz_oauth_windows.exe

The screenshot above shows the execution of Refresh Token tool. Fill in related information when we apply for Alexa Voice Service.

1. device id: fill in Product ID
2. device dsn: Usually a set of serial numbers, there is no specific use of the situation, let’s fill in any
3. client id: fill in Client ID
4. client secret: fill in client secret
5. allowed origins: fill in https://localhost:3000
6. allowed return urls: fill in https://localhost:3000/authresponse
7. server port: fill in 3000

Then it shows a link with red box above, copy this link to the browser to apply for the Refresh Token. It shows Refresh Token parameter when the application is successful. And the cmd console also shows the Refresh Token(as shown in blue box above)

Preparation

• Audio codec

Ameba interface I2S can handle Audio data, and we use Realtek Audio Shield as I2S data source in this example.

Realtek Audio Shield IC is ALC5680, and Audio codec often uses at least one I2S and I2C. I2S is for data I/O and I2C is for initialization and option setting. The following is the Realtek Audio Shield official website introduction.
http://www.realtek.com.tw/press/newsViewOne.aspx?NewsID=441

• Speaker

There are no speakers on the Audio codec but we need them.

Use general speakers such as the speakers that support LINE IN.

Wiring and Example Setting

• Wiring

Realtek Audio Shield can combine to Ameba completely. It requires another button to wake up Ameba Alexa service with Realtek Audio Shield(red box shown below)

Connect Audio Shield and speaker to Ameba.

• Upload the program

Open the example “File” -> “Examples” -> “AmebaAudio” -> “alexa_basic”
Alexa service operates via the Internet and we modify SSID & PASS to the WiFi AP SSID & PASS to be connected.

Fill in the AVS service related information:

• avs_refresh_token: fill in the refresh token
• avs_client_id: the client id obtained from AVS set up
• avs_client_secret: the client secret obtained from AVS set up
• avs_http2_host: the host provided speech recognition，Amazon provides multiple hosts for different languages usage. We fill in the host with America.

Compile and upload to Ameba after complete filling in these fields.

• Test

There is a button to wake Ameba with Realtek Audio Shield. We push it and talk then we release it. Ameba will upload the voice data to the cloud and the cloud returns the response back to Ameba. Then, Ameba plays them.
Please refer to the demo video below:

Code Reference

• Code

• setup_alexa()

Set up all the information Alexa needs in function setup_alexa().

Alexa.setAvsClientId(avs_client_id,sizeof(avs_client_id)-1);

Set up Client ID, Client ID can be obtained when users register for Alexa APP.

Alexa.setAvsClientSecret(avs_client_secret, sizeof(avs_client_secret)-1);

Set up Client Secret, Client Secret can be obtained when users register for Alexa APP.

Alexa. setAvsRefreshToken (avs_refresh_token, sizeof(avs_refresh_token)-1 );

Set up the function to store Refresh token. It calls the function to store Refresh token into Flash memory when the first set is complete.

Alexa.setAvsHttp2Host(avs_http2_host, sizeof(avs_http2_host)-1);

Set up the website to offer AVS voice service. Amazon provides voice service which is based on HTTP2 and different languages uses different websites.

Alexa.begin();

And start AVS voice service.

• loop()

In the loop() function，continue checking the Internet status and reconnect to it when the status is abnormal.

void loop() {
　　if (WiFi.status() != WL_CONNECTED) {
　　    while (WiFi.begin(ssid, pass) != WL_CONNECTED) 
　    　{
     　　 delay(1000);
    　　}
　　    Serial.println("Connected to wifi");
　　}
	delay(100);
}

• Log Description

This section is to explain the log of Serial Monitor and the content below is a complete log of the Ameba with registered Alexa.

Interface 0 IP address : 192.168.88.32Connected to wifi

[HTTPD] httpd_server_thread started
init mdns
SGTL5000 CHIP ID:0xA011

[HTTPC] Use ciphersuite TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256

version=[HTTP/1.1]

status=[200 OK]

content_type=[application/json]

content_lenght=1233
[14316] update access_token:Atza|IwEBIJYyUjyCv8luvwAI7TQyL-j
[14333] update refresh_token:Atzr|IwEBIIW2_8KvJd9qake23w12WQT
[15664] connecting...
[24210] handshake done
[24239] C->S HEADERS
        :method: GET
        :path: /v20160207/directives
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
[24308] C->S HEADERS
        :method: POST
        :path: /v20160207/events
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
        content-type: multipart/form-data; boundary=bo
[24546] CS HEADERS
        :method: POST
        :path: /v20160207/events
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
        content-type: multipart/form-data; boundary=bo
[33363] upload audio................................
[36861] C<-S HEADERS
        :status: 200
        access-control-allow-origin: *
        x-amzn-requestid: 0ed21ffffef23a7d-000069fb-00025786-cd4862c89753e4bd-3736a86a-5
        content-type: multipart/related;boundary=36697d04-4d4e-4595-b6e6-a390edd0295a;start=metadata.1494256392289;type="application/json"
[37551] json: {"directive":{"header":{"namespace":"Speaker","name":"SetMute","messageId":"664d09e5-d664-4948-bbac-c70e08a1424d","dialogRequestId":"64567a10-0a67-48e6-8d66-4a936508f032"},"payload":{"mute":false}}}
[37606] json: {"directive":{"header":{"namespace":"SpeechSynthesizer","name":"Speak","messageId":"fe57a3bb-2280-4d88-9345-529e51cc30f2","dialogRequestId":"64567a10-0a67-48e6-8d66-4a936508f032"},"payload":{"url":"cid:DeviceTTSRenderer_0ae4a760-fee0-407a-b8d1-e5922c967805_1092807486","format":"AUDIO_MPEG","token":"amzn1.as-ct.v1.Domain:Application:Notifications#ACRI#DeviceTTSRenderer_0ae4a760-fee0-407a-b8d1-e5922c967805"}}}
[37718] download mp3.......
(3096)

• Connect to Alexa

Try to connect Alexa voice service which is based on HTTP2. Note that the time from connecting to receiving first HTTP data can’t exceed 10 seconds. You need to re-connect to Amazon again after this time period.
For example, we try to connect at 15.664 seconds and finish the SSL handshake at 24.210 seconds. It sends HTTP data at 24.239 seconds and receives HTTP data from Amazon at 24.546 seconds.

[15664] connecting...
[24210] handshake done
[24239] C->S HEADERS
        :method: GET
        :path: /v20160207/directives
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
[24308] C->S HEADERS
        :method: POST
        :path: /v20160207/events
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
        content-type: multipart/form-data; boundary=bo
[24546] C<-S HEADERS
        :status: 200
        access-control-allow-origin: *
        x-amzn-requestid: 0ed21ffffef23a7d-000069fb-00025786-cd4862c89753e4bd-3736a86a-1
        content-type: multipart/related; boundary=------abcde123; type=application/json
[25716] C<-S HEADERS
        :status: 204
        access-control-allow-origin: *
        x-amzn-requestid: 0ed21ffffef23a7d-000069fb-00025786-cd4862c89753e4bd-3736a86a-3

• Upload and Download Voice

The log below shows that Ameba uploads the data to Amazon when users push the button to start recording simultaneously. A comma “.” is printed for uploading each piece of data.

[33295] C->S HEADERS
        :method: POST
        :path: /v20160207/events
        :scheme: https
        :authority: avs-alexa-na.amazon.com:443
        accept: */*
        authorization: Bearer Atza|IwEBIJYyUjyCv8luvwAI
        content-type: multipart/form-data; boundary=bo
[33363] upload audio................................

It may wait for a while to get voice response contents after finish uploading.
The voice contents will be playing while downloading and it also prints a comma “.” for downloading each piece of data.

We don’t need to play the voice after finish downloading the contents completely. Take this log as an example, “(3096)” means the time period that users hear the voice reply from the time users release the button is 3.096 seconds. The shorter this value is, the shorter time delay users feel.

[36861] C<-S HEADERS
        :status: 200
        access-control-allow-origin: *
        x-amzn-requestid: 0ed21ffffef23a7d-000069fb-00025786-cd4862c89753e4bd-3736a86a-5
        content-type: multipart/related;boundary=36697d04-4d4e-4595-b6e6-a390edd0295a;start=metadata.1494256392289;type="application/json"
[37551] json: {"directive":{"header":{"namespace":"Speaker","name":"SetMute","messageId":"664d09e5-d664-4948-bbac-c70e08a1424d","dialogRequestId":"64567a10-0a67-48e6-8d66-4a936508f032"},"payload":{"mute":false}}}
[37606] json: {"directive":{"header":{"namespace":"SpeechSynthesizer","name":"Speak","messageId":"fe57a3bb-2280-4d88-9345-529e51cc30f2","dialogRequestId":"64567a10-0a67-48e6-8d66-4a936508f032"},"payload":{"url":"cid:DeviceTTSRenderer_0ae4a760-fee0-407a-b8d1-e5922c967805_1092807486","format":"AUDIO_MPEG","token":"amzn1.as-ct.v1.Domain:Application:Notifications#ACRI#DeviceTTSRenderer_0ae4a760-fee0-407a-b8d1-e5922c967805"}}}
[37718] download mp3.......
(3096)