Basic /speech integration

Summary:
* Basic `POST /speech` integration
* `!speech` support for `interactive`, using `mic` (3s)
* Kaizens.

Reviewed By: ruoyipu

Differential Revision: D34931521

fbshipit-source-id: db3d7ea3d2c5f1a6df5492c4ed1aade21d7adc4e

.gitignore

@@ -2,3 +2,4 @@
 node_modules
 .DS_Store
 package-lock.json
+yarn.lock

CHANGES.md

@@ -1,17 +1,26 @@
+## v6.1.1
+
+- Basic `POST /speech` integration.
+- `!speech` support for interactive.
+
 ## v6.1.0
+
 Bumped API version to `20210928`.
 Moved API version from `Accept` header to `v` HTTP parameter.
 Kaizens.
 
 ## v6.0.1
+
 Removed unused `request` dependency
 Updated various dependencies.
 
 ## v6.0.0
+
 Updated API version to latest: `20200513`.
 Browse the latest HTTP API documentation [here](https://wit.ai/docs/http/20200513#get__message_link).
 
 ## v5.0.0
+
 The most important change is the removal of `.converse()` and `.runActions()`. Follow the migration tutorial [here](https://github.com/wit-ai/wit-stories-migration-tutorial), or [read more here](https://wit.ai/blog/2017/07/27/sunsetting-stories).
 
 ### Breaking changes
@@ -21,6 +30,7 @@ The most important change is the removal of `.converse()` and `.runActions()`. F
 - updated wit-ai-basic-app-for-tests.zip for testing
 
 ## v4.3.0
+
 - `converse` and `runActions` are deprecated
 - `interactive` now calls `message`
 
@@ -51,7 +61,6 @@ We moved to a Promise-based API, instead of callbacks. This makes the code simpl
 
 See `./examples` to see how to use the new API.
 
-
 ### Breaking changes
 
 - `say` renamed to `send` to reflect that it deals with more than just text
@@ -66,6 +75,7 @@ See `./examples` to see how to use the new API.
 - allows for overriding API version, by setting `WIT_API_VERSION`
 
 ## v3.3.1
+
 - adding API versioning (defaults to `20160516`)
 - warns instead of throwing when validating actions
 - fixing null values when cloning context

README.md

@@ -30,64 +30,86 @@ See `examples/messenger.js` for a thoroughly documented tutorial.
 ### Overview
 
 The Wit module provides a Wit class with the following methods:
-* `message` - the Wit [message](https://wit.ai/docs/http/20200513#get__message_link) API
 
-You can also require a library function to test out your bot in the terminal. `require('node-wit').interactive`
+- `message` - the Wit [message](https://wit.ai/docs/http#get__message_link) API;
+- `speech` - the Wit [speech](https://wit.ai/docs/http#post__speech_link) API.
+
+You can also require a library function to test out your Wit app in the terminal. `require('node-wit').interactive`
 
 ### Wit class
 
 The Wit constructor takes the following parameters:
-* `accessToken` - the access token of your Wit instance
-* `logger` - (optional) the object handling the logging.
-* `apiVersion` - (optional) the API version to use instead of the recommended one
+
+- `accessToken` - the access token of your Wit instance
+- `logger` - (optional) the object handling the logging.
+- `apiVersion` - (optional) the API version to use instead of the recommended one
 
 The `logger` object should implement the methods `debug`, `info`, `warn` and `error`.
 They can receive an arbitrary number of parameters to log.
 For convenience, we provide a `Logger` class, taking a log level parameter
 
 Example:
+
 ```js
 const {Wit, log} = require('node-wit');
 
 const client = new Wit({
   accessToken: MY_TOKEN,
-  logger: new log.Logger(log.DEBUG) // optional
+  logger: new log.Logger(log.DEBUG), // optional
 });
 
 console.log(client.message('set an alarm tomorrow at 7am'));
 ```
 
 ### .message()
 
-The Wit [message](https://wit.ai/docs/http/20200513#get__message_link) API.
+The Wit [message](https://wit.ai/docs/http/#get__message_link) API.
 
 Takes the following parameters:
-* `message` - the text you want Wit.ai to extract the information from
-* `context` - (optional) the object representing the session state
+
+- `q` - the text input you want Wit.ai to extract the information from
+- `context` - (optional) the [Context](https://wit.ai/docs/http/#context_link) object
+- `n` - (optional) the max number of intents and traits to get back
 
 Example:
+
 ```js
 const client = new Wit({accessToken: 'MY_TOKEN'});
-client.message('what is the weather in London?', {})
-.then((data) => {
-  console.log('Yay, got Wit.ai response: ' + JSON.stringify(data));
-})
-.catch(console.error);
+client
+  .message('what is the weather in London?', {})
+  .then(data => {
+    console.log('Yay, got Wit.ai response: ' + JSON.stringify(data));
+  })
+  .catch(console.error);
 ```
 
+### .speech()
+
+The Wit [speech](https://wit.ai/docs/http#post__speech_link) API.
+
+Takes the following paramters:
+
+- `contentType` - the Content-Type header
+- `body` - the audio `Readable` stream
+- `context` - (optional) the [Context](https://wit.ai/docs/http/#context_link) object
+- `n` - (optional) the max number of intents and traits to get back
+
+See `lib/interactive.js` for an example.
+
 ### interactive
 
-Starts an interactive conversation with your bot.
+Starts an interactive conversation with your Wit app.
+Use `!speech` to send an audio request from the microphone, or enter any text input.
 
 Example:
+
 ```js
 const {interactive} = require('node-wit');
 interactive(client);
 ```
 
 See the [docs](https://wit.ai/docs) for more information.
 
-
 ## Changing the API version
 
 On May 13th, 2020, the `GET /message` API was updated to reflect the new data model: intents, traits and entities are now distinct.
@@ -97,11 +119,13 @@ You can target a specific version by passing the `apiVersion` parameter when cre
 ```json
 {
   "text": "hello",
-  "intents": [ {
-    "id": "1353535345345",
-    "name": "greet",
-    "confidence": 0.9753
-  } ],
+  "intents": [
+    {
+      "id": "1353535345345",
+      "name": "greet",
+      "confidence": 0.9753
+    }
+  ],
   "entities": [],
   "traits": []
 }

lib/interactive.js

@@ -4,34 +4,79 @@
 
 'use strict';
 
+const fs = require('fs');
+const mic = require('mic');
 const readline = require('readline');
 
-module.exports = (wit, handleMessage, context) => {
+const AUDIO_PATH = '/tmp/output.raw';
+const MIC_TIMEOUT_MS = 3000;
+
+module.exports = (wit, handleResponse, context) => {
   const rl = readline.createInterface({
     input: process.stdin,
     output: process.stdout,
   });
   rl.setPrompt('> ');
+
   const prompt = () => {
     rl.prompt();
     rl.write(null, {ctrl: true, name: 'e'});
   };
   prompt();
+
+  const makeResponseHandler = rsp => {
+    if (handleResponse) {
+      handleResponse(rsp);
+    } else {
+      console.log(JSON.stringify(rsp));
+    }
+    prompt();
+  };
+
   rl.on('line', line => {
     line = line.trim();
     if (!line) {
       return prompt();
     }
-    wit
-      .message(line, context)
-      .then(rsp => {
-        if (handleMessage) {
-          handleMessage(rsp);
-        } else {
-          console.log(JSON.stringify(rsp));
-        }
-        prompt();
-      })
-      .catch(err => console.error(err));
+
+    // POST /speech
+    if (line === '!speech') {
+      const microphone = mic({
+        bitwidth: '16',
+        channels: '1',
+        encoding: 'signed-integer',
+        endian: 'little',
+        fileType: 'raw',
+        rate: '16000',
+      });
+
+      const inputAudioStream = microphone.getAudioStream();
+      const outputFileStream = fs.WriteStream(AUDIO_PATH);
+      inputAudioStream.pipe(outputFileStream);
+
+      inputAudioStream.on('startComplete', () => {
+        setTimeout(() => {
+          microphone.stop();
+        }, MIC_TIMEOUT_MS);
+      });
+      inputAudioStream.on('stopComplete', () => {
+        const stream = fs.ReadStream(AUDIO_PATH);
+        wit
+          .speech(
+            'audio/raw;encoding=signed-integer;bits=16;rate=16000;endian=little',
+            stream,
+            context,
+          )
+          .then(makeResponseHandler)
+          .catch(console.error);
+      });
+
+      microphone.start();
+
+      return;
+    }
+
+    // GET /message
+    wit.message(line, context).then(makeResponseHandler).catch(console.error);
   });
 };