Wiki old

Golem: Build your own bot

TL;DR: Here's what you'll need to do to build your own bot:

Think up a useful bot
Figure out what phrases and entities it should understand and teach them to your Wit.ai app
Clone and setup our repo
Design the states of the conversation
Create actions for each state
- use templates for common actions such as text messages or asking for input
- implement python functions for custom actions such as returning items from the database
Profit!

How to use this module

First install the following tools:

Python 3.5+: https://www.python.org/downloads/

Redis database: http://redis.io/download

Ngrok (recommended, public tunnels to localhost): https://ngrok.com/

Django

The easiest way to use our module is using our example project.

Here's how to get started:

# create our project directory
mkdir mychatbot
cd mychatbot

# clone the example project
git clone https://github.com/prihoda/golem-example .

# create a virtual environment with python 3.6
virtualenv -p python3.6 env
source env/bin/activate

# install django-golem dependency
pip install ./requirements.txt

Wit and Facebook settings

Now you have to specify your settings in the example/settings.py file.

For Facebook Messenger: Follow the steps on Messenger API Quickstart

For Wit: create an account and and app on Wit.ai.

Running the server

Golem uses Django + Redis + Celery. First run the Redis server (in a separate terminal window):

# Run Redis database server
./PATH_TO_REDIS/src/redis-server

And the ngrok tunnel to make your localhost visible to Facebook:

# Run Ngrok tunnel to localhost
./PATH_TO_NGROK/ngrok http 8000

When you're done, you can finally run the chatbot server:

python manage.py migrate   # creates database
python manage.py runserver

And the celery workers that will process our messages:

# example == name of your django app
celery -A example worker -l info
celery -A example beat -l info

And you're done!

Context

Context a storage of entity values extracted from message text. When entities are parsed from a user's message we push the values to the context.

We also keep the previous values. Each value in the context has an age which starts at 0 and is incremented with each user message. This means that entities received in the current message have age 0, the ones in the previous one have age 1, etc.

Context values

For each entity, the context stores a list of its values. An entity value is a dictionary with the following fields:

query = context.get('query')
# query:
{
  'value' : 'concert',  # Value field, contains the extracted value
  'age' : 2,            # Age of the value (0 = received in last message, 1 = previous, ...)
  'metadata' : { ... }, # Optional metadata field from Wit metadata of this value (string parsed as JSON)
  'confidence' : 0.81346740881314 # Wit confidence about the parsed entity
  'counter' : 65, # Context counter used to calculate the value age, can be ignored
  'custom_field' : 'other value', # any custom field can be added when adding values to context via a button payload
  ...
}

Special entities

Some entities have special meaning:

_state: Used to move to a different state when received in payload
_message_text: Plaintext of the message received from the user

Context functions

Context can be accessed and modified using the following functions:

context.get(entity, max_age=None, key='value', ...)

Get the single newest value of an entity up to max_age. Returns None if no value is present.

The key parameter defines whether to return the whole dictionary (key=None) or only the value field (key='value').

query = context.get('query', max_age=0)
print(query) # 'concert'
query = context.get('query', max_age=0, key=None)
print(query) # {'value' : 'concert', 'age':0, ...}
text = context.get('_message_text')
print(text) # 'Are there any concerts?'

context.get_all(entity, max_age=None, key='value', ...)

Get list of all values of an entity up to max_age.

queries = context.get('query', max_age=5)
print(queries) # ['concert','exhibition']

context.get_age(entity, max_age=None, key='value', ...)

Shortcut to get value, age pair:

query_value, query_age = context.get_age('query')
# Is the same as calling
query = context.get('query', key=None) or {}
query_value = query.get('value')
query_age = query.get('age')

Returns None, None if no value up to max_age is present.

context.get_all_first(self, entity, max_age=None, key='value')

Get list of all newest values (received in the same message) up to max_age. Returns empty list if no values are present.

context.has_any(self, entities, max_age=None):

Check whether the context contains values of any of the specified entities. Can be used to check whether this state understands the message or whether we should manually move somewhere else (e.g. to the default state).

context.set(self, entity, value_dict):

Add a custom value with age=0 to the context:

context.set('tag', {'value':'rock'})
context.set('query', {'value':'concert', 'metadata':{'some':'field'}})

Flows and states

The dialog is managed by flows. Each flow represents one or more intents and contains several states.

Your dialog begins in a flow called default. Each flow starts in the root state.

Flows are defined in python like so:

{
    'default' : { # name of the flow, automatically accept intent of the same name
        'intent' : '(greeting|help)', # regex of additional intents to accept 
        'states' : { # states of the flow
            'root' : { # the root state, required. Will move here on intent 'greeting' or 'help'
                'accept' : { # action defined by a template
                    'template' : 'message',
                    'params' : {
                        'message' : 'Hi, welcome to IT support! :)',
                        'next' : 'question'
                    }
                }
            },
            'question' : { 
                'init' : my_custom_action, # action defined by a function
                'accept' : my_input_action
            },
            'like' : {
                'intent' : 'like', # move to this state if intent 'like' is accepted in the 'default' flow
                'accept' : my_like_action
            }
         ...
        }
    }

States

State represents state of the conversation, each state provides custom actions.

A state can be referenced by a string path: 'flow_name.state_name'. This is used when we want to move to this state from other states. For example, the 'question' state above is represented by the string 'default.question'.

When referencing a state in the same flow, it is possible to use only 'state_name' without the flow_name prefix.

Actions

Each state has two actions - init and accept.

Init is run when you arrive directly from another state. It can be used to send messages to the user without requesting any input.

Accept is run when a message from the user is received. It can be used for processing input and returning a response.

An action can be defined either by a template or by a function.

Custom function actions

You can define actions by referencing python functions.

For example, this function will send a message and move to the 'default.how_are_you' state, where it will wait for user response.

def action_intro(state):
    message = 'Hi, how are you?'
    return message, 'default.how_are_you'

The function takes the parameter state which is used to access the context and other properties of the dialog. It returns a tuple:

A response (a String or a Response object) or an array of responses
Name of the new state (or None to stay in current state)
- Add ':accept' to the state name to run the accept action right after moving, as if the user received a message in the new state.
- If no action suffix is specified, the init action will be executed

Consider this function that recommends items based on a query, time and budget:

def action_recommend(state):
    query = state.dialog.context.get('query', max_age=0) # get entity query (from last message)
    time = state.dialog.context.get('datetime', max_age=0) # get entity datetime (from last message)
    budget = state.dialog.context.get('budget') # get entity budget from any previous message

    results = look_for_something(at=time, query=query, budget=budget)

    response = []
    response.append(TextMessage('Ok, got it.'))
    response.append('Here\'s what I found:')
    carousel = GenericTemplateMessage()
    for result in results:
      element = carousel.create_element(title='Title', image_url='http://example.com/image.jpg')
      element.create_button(title='Show', url='http://example.com')
    response.append(carousel)
    return response, None # return None to stay in current state

Since None is returned, we will stay in the same state. Now, when any message is received, we will run the action again, returning an updated response.

However, if none of the 'query', 'time', and 'budget' entities are extracted, the returned message will be exactly the same. In this case, we rather want to move to a different state and check what to do. We can improve our action like this:

def action_recommend(state):
    if not state.dialog.context.has_any(['_state','intent','query','budget','time'], max_age=0):
        return None, 'default.root:accept'
    ...

Since this is quite common, we provide a shorthand for this called require_one_of:

from golem.core.flow import require_one_of

@require_one_of(['query','budget','time'])
def action_recommend(state):
    ...

When calling the function, we will automatically move to the 'default.root' state if none of the supplied messages were provided in the last user message.

And in the default.root state, we will decide where to go based on extracted entities:

def action_default(state):
    # If 'query' or 'budget' entity is present, move to the search.root state and call the accept action
    if state.dialog.context.has_any(['query','budget'], max_age=0):
        return None, 'recommend.root:accept'
    # If 'currency' entity is present, move to currency.update state and call the accept action
    if state.dialog.context.has_any(['currency'], max_age=0):
        return None, 'currency.update:accept'
    
    return 'Not sure what you mean :P', None

Action templates

Templates are used for common simple actions like sending text messages or requesting input.

  ...
  'root' : {
      'accept' : {
          'template' : 'message', # template type
          'params' : { # template params 
              'message' : 'Hi, welcome to IT support! :)', # message to send
              'next' : 'question' # name of next state
          }
      }
  }
  ...

See the Action templates section below for all templates.

How it works

This is how the whole process works:

Wait for new message
Parse message using Wit
- If _state entity is received in message, move to _state. Uused in button callbacks. If _state has :action suffix, run the action, otherwise run the init action.
- Otherwise if intent is received in message:
  - Look at all states of the current flow. If they have an 'intent' field that matches the intent, move there.
  - Look at all flows, if a flow's name or its 'intent' field matches the current intent, move to the flow's root state.
Run accept action of current state, return tuple (messages, new_state).
Send messages returned by action (if any).
Check state name returned by action:
- If None, stay in current state. Continue from 1.
- Otherwise if negative number, move back that number of times and run the init action. For example, -1 moves to the previous state.
- Otherwise if state name has ':init' suffix or no suffix, move there and continue from 6.
- Otherwise if state has ':accept' suffix, move there and continue from 3.
Run init action of new state, return tuple (messages, new_state) and continue from 4. Or if no init action was defined, continue from 1.

To sum up, moving between states can be done in three ways:

By receiving the 'intent' entity. This way we can move between states of the current flow, or to the root state of a different flow.
By receiving the '_state' entity, which contains a state name (optionally with action suffix). This way we can move to any state of any flow when the user clicks a quick reply or button.
By returning a state name as a second parameter of a custom function (optionally with action suffix), this works same as the '_state' entity. This way we can move between states right away.

Response types

Docs in progress, check the golem/core/responses.py file.

Text message

# can be defined directly as string
message = "Hi :) How can I help you?"
# or as a TextMessage (this way we can add buttons and quick replies)
message = TextMessage("Hi :) How can I help you?")

Buttons

message = TextMessage("Hi :) How can I help you?")
# Open a webpage
message.create_button(title='Example', url='http://example.com')
# Or send payload - add entities to the context
message.create_button(title='Suggestions', payload={'query':{'value':'concert', 'metadata' : { ... }}})
# Entity values can be also passed directly (without the dict wrapper):
# The '_state' entity is used to move to a different state
message.create_button(title='Suggestions', payload={'_state':'faq.root:accept'})

Quick replies

message = TextMessage("Hi :) How can I help you?")
# With payload, when clicked, behaves just like a button
message.create_quick_reply(title='Top events', payload={'intent':'search'})
# Or without payload to be parsed as text
message.create_quick_reply(title='Weather')

Generic template

carousel = GenericTemplateMessage()
# Element with image (full-sized image is downloaded!)
element = carousel.create_element(title='Required title', subtitle='Optional subtitle', image_url='Optional image url')
# Add button to element just like to a text message
element.create_button(title='Read article', url='http://example.com')

Attachment

message = AttachmentMessage(attachment_type='image', url='http://example.com/image.jpg')

Action templates

Message

Usage: Send one or more pre-defined messages.

{
    'type' : 'message',
    'params' : {
        'message' : ['Hi :)','How are you?'],
        'next' : 'search.root'
    }
}

Parameter	Param type	Default value	Description
message	Response, String or a list of these	required	List of messages to send to the user
next	String	None	Name of the state to transfer to

Input

Usage: Request input of a specific entity

{
    'template' : 'input',
    'params' : {
        'entity' : 'yes_no',
        'missing_message' : TextMessage('Choose one :)', quick_replies=[
            {'title':'Yes'},
            {'title':'No'}
        ]),
        'next' : 'search.root'
    }
}

Parameter	Param type	Default value	Description
entity	String	required	Name of entity to wait for
text	Boolean	False	If True, accept raw text input and save it into the specified entity (in this case, missing_message is never used)
missing_message	Response, String or a list of these	None	Text response to send when the entity is not received in the message
next	String	None	Name of the state to transfer to after successful input

Value transition

Usage: Change states based on entity value (e.g. after input).

Uses max_age=0, which means only values received in the last message are used.

{
    'template' : 'value_transition',
    'params' : {
        'entity' : 'yes_no',
        'transitions' : {
            'yes' : 'show_result',
            'no' : 'say_sorry',
            None : 'some_question_state'
        },
        'next' : 'some_question_state'
    }
}

Parameter	Param type	Default value	Description
entity	String	required	Name of entity to check
transitions	Dictionary of (value: state) pairs	required	Dictionary that maps values to states. Use (None: 'state') for handling missing values.
next	String	None	State to transfer to if value is not present in transition keys

To see all the templates in action check out the Example chatbot.

Platforms

Currently only Facebook Messenger platform is supported.

Contact us at david.prihoda at gmail.com, for public questions submit an issue.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wiki old

Golem: Build your own bot

How to use this module

Django

Wit and Facebook settings

Running the server

Context

Context values

Special entities

Context functions

context.get(entity, max_age=None, key='value', ...)

context.get_all(entity, max_age=None, key='value', ...)

context.get_age(entity, max_age=None, key='value', ...)

context.get_all_first(self, entity, max_age=None, key='value')

context.has_any(self, entities, max_age=None):

context.set(self, entity, value_dict):

Flows and states

States

Actions

Custom function actions

Action templates

How it works

Response types

Text message

Buttons

Quick replies

Generic template

Attachment

Action templates

Message

Input

Value transition

Platforms

Clone this wiki locally