NLU and Context

Natural language understanding

Let's say you want a chatbot that people can use to book plane and train tickets, among other things.

If a user sends the message Find a flight from Prague to London for tomorrow, it would be nice if the bot could understand:

• that the user is looking for a flight - the intent,
• that the user wants to go from Prague to London - the location,
• that the user is just interested in tickets for tomorrow - the datetime.

It turns out that many bots today can't do this.

First thought: you could use regular expressions. Would that work? Maybe for some sentences, yes. But people don't always write everything in the same way and it would be really difficult, if not impossible, to find the perfect regex.

A better option would be to use a NLU service. NLU stands for Natural Language Understanding and there are many free online services today like Facebook's Wit.ai or Google's Dialogflow.

Basically, it works like this:

• You give example sentences to the NLU service
• You specify labels for these sentences (e.g. "Hello world" -> {intent: greeting})
• The NLU service uses AI algorithms to predict labels for previously unseen sentences Of course, this is far from perfect and many times, the sentences may be misclassified. The NLU service will always give you a confidence between 0 and 100 %.

The more example sentences you give, the better it will work.

Entity extraction

We make it easy to use these with the EntityExtractor modules. Right now, you can use Wit.ai or your own logic for entity extraction. The config looks like this:

(settings.py)

GOLEM_CONFIG = {
    "ENTITY_EXTRACTORS": [
        WitExtractor("MY WIT TOKEN"),  # Wit.AI
        GolmNLU(),           # our own custom NLU engine (contact us for more information)
        CourseCodeParser(),  # your own parser (used here to detect university course codes)
    ],
}

The Context

So how can you actually use these entities the NLU has extracted?

There is one special entity, the intent, that tells us the meaning of the sentence - what the user wants, that will have special meaning when you're writing conversation flows. Let's skip it for now.

You can get to the entities through class Context, that has methods for querying and filtering. For example:

TODO

context.get("location")             # returns the last known entity location
context.get("location", max_age=0)  # returns the location received in last message
context.has_any(["person", "company"], max_age=1)  # checks if there are any such entities

TODO