Say Hello to Red Samurai Contextual Chatbot with TensorFlow Deep Neural Network Learning

We are building our own enterprise chatbot. This chatbot helps enterprise users to run various tasks - invoice processing, inventory review, insurance cases review, order process - it will be compatible with various customer applications. Chatbot is based on TensorFlow Machine learning for user input processing. Machine learning helps to identify user intent, our custom algorithm helps to set conversation context and return response. Context gives control over  sequence of conversations under one topic, allowing chatbot to keep meaningful discussion based on user questions/answers. UI part is implemented in two different versions - JET and ADF, to support integration with ADF and JET applications.

Below is the trace of conversations with chatbot:

User statement Ok, I would like to submit payment now sets context transaction. If word payment is entered in the context of transaction, payment processing response is returned. Otherwise if there is no context, word payment doesn't return any response. Greeting statement - resets context.

Intents are defined in JSON structure. List of intents is defined with patterns and tags. When user types text, TensorFlow Machine learning helps to identify pattern and it returns probabilities for matching tags. Tag with highest probability is selected, or if context was set - tag from context. Response for intent is returned randomly, based on provided list. Intent could be associated with context, this helps to group multiple related intents:

Contextual chatbot is implemented based on excellent tutorial - Contextual Chatbots with Tensorflow. Probably this is one of the best tutorials for chatbot based on TensorFlow. Our chatbot code follows closely ideas and code described there. You could run the same on your TensowFlow environment - code available on GitHub. You should run model first and then response Python notebooks.

Model notebook trains neural network to recognize intent patterns. We load JSON file with intents into TensorFlow:

List of intent patterns is prepared to be suitable to feed neural network. Patterns are translated into stemmed words:

Learning part is done with TensorFlow deep learning library - TFLearn. This library makes it more simple to use TensorFlow for machine learning by providing higher-level API. In particular for our chatbot we are using Deep Neural Network model - DNN:

Once training is complete and model is created, we can save it for future reuse. This allows to keep model outside of chatbot response processing logic and makes it easier to re-train model on new set of intents when required:

In response module, we load saved model back:

Function response acts as entry point to our chatbot. It gets user input and calls classify function. Classification function, based on learned model, returns list of suggested tags for identified intents. Algorithm locates intent by its tag and returns random reply from associated list of replies. If context based reply is returned, only if context was set previously:

Stay tuned for more blog posts on this topic.