Creating Smart Chatbots Using NLP: A Complete Beginner’s Guide to Intelligent Conversational Agents

📗 Chapter 3: Understanding Intents and Entities (NLU)

Train Your Chatbot to Understand What the User Really Means

🧠 Introduction

At the heart of every smart chatbot lies the ability to understand what the user wants (intent) and what information they’re providing (entities). This layer of intelligence is called Natural Language Understanding (NLU).

NLU allows your chatbot to go beyond keywords and truly grasp user intent while extracting actionable data from their messages.

In this chapter, we’ll explore:

• What intents and entities are
• How to create and label them
• How NLU pipelines work
• Tools like spaCy, Rasa NLU, and Hugging Face Transformers
• Step-by-step coding examples

Let’s teach your bot how to understand.

📘 Section 1: What Are Intents and Entities?

🔹 Intent

An intent represents the goal or purpose of a user message.

🔹 Entity

An entity is a piece of information extracted from the input — like a name, location, date, or product.

📘 Section 2: How Intents & Entities Work Together

A chatbot interprets each message by first classifying the intent, then extracting entities.

📌 Example:

User: "I'd like to book a flight to Delhi tomorrow."

• Intent: book_flight
• Entities:
• destination: Delhi
• date: tomorrow

📘 Section 3: Intent Classification – Code with scikit-learn

python

from sklearn.feature_extraction.text import CountVectorizer

from sklearn.linear_model import LogisticRegression

from sklearn.pipeline import make_pipeline

X = [

    "Book me a flight to Delhi",

    "Cancel my booking",

    "I want to fly to Mumbai",

    "What’s the weather in Paris?",

    "Please cancel the trip"

]

y = [

    "book_flight",

    "cancel_booking",

    "book_flight",

    "get_weather",

    "cancel_booking"

]

model = make_pipeline(CountVectorizer(), LogisticRegression())

model.fit(X, y)

# Test it

print(model.predict(["I want to book a trip to Tokyo"]))

📘 Section 4: Entity Recognition with spaCy

python

import spacy

nlp = spacy.load("en_core_web_sm")

text = "Book a flight to Mumbai for tomorrow"

doc = nlp(text)

for ent in doc.ents:

    print(ent.text, ent.label_)

Output:

java

Mumbai     GPE (Geo-political entity)

tomorrow   DATE

spaCy uses pre-trained pipelines to recognize common entities. You can also train custom entity types (e.g., origin_city, hotel_name).

📘 Section 5: Using Rasa NLU for Intent + Entity Recognition

📁 nlu.yml Example:

yaml

version: "2.0"

nlu:

- intent: book_flight

  examples: |

    - I want to book a flight to [Paris](destination)

    - Can you get me a ticket to [Delhi](destination)?

    - Fly me to [Mumbai](destination) tomorrow

- intent: cancel_booking

  examples: |

    - I need to cancel my trip

    - Cancel my booking

    - Please cancel my reservation

CLI Commands:

bash

rasa train nlu

rasa shell nlu

Test:

vbnet

User: Book a flight to Pune for next week

Bot NLU:

 - intent: book_flight

 - entities: destination: Pune, date: next week

📘 Section 6: Slot Filling and Entity Storage

Once an entity is extracted, it should be stored in a slot to be reused later.

Example:

python

user_input = "Book a table for 2 at 7 PM"

slots = {

    "party_size": 2,

    "time": "7 PM"

}

📘 Section 7: Handling Ambiguous Intents

Sometimes messages are unclear or map to multiple intents.

Example:

“I need to go to Delhi”

Is this:

• A booking request?
• A price inquiry?
• Just a location query?

Solution:

• Ask follow-up questions
• Train with more diverse examples
• Use confidence scores from your model

📌 Sample Intent Prediction with Confidence

python

probs = model.predict_proba(["I need to go to Delhi"])

for intent, prob in zip(model.classes_, probs[0]):

    print(f"{intent}: {round(prob, 2)}")

📘 Section 8: Creating Custom Entities with spaCy

python

from spacy.pipeline import EntityRuler

ruler = nlp.add_pipe("entity_ruler", before="ner")

patterns = [{"label": "CITY", "pattern": "Tokyo"}]

ruler.add_patterns(patterns)

doc = nlp("I want to travel to Tokyo")

for ent in doc.ents:

    print(ent.text, ent.label_)

📘 Section 9: Intent and Entity Training Tips

📘 Section 10: Testing & Evaluation

Metrics to Track:

Use sklearn.metrics for evaluation and Rasa test CLI for model quality inspection.

✅ Chapter Summary Table