Build a chatbot to query your documents using Langchain and Azure OpenAI (2023)

In this article, I introduce LangChain and explore its capabilities by building a simple question answer to query PDFs in the Azure Functions documentation.

wavy necklace

Harrison Chasewavy necklaceis a powerful Python library that simplifies the process of building NLP applications with large language models. The main goal is to create intelligent agents that can understand and execute human language instructions. With LangChain, you can connect to various data and computing resources and build applications that perform NLP tasks on domain-specific data sources, private repositories, and more.

As of May 2023, the LangChain GitHub repository has received over 42,000 stars and contributions from over 270 developers around the world.

Build a chatbot to query your documents using Langchain and Azure OpenAI (1)

The langchain library consists of several modules:

(Video) Build AI chatbot with custom knowledge base using OpenAI API and GPT Index

  • LLM and tips

This includes just-in-time management, just-in-time optimization, a common interface for all LLMs, and common tools for working with LLMs such as Azure OpenAI. It supports several LLMs, including OpenAI, LLama, and GPT4All.

  • the chain

A chain in LangChain refers to a series of calls that can be chained together to perform a specific task. For example, you may need to retrieve data from a specific URL, summarize the returned text, and use the resulting summary to answer questions. Chains can also be as simple as reading user input, building a prompt, and generating a response.

  • Data augmentation generation

Data augmentation generation involves specific types of chains that first communicate with external data sources to obtain data for use in the generation step. Examples are long text summaries and question/answer for specific data sources. LangChaindocumentladerIntoolModules help connect to data sources and calculations. If you combine text files, PDF documents, HTML web pages, etc., you can use the document loader in Langchain.

  • intermediary

Agency involves the LLM in deciding what action to take, taking that action, observing the observation, and repeating it until it is done.

As we explained earlier, chains can help chain together a series of LLM calls. However, in some tasks, the sequence of calls is often undefined, and the next step will depend on user input and responses from previous steps.

intermediary' can respond to input along the way instead of a hard-coded deterministic sequence.

  • memory

Memory refers to the persistent state using VectorStores. Vector databases are optimized for fast searching in high-dimensional spaces. LangChain makes this effortless.

(Video) How To Use Langchain With Azure OpenAI


Embeddings are the mapping of discrete categorical variables to continuous numerical vectors.In the context of neural networks, embeddingis a low-dimensional learned continuous vector representation of discrete variables. Embedding neural networks is useful because they reduce the dimensionality of categorical variables and represent categories in the transformed space in a meaningful way.

Neural network embedding has 3 main goals:

  • Find nearest neighbors in the embedding space. These can be used to make recommendations based on user interests or cluster categories.
  • As input to machine learning models for supervised tasks.
  • Used for visualization of relationships between concepts and categories.

Clustering Example of Sentence Vector Values

Build a chatbot to query your documents using Langchain and Azure OpenAI (2)

vectoropslag of vectordatabase

A vector database is a special type of database that stores data as high-dimensional vectors. These vectors are mathematical representations of characteristics or properties of the stored data. The number of dimensions in each vector can vary widely, from tens to thousands, depending on the complexity and detail of the data. In this article, we will explore the concept of vector databases and their applications in various fields.

Let's build the application

Build a chatbot to query your documents using Langchain and Azure OpenAI (3)

Let's build a tool that can read developer documentation– in this case the Azure Functions documentation in PDF format.

(Video) How To Get Answers From Your PDF Using Azure OpenAI And Langchain

TonThen answer each question by referring to the document text.

We will follow these steps:

One-time procedure:

  • Indexes pdf documents (azure function docs), divides documents into pieces, indexes all created embedded text.
  • Store all embeddings in a vector store (in our case Faiss) which is searchable in the application.


  • When a user asks a question, we use the FAISS vector index to find the best matching text.
  • Enter this as context in the prompt for GPT-3.5
  • GPT-3.5 generates answers that accurately answer questions.

Build a chatbot to query your documents using Langchain and Azure OpenAI (4)


  • Download documents to search. In our case, we can download the Azure Functions documentation fromhereand save it in the data/documents folder.
  • Deploy to Azure OpenAI
    • Ada
    • GPT35

Build a chatbot to query your documents using Langchain and Azure OpenAI (5)

(Video) Create ChatBot Based On The Data Feed By You - GPT-Index | OpenAI | Python

Get the Azure OpenAI endpoint and key and add them to a file named .env as follows:


create containment


  • pdf barn
  • Split all text into chunks.
  • These chunks are sent to the OpenAI Embedding API, which returns a 1536-dimensional vector for each chunk.
  • Index all vectors in a FAISS index.
  • Save the FAISS index in a .To trustIn.pkldocument.

Note: As you probably know, LLM cannot accept long statements due to token restrictions, so we've split the document into chunks, see below.

Running this code takes time because we have to read and split the entire document and send the chunks to the Ada model to get the embeddings.

Here's the

from langchain.document_loaders import PyPDFLoader from langchain.embeddings.openai import OpenAIEmbeddings from langchain.vectorstores import FAISSfrom dotenv import load_dotenvimport openaiimport os#load environment variablesload_dotenv()OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")OPENAI_DEPLOYMENT_ENDPOINT = os.get环境(“OPENAI_DEPLOYMENT_ENDPOINT ”)OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")OPENAI_MODEL_NAME = os.getenv("OPENAI_MODEL_NAME")OPENAI_EMBEDDING_DEPLOYMENT_NAME = os.getenv("OPENAI_EMBEDDING_DEPLOYMENT_NAME")OPENAI_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_EMBEDDING _MODEL_NAME")OPENAI_DEPLOYMENT_VERSION = os.getenv(" OPENAI_DEPLOYMENT_VERSION")# init Azure OpenAIopenai.api_type = "azure"openai.api_version = OPENAI_DEPLOYMENT_VERSIONopenai.api_base = OPENAI_DEPLOYMENT_ENDPOINTopenai.api_key = OPENAI_API_KEYif __name__ == "__main__": embeddings = OpenAIEmbeddings(模型=OP ENAI_EMBEDDING_MODEL_NAME, chunk_size=1 ) dataPath = "./ data/documentation/ " fileName = dataPath + "azure-azure-functions.pdf" #use langchain PDF loader loader = PyPDFLoader(fileName) #split the document in chunks pages = loader.load_and_split() #Use Langchain 使用文本嵌入创建嵌入ada-002 db = FAISS.from_documents(documents=pages, embedding=embeddings) #将嵌入保存到 FAISS 向量存储中 db.save_local("./dbs/documentation/faiss_index")

make application

Build a chatbot to query your documents using Langchain and Azure OpenAI (6)

trafficapplicatie.pyWorks as:

  • FAISS index loaded in RAM
  • user question
  • The user's query is sent to the OpenAI Embeddings API, which returns a 1536-dimensional vector.
  • Look up the FAISS index for the closest vector match.
  • Returns the closest vector and the text it produced.
  • The returned text is entered in GPT-35 as context in the GPT-35 prompt
  • GPT-35 generates a response that is sent back to the user.

NOTE: The important thing to note is that Langchain does most of the heavy lifting for us, behind the scenes.

Here's the

从 langchain.vectorstores 手机 FAISSfrom dotenv import load_dotenvimport openaiimport osfrom langchain.chains import RetrievalQAfrom langchain.vectorstores 拉手 FAISSfrom langchain.chains.question_answering import load_qa_chainfrom dotenv import load_dotenvfrom langchain.chat_models import AzureChatOpenAIfrom langchain.embedding s.openai import OpenAIEmbed dings# 英语下载在电影load_dotenv ( )OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")OPENAI_MODEL_NAME = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT") v( "OPENAI_MODEL_NAME ") DDING_DEPLOYMENT_NAME ")OPENAI_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_EMBEDDING_MODEL_NAME")OPENAI_DEPLOYMENT_VERSION = os.getenv("OPENAI_DEPLOYMENT_VERSION")#init Azure OpenAIopenai.api_type = "azure" openai.api_base = OPENAI_DEPLOYMENT_VERSION OYMENT_ENDPOINTopenai.api_key = OPENAI_API_KEYdef ask_question( qa, vraag ): result= qa({"query": vraag}) print("Vraag:", vraag) print("Antwoord:", result["resultat"])if __name__ == "__main__" : #init openai llm = AzureChatOpenAI(deployment_name =OPENAI_DEPLOYMENT_NAME,model_name=OPENAI_MODEL_NAME,openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,openai_api_version=OPENAI_DEPLOYMENT_VERSION,openai_api_key=OPENAI_API_KEY) embeddings = OpenAI Embeddings(model=OPENAI_EMBEDDING_MODEL_NAME,chunk_size=1) store vector Store = FAISS.load_local(". / dbs/documentation /faiss_index", embeddings) #使用我们的faiss影兰最作来杰地地Documentation retriever = vectorStore.as_retriever(search_type="similarity", search_kwargs={"k":2}) 更多之是电视器 qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=False) while True: query = input('you: ') if query == 'q': break ask_question ( qa, inquire)

Now we can and start asking:

(Video) LangChain + OpenAI to chat w/ (query) own Database / CSV!

You: what is an azure function? Question: What is an Azure function? Answer: Azure Functions is an on-demand cloud service that provides all the constantly updated infrastructure and resources needed to run your applications. You can use the most efficient language to focus on the code that matters most to you, and Functions will do the rest. Functions provides serverless computing for Azure. You can use Functions to build web APIs, respond to database changes, handle IoT flows, manage message queues, and more.
You: Can I use Event Hub as a trigger for an Azure function? Question: Can I use Event Hub as a trigger for an Azure function? Answer: Yes, you can use Event Hubs as a trigger for an Azure function. `Azure Functions supports event hub triggers and output bindings. Use function triggers to respond to events sent to the Event Hubs event stream. You must have read access to the underlying event hub to set triggers. When the function is triggered, the message passed to the function is typed as a string. `<|im_end|>
You: What languages ​​can I use to build Azure functions? Question: What languages ​​can I use to build Azure functions?
You: Can I deploy Azure features in multiple regions? Question: Can I deploy Azure functions in multiple regions? Answer: Yes, you can deploy Azure features in multiple regions. There are two modes to consider: active/active for HTTP-triggered functions and active/passive for event-driven non-HTTP-triggered functions. When using active/active mode for HTTP-triggered functions, Azure Front Door is required to coordinate requests between the two regions. When using active/passive mode, a second region is activated and takes over processing when failover is required. For more information on multi-region deployments, see our guide in Highly Available Multi-Region Web Applications. <|im_end|>

Thanks for reading, I hope you enjoy it.



How to build a chatbot Azure? ›

Create an Azure Bot resource
  1. Go to the Azure portal.
  2. In the right pane, select Create a resource.
  3. In the search box enter bot, then press Enter.
  4. Select the Azure Bot card.
  5. Select Create.
  6. Enter the required values.
  7. Select Review + create.
  8. If the validation passes, select Create.
Oct 17, 2022

Which Azure cognitive services can you use to build conversation AL solutions? ›

  • Azure Managed Instance for Apache Cassandra. Cloud Cassandra with flexibility, control and scale.
  • Azure Database for MariaDB. Managed MariaDB database service for app developers.

What can bots built by using the Azure bot Service communicate with? ›

Configure chatbots to engage with customers and employees in a wide range of languages and channels including websites, mobile apps, Facebook, and Microsoft Teams.

Which service is used when building a no code question and answers bot for Microsoft teams? ›

SharePoint list is used as a database to feed QnAMaker knowledge base. QnAMaker Cognitive Service is used for building a knowledge base and the community bot was created automatically in QnAMaker portal with zero code.

How can I create my own chatbot? ›

Build Your Own Chatbot With ChatGPT API (2023)
  1. Things to Remember Before You Build an AI Chatbot.
  2. Set Up the Software Environment to Create an AI Chatbot. Install Python. Upgrade Pip. ...
  3. Get the OpenAI API Key For Free.
  4. Build Your Own AI Chatbot With ChatGPT API and Gradio.
  5. Create Your Personalized ChatGPT API-Powered Chatbot.
Mar 7, 2023

Which Azure cognitive service can be used to identify documents? ›

Azure Form Recognizer can analyze and extract information from government-issued identification documents (IDs) using its prebuilt IDs model.

Which Azure Cognitive Services service can be used to identify documents? ›

Form Recognizer is an AI service that applies advanced machine learning to extract text, key-value pairs, tables, and structures from documents automatically and accurately.

How do I create an Azure Cognitive Search service? ›

Find your search service and on the Overview page, select Import data on the command bar to set up cognitive enrichment in four steps.
  1. Step 1 - Create a data source. In Connect to your data, choose Azure Blob Storage. ...
  2. Step 2 - Add cognitive skills. ...
  3. Step 3 - Configure the index. ...
  4. Step 4 - Configure the indexer.
May 31, 2023

What is the difference between Azure bot and bot service? ›

Bot Channels Registration and Azure Bot are basically the same capabilities renamed. The UX is slightly different in the Azure portal to help customers connect to the Bot Framework Composer.

What is the difference between bot framework and Azure bot service? ›

What's the difference between Azure Bot Service and Bot Framework SDK and tools? Bot Framework is comprised of an open-source SDK and tools for end-to-end bot development. Microsoft Azure Bot Service helps you manage, connect, and deploy your bot across devices and popular channels.

Can you communicate with a bot by using Microsoft? ›

Statement 1: You can communicate with a bot by using Cortana.

Which API can be used to build chatbots that are capable of natural interaction with human beings? ›

A chatbot API is an API that offers features for developers to build chatbots. Chatbot APIs help combine natural language processing (NLP) to build an efficient chatbot—they take in the request, extract the intent or meaning of the message, and deliver the response.

What is the best chatbot for writing code? ›

Best AI Chatbots for 2023
RankAI Chatbot
4.Microsoft Bot Framework
12 more rows
Mar 14, 2023

How to use bot framework without Azure? ›

Also, you can register your bot and even enable the channels without deploying to Azure or any other hosting. You can temporarily use ngrok to create a secure tunnel to your localhost environment and test the bot in your email channel before exposing it to other users.

What's the best strategy when creating a chatbot? ›

  1. Gather information about your target audience from a variety of sources.
  2. Plan the type of chatbot, and what the bot is going to do to meet customer expectations.
  3. Select a platform & build your bot to create a great chatbot experience.
  4. Check if the chatbot works & improve it further.

What programming language is used for chatbots? ›

Java. You can choose Java for its high-level features that are needed to build an Artificial Intelligence chatbot. Coding is also seamless because of its refined interface. Java's portability is what makes it ideal for chatbot development.

What is the easiest chatbot builder to use? ›

Aivo is one of the chatbot builders that offer conversational artificial intelligence. This can help your brand with customer service and keep the authenticity while you chat with clients. It's easy to use, so you can create your bot, launch it, and track its performance with analytics effectively.

How to make a chatbot without coding for free? ›

To create a no-code chatbot, you need to follow the following steps:
  1. Identify the purpose of your chatbot. ...
  2. Where do you want it to appear? ...
  3. Choose a platform. ...
  4. Design your chatbot. ...
  5. Test your chatbot. ...
  6. Train your chatbot. ...
  7. Collect feedback from users. ...
  8. Use analytics to make your chatbot better.

How to create a chatbot without coding? ›

How to Make a Chatbot for a Website in Minutes
  1. Set Up Free Landbot Account.
  2. Optimize the Welcome Message.
  3. Add Your First Sequence.
  4. Ask a Question (Name)
  5. Ask Questions (Button Choice)
  6. Ask a Question (Email)
  7. Export Data to Google Sheets.
  8. Ask a Question (Buttons with Pics)

What are the 3 types of data that can be stored in Azure? ›

There are 4 types of storage in Azure, namely:
  • File.
  • Blob.
  • Queue.
  • Table.
May 3, 2017

What does Azure use for a document database? ›

Azure DocumentDB is a NoSQL document database that uses the JSON data format for storing and querying documents.

What are the document database options in Azure? ›

Azure provides options for four types of NoSQL databases, including key-value, document, columnar, and graph.

Which three task can be performed by using Azure? ›

Identity Protection allows organizations to accomplish three key tasks:
  • Automate the detection and remediation of identity-based risks.
  • Investigate risks using data in the portal.
  • Export risk detection data to other tools.
Mar 10, 2023

Which Azure service is the best choice to store documentation? ›

There are various Azure Storage services you can use to store data. The most flexible option for storing blobs from many data sources is Blob storage. Blobs are basically files. They store pictures, documents, HTML files, virtual hard disks (VHDs), big data such as logs, database backups—pretty much anything.

Which five service components make up Azure cognitive services? ›

Cognitive Services can be categorized into five main areas:
  • Vision.
  • Speech.
  • Language.
  • Decision.
  • Azure OpenAI Service.
May 12, 2023

How do I create a query in Azure? ›

Run a SELECT query
  1. To query for the top 20 products in the database, paste the following SELECT query into the query editor: SQL Copy. ...
  2. Select Run, and then review the output in the Results pane.
  3. Optionally, you can select Save query to save the query as an . sql file, or select Export data as to export the results as a .
Mar 20, 2023

What are the 4 key steps involved in creating a Azure Cognitive Search Index? ›

Configure a search indexer to extract plain text from Azure blobs for full text search in Azure Cognitive Search. Import and refresh data in a search index using the portal, REST APIs, or an Azure SDK. Populate and upload data to an index in Azure Cognitive Search from external data sources.

How do I make my data searchable with Azure search and AI? ›

Creating a search index with Azure Search and AI
  1. Go to the Azure portal.
  2. Click the Create a resource button (the plus-sign in the top left corner)
  3. Search for Azure Search and click on the result to start creating one a. This brings you to the create blade of Azure Search b.
Jan 11, 2020

What are the two types of bot? ›

What are common types of good bots?
  • Chatbots. Chatbots simulate human conversation with artificial intelligence and machine learning (AI/ML) technologies. ...
  • Web crawlers. Web crawlers, or spiders, are search engine bots that scan and index webpages on the internet. ...
  • Scrapers. ...
  • Shopping bots. ...
  • Monitoring bots. ...
  • Transaction bots.

What is bot framework in Azure? ›

Microsoft Bot Framework and Azure Bot Service are a collection of libraries, tools, and services that let you build, test, deploy, and manage intelligent bots. The Bot Framework includes a modular and extensible SDK for building bots and connecting to AI services.

Is virtual agent same as chatbot? ›

A chatbot simulates human conversation through auditory or textual methods. A virtual agent also known as a virtual assistant, or VA is a program but with similarities to an actual assistant: they can answer specific questions, perform specific tasks, and even make recommendations.

How do I create a bot in Azure? ›

Create an Azure Bot resource
  1. Go to the Azure portal.
  2. In the right pane, select Create a resource.
  3. In the search box enter bot, then press Enter.
  4. Select the Azure Bot card.
  5. Select Create.
  6. Enter the required values.
  7. Select Review + create.
  8. If the validation passes, select Create.
Oct 17, 2022

Is bot framework free? ›

Free plan. One of the two pricing plans for Microsoft's chatbot is the free plan. Within it, you have unlimited messages for standard channels, as well as up to 10 000 messages per month for premium channels.

What are the advantages of Microsoft bot framework? ›

It lets you create new experiences and reach billions of users connected through Skype, Facebook Messenger, Telegram and other channels. Microsoft Cognitive Services includes many intelligent APIs you can add to your bot.

Can a bot open an email? ›

Bot activity in email is typically security software sitting in front of a recipient's (or contact's) inbox. That security software will open the email and, typically, engage with some (if not all) of the corresponding links in that email, following them through the redirect.

How do you know if a bot is messaging you? ›

Look for repetitive patterns

Chatbots typically reply using the same answer pattern every time you ask the same question. Humans are not good at that, even if we try.

How do I create a QnA bot in Azure? ›

Create a bot
  1. In the QnA Maker portal, on the Publish page, select Create bot. ...
  2. A new browser tab opens for the Azure portal, with the Azure Bot Service's creation page. ...
  3. After the bot is created, open the Bot service resource.
  4. Under Bot Management, select Test in Web Chat.
  5. At the chat prompt of Type your message, enter:
Apr 1, 2022

Which API is used to build a chatbot? ›

Chatbot API
Chatbot APIRatingBest for
Wati4.6/5 ⭐️ (150+ reviews)WhatsApp chatbot API
Ada4.6/5 ⭐️ (140+ reviews)Access permissions for user types
ManyChat4.6/5 ⭐️ (110+ reviews)Facebook Messenger chatbot API
ChatBot4.5/5 ⭐️ (15+ reviews)Variety of integrations
9 more rows
May 29, 2023

What is the best API to create a chatbot in? ›

1. Twilio Studio. Twilio Studio lets you build a friendly, conversational chatbot that can respond to incoming messages, chat with customers, and process orders. It's a visual no-code workflow builder that you can use to build chatbots over SMS, webchat, WhatsApp, and Facebook Messenger.

Is there a free AI chatbot? ›

The best overall AI chatbot is the new Bing due to its exceptional performance, versatility, and free availability. It uses OpenAI's cutting-edge GPT-4 language model, making it highly proficient in various language tasks, including writing, summarization, translation, and conversation.

How do I make a chatbot for free? ›

How to create a chatbot in 3 easy steps?
  1. Enter your bot name to get started. Select the type of bot that meets your business needs.
  2. Customize the chatbot the way you want. Make a chatbot in a few minutes without any coding.
  3. Add Chatbot to your website or mobile app. Respond automatically to customers in real-time.
7 days ago

How much does it cost to have an AI chat bot? ›

Custom chatbot development: from $10,000/mo to $500,000/project. Outsourced chatbot development: from $1,000 to 5,000/project and more. Small business chatbot software pricing: from $0 to $500/mo. Enterprise chatbot software pricing: from $1,000 to 10,000/mo and more.


1. Use Your Locally Stored Files To Get Response From GPT - OpenAI | Langchain | Python
(Shweta Lodha)
2. How to Build an AI Document Chatbot in 10 Minutes
(Dave Ebbelaar)
3. Using Azure Search With Azure OpenAI & Langchain (End-to-End Flow Using Non-Semantic Search)
(Shweta Lodha)
4. How To Query OpenAI Embeddings - OpenAI | Langchain | Python | Pinecone
(Shweta Lodha)
5. Chat with Your SQL Data Using ChatGPT
6. How To Get Answers From Your PDF - OpenAI | Langchain | Python
(Shweta Lodha)


Top Articles
Latest Posts
Article information

Author: Geoffrey Lueilwitz

Last Updated: 08/01/2023

Views: 6170

Rating: 5 / 5 (60 voted)

Reviews: 91% of readers found this page helpful

Author information

Name: Geoffrey Lueilwitz

Birthday: 1997-03-23

Address: 74183 Thomas Course, Port Micheal, OK 55446-1529

Phone: +13408645881558

Job: Global Representative

Hobby: Sailing, Vehicle restoration, Rowing, Ghost hunting, Scrapbooking, Rugby, Board sports

Introduction: My name is Geoffrey Lueilwitz, I am a zealous, encouraging, sparkling, enchanting, graceful, faithful, nice person who loves writing and wants to share my knowledge and understanding with you.