Businesses are increasingly relying on AI-powered chatbot solutions to streamline operations and customer service. Enterprise chatbots that can be trained on your company's data provide instant responses to most user queries while seamlessly handing over more complex issues to human agents.
Enterprise Bot is an AI-driven chatbot platform focused on solutions for customer service and support. Proprietary Enterprise Bot DocBrain technology connects your data and knowledge base to LLMs to answer queries and the platform offers extensive customization options and integrations, with support for GPT and a variety of other models.
Copilot Studio is an enterprise chatbot platform from Microsoft that provides tools for creating customized business chatbots that integrate with Microsoft services.
This analysis compares Enterprise Bot and Microsoft Copilot Studio in terms of each platform's setup process, features, and effectiveness in real-world applications and examines their unique characteristics. We’ll also discuss qualities that are important for enterprise chatbots.
We used the BASIC framework to compare the platforms' performance and found that Enterprise Bot consistently provided more accurate responses and stayed within the bounds of its instructions and context better. To see how Enterprise Bot compares to other enterprise AI services, take a look at our post, Enterprise Bot Vs. Copilot Studio Vs. Cognigy Vs. Kore.ai.
To compare Enterprise Bot and Copilot Studio, we set up a basic chatbot using each platform and trained it to answer medical insurance queries. We uploaded a set of three PDFs from AXA Healthcare to each platform that the bots could use as context to answer simulated customer queries about the AXA offerings.
We used a standard prompt and the recommended settings for both chatbots.
We documented our experience with each chatbot service and took note of features such as:
This analysis includes some standard Q&A examples from each chatbot so that you can compare responses and references. For an in-depth look at the Q&A capabilities of various chatbot providers, read our Enterprise Bot vs. competitors article.
A key difference between the two platforms is how they approach AI-chatbot development:
We found setting up a new bot much simpler and more intuitive using Enterprise Bot compared to Copilot Studio, with fewer steps needed to build a Generative AI assistant that can access a specific knowledge base. Copilot Studio required us to create custom flows to configure bots to our data.
The table below gives an overview of the steps needed to create a new bot on each platform, showing that the initial setup for Enterprise Bot is shorter and more straightforward.
|
Enterprise Bot |
Microsoft Copilot |
|---|---|
|
|
Copilot Studio lacks the option to change LLM as only GPT-3.5 is supported.
When selecting "Create bot" on Enterprise Bot, you are presented with a set of simple steps to create and configure your bot, including adding documents to be used as the bot’s knowledge base and choosing your language model.
When setting up a chatbot with Copilot Studio, you give it a name and a prompt. Note that you cannot configure which LLM to use (Copilot Studio only offers GPT 3.5), and you need to configure the knowledge base separately. Further, although there’s a “language” dropdown, only English is currently supported for generative AI chatbots.
Both platforms support extracting knowledge from various sources. In our case, we were mainly interested in PDF files as a knowledge source.
Enterprise Bot offers the following sources as options when adding new knowledge to a bot.
Copilot offers:
To effectively use custom data, each platform ingests uploaded files into a RAG system, dividing them into chunks and vectorizing them. Enterprise Bot uses its own embedding system that runs locally, separate from any LLM models. This ensures that only limited data (the user’s query and relevant chunks of text from the knowledge base) are sent to external LLM providers.
On Copilot Studio, vectorization is done with OpenAI embeddings, so the full text of all documents is processed by the LLM provider.
Once the processing is complete, Enterprise Bot allows you to inspect how the data was processed into chunks.
Enterprise Bot uses its patent-pending DocBrain system to break documents into chunks and assign each chunk a type. It also adds metadata to each chunk, specifying which page of the document it can be found on. You can browse each chunk and see its associated text and metadata through the Enterprise Bot UI.
You can view each chunk in the Enterprise Bot data management console
Copilot lacks the ability to view chunks of your data, which makes it hard to verify that the data has been correctly parsed.
There’s no option to see chunked data in Copilot Studio
Enterprise Bot supports a range of models from OpenAI, Anthropic, and Google, while Copilot Studio only supports OpenAI GPT-3.5 Turbo. Enterprise Bot also supports on-the-fly LLM switching, so you can test different models on the same assistant, and switch to the model that performs best. Enterprise Bot also regularly adds new models, so you can upgrade your existing bots with no extra effort.
By contrast, Copilot Studio only offers GPT 3.5.
|
Enterprise Bot |
Microsoft Copilot |
|
Azure OpenAI or OpenAI
GPT-3.5, GPT-4, GPT-4o
Google
Gemini Family
Amazon Bedrock
AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, Amazon
|
AzureOpenAI
Only GPT-3.5
|
Above we walked through the three main stages of creating a new generative AI assistant, and we noted that Enterprise Bot allows you to create a chatbot in fewer steps while offering more features.
We also found that for those steps, Enterprise Bot offered an intuitive bot-building experience we could follow without needing additional documentation. By contrast, Copilot Studio required us to navigate through more complicated menus, and often settings were not where we expected them to be.
We also encountered several unexpected error messages while setting up a bot on Copilot Studio. Such errors were erratic and results were sometimes different a few hours later, even though nothing else had changed.
To configure our assistant on Copilot Studio, we needed to find the “Generative answers” option under an “Advanced” menu. We then had to build out a flow, specifying how a conversation should start, setting up a UserQuery variable, and mapping that to the “Create generative answers” node.
When trying to edit the settings, we received the error, "This feature isn't available until your bot has finished setting up". As we had already been chatting with the Copilot bot before changing the configuration, it wasn’t clear what part of the setup process had not yet been completed. For a few hours following the error, we were unable to update or delete any bots, and the issue seems to still affect users periodically.
The Enterprise Bot setup process was easier and more intuitive than our experience with Copilot Studio. We gave the bot a name, chose an LLM provider, and uploaded the documents. During setup, Enterprise Bot gave us the option to customize prompts globally, a convenient feature that Copilot lacks.
After the assistant is set up, the chat experience is fairly similar on both platforms. Users can enter queries into a chat window and get responses, with the assistants taking into account previous messages and relevant content from the uploaded documents.
A significant difference between the platforms is in how they provide references to their internal knowledge. Providing references helps a user verify the information and also easily find more information and context about a specific topic.
In our evaluation, both the assistants from Enterprise Bot and Copilot Studio provided relevant references, however, we found that Copilot only gives chunks of text while Enterprise Bot gives links to the documents referenced, as well as the page number of the source text.
Enterprise Bot included references at the end of every response
Microsoft Copilot also gave references…
…but no links to uploaded documents, only chunks of text
The references provided by Enterprise Bot were more useful and better presented than the wall-of-text effect from Copilot Studio that end users would likely struggle to comprehend.
Most of the AI features in Copilot Studio are labeled as "preview" while the Enterprise Bot equivalents are already actively used in production chatbots. Specifically:
You can customize the look of the chat UI with both Enterprise Bot and Copilot Studio, but Enterprise Bot offers more extensive customization options that make it easy for even non-technical team members to configure.
In Enterprise Bot, you can effortlessly customize header colors and message backgrounds for both the human and bot participants. More advanced customization options are available for Enterprise Bot chatbots using Botforge.
Customizing the UI in Enterprise Bot
Copilot Studio does not offer a UI to customize the look and feel of assistants, and instead requires custom CSS and JavaScript configuration, making it harder for non-technical team members to implement.
For an in-depth look at the Q&A capabilities of different chatbot providers, read our Enterprise Bot vs. competitors article.
To evaluate the assistants, we looked at how accurately they could answer questions that a typical user might ask, as well as how well the chatbot stayed on topic, how fast it produced responses, and how concise those answers were. You can read more about how and why we chose these metrics in our post, Back to BASICs: A Generative AI benchmark for Enterprise.
We also tested the assistants’ multilingual capabilities in this evaluation. We uploaded a document in German to the knowledge base of each platform and added some questions in German to the evaluation set.
The table below shows some of the questions we asked both chatbots to test their ability to answer queries reliably. We’ve also included an expected answer for each question to show how a human would respond, and a comments column, summarizing what was wrong with specific answers. The full set of questions we used is available here.
As Copilot Studio only supports GPT-3.5 (while Enterprise Bot supports several different models and providers), we used GPT-3.5 for both assistants.
ments
You can see some more example questions and answers here.
The Enterprise Bot assistant clearly outperformed Copilot Studio in most of our tests. The Enterprise Bot assistant always stuck to the relevant context, while the Copilot Studio assistant often got confused and hallucinated, talking about shipping parcels or other unrelated topics. Enterprise Bot was also faster, with an average of 2.7 seconds per response compared to 3.3 seconds from Copilot Studio (a more than 20% speed difference).
Copilot hallucinated being a shipping assistance bot, despite being set up as a customer service policy bot
While Copilot Studio provided more concise answers, these were often verbatim quotations from the knowledge base documents, while Enterprise Bot offered answers that were put into context and more accurate than the answers from Copilot Studio.
Summary of results for Enterprise Bot and Copilot Studio on the BASIC benchmark
One of the most notable differences between the assistants was that we occasionally saw the Copilot Studio assistant switch languages. Specifically, when faced with our "trick" questions where the user asks the assistant for help with something inappropriate, the Copilot Studio assistant would sometimes respond in Spanish, even though none of the test messages or uploaded data used Spanish at all.
Curiously, when answering inappropriate questions, Copilot tended to reply in Spanish rather than English
Copilot Studio does not officially offer multilingual support for generative AI assistants. However, because the GPT 3.5 LLM it relies on is multilingual, assistants will generally reply in the language of the query, especially if prompted to do so.
Enterprise Bot officially supports multilingual assistants in over 50 languages.
We tested both chatbots on their multilingual capabilities by giving each a German AXA brochure for vehicle insurance to see how they handle queries in German. The table below summarizes the bots' performance and shows the German questions we tested on each assistant. For the evaluation, we used only the German versions of each question. The English translation is given here for convenience.
You can see some more example questions and answers here.
Microsoft Copilot often provided shorter answers with less information compared to Enterprise Bot. This made it seem as if Copilot was reading the documents verbatim without giving comprehensive answers. This tendency to over-summarize resulted in Copilot failing to reference key services and details present in the documents.
Copilot also struggled with correctly chunking non-English paragraphs, leading to incomplete or inaccurate answers.
Conversely, Enterprise Bot excelled in these areas, successfully referencing multiple points and providing detailed and accurate answers. Enterprise Bot also provided citations for non-English queries, while Copilot failed to provide valid citations at all in some cases.
Overall, Enterprise Bot outperformed Microsoft Copilot in our multilingual accuracy metrics, scoring 83% compared to Copilot's score of 66%. Similarly to our English evaluation, Enterprise Bot provided much more comprehensive answers, but as these were more accurate and useful, this is not a negative point.
Multilingual accuracy metrics
Here’s a summary of the features offered by each platform, based on our tests and each platform's documentation.
|
Feature |
Enterprise Bot |
Copilot Studio |
|---|---|---|
|
Services offered |
Text · Voice · Customer support assistance |
Text · Voice |
|
Maturity |
Production solutions since 2017 |
|
|
Models supported |
Multiple models via Open AI, Azure OpenAI, Google, and Amazon Bedrock. |
Only GPT-3.5, via Azure OpenAI |
|
Languages |
50+, including Arabic · Cantonese · English · Dutch · French · German · Italian · Malay · Mandarin · Portuguese · Spanish · Swedish |
English |
|
Document vectorization |
DocBrain |
Standard GPT embedding using OpenAI |
|
Document upload security |
All knowledge indexing done locally using DocBrain |
All data indexed by OpenAI embedding models. |
|
Document upload formats |
PDF · URL · sitemap · Confluence · Topdesk · SharePoint · other enterprise integrations (using Blitzico) |
PDF · URL (only indexed by Bing) · sitemap · SharePoint · other enterprise integrations (preview) |
|
Document size limit |
20MB |
3MB |
|
Chatbot references training material |
Gives references to source docs, with links and page numbers, also hosts a copy of your docs |
Text chunk reference only |
|
Can switch LLM |
Yes |
No |
|
Setup simplicity |
Straightforward and easy to follow |
Somewhat complicated, requires creating flows and setting up nodes |
|
Bot can fallback to default answer |
Yes, default |
Requires manual configuration |
|
Bot can handover conversation to human agent if necessary |
Yes |
No |
|
Translation support |
Yes |
No |
|
Connect bot to your site |
Yes |
Yes |
|
Train bot conversations manually |
Yes |
No |
|
Dashboard & analytics |
Yes |
Yes |
|
Chat UI customization |
Yes, with advanced customization using BotForge |
Limited, HTML canvas on site only |
While Microsoft has clearly put significant effort into building Copilot Studio, there is more work to do before the platform is suitable for production use, with many of the core features still in preview. Copilot Studio is also missing important features such as multilingual support and LLM switching.
Enterprise Bot offers a more mature, sophisticated platform that is intuitive and easy to use.
If you’re looking to build Generative AI assistants that your customers will love, reach out to us today.