ChatGPT got quite some buzz. It has been discussed in every news show, and all our (grand)parents heard about it, or even tried it. Copilot is coming to all Microsoft products too. For this challenge I want step away from the buzz and show how it might bring true value to any organization.
Challenge Objectives
🎯 Utilize open-source assets
🎯 Play around with Power Virtual Agents
🎯 Add functionality to a chatbot
🎯 Understand API documentation
Introduction
Every month, week, or even day, Microsoft is releasing a ton of new features to their range of products. You, as an enthusiast are doing your best to keep track of all these changes, which can be truly challenging. I totally agree on that. Imagine what those that don't have some sort of Microsoft crush would feel like.
This is where adoption comes in. Some organizations are fortunate to have dedicated Adoption Consultants, some don't. For both, a chatbot could take a piece of this work. Now that the OpenAI models are available to us in the Microsoft ecosystem, we can finally make chatbots that does it all. Answer very enterprise specific questions, and give answers to unforeseen questions. In this challenge, you will use an open-source adoption bot, and enrich its capabilities with OpenAI's GPT Large Language Model (LLMs).
Install the Adoption Bot
There is a great pre-made chatbot available, that answers over 100 common questions about Microsoft Teams and Microsoft 365. A great starting point for this challenge. Instructions on how to download and install can be found on GitHub. Please follow those instructions, including the tips given below.
Use the tenant from the Microsoft Developer Program
Create a new team in Microsoft Teams called Challenge 018
Add two channels, Ask the Expert, and Feedback
While adjusting the team and channels in the flows, you might need to select the team and channel twice to see it in the action dropdown
Add the Adoption Bot Admin App to the General channel
After you have followed the instructions, you should be able to use the chatbot. Give it a try. Take all perspectives into account. Those who have a question, the ones who are deemed the expert, and maybe a manager who is mostly interested in the success rate and the CSAT (Customer Satisfaction) score.
Do you notice that some questions give quite an OK answer, but at some points, it is just not really great? That is my personal generic experience with these type of chatbots so far to be fair. Of course, we can use Power Virtual Agents monitoring functionalities to see what questions are being asked that have not been answered by the chatbot to add or adjust topics.
This is a reactive way of doing it. Each user with unprogrammed questions will leave unsatisfied. That's why we will try to add the magic of ChatGPT to drastically reduce the unanswered rate. The questions answered by OpenAI can be monitored and where required, added as dedicated topic. In some cases, a generic answer bij OpenAI might just be fine.
Extend the chatbot with OpenAI
As you might have notices, if the chatbot doesn't understand your question, it will ask to rephrase, or to forward it to an expert. We will add a third option, which we will call Ask ChatGPT. For this, we need to add the option, a topic, and a flow that will delegate answering the question to OpenAI.
OpenAI account
Because of Troy Taylor, who created an Independent Publisher Connector for OpenAI, we can quite simply add GPT capabilities to our flows. According to his documentation, we need an API key for authentication. To get one, you will need an OpenAI account. Every new user get's 3 months of trial for free. This includes using their APIs, which is perfect for our challenge. You can register here.
Once you are registered, you can click on your profile to view your API keys. That's where you can also create new ones. I created one called Challenge018. The key itself is only shown once, so you should store that somewhere safe.
Power Automate Flow
Now that we have an account, let's see if we can get some output from the connector.
Open Power Apps in Teams
Go to the Build tab
Select team Challenge 018
Select See all
Add a new instant cloud flow
Name it Adoption Bot - Ask ChatGPT
Select Manually trigger a flow as the trigger
Select Create
Search for OpenAI GPT
Select the Get chat completion action
Before we can start using the connector, we have to create a connection. For the name I used OpenAI_Challenge018. For the API key, you need to type Bearer, a space, and the key you copied. Luckily this is properly documented.
When the connection is created, we can start giving some input to this action. the parameters are listed in the connector documentation, but we don't know which field are mandatory, and which not, what can be given as an input, etc. Based on the parameters in the connector, we can see that it is based on this OpenAI API. Here we can see that the role parameter has the options system, user, and assistant. The content parameter is where the question is given in a text string.
For testing purposes, we can enter user for the role, and as a question we will ask What is the most famous song from Stromae? It might say you need a flow license. You can start or extend your trial if you are using a developer tenant.
As you can see, it is working. Not my favorite song, but I guess it is his most famous one. Now that we know it is working, we should pass the question from the bot user to it.
Before we leave the flow, replace the manual trigger with the trigger When Power Virtual Agents calls a flow, and add the action Return value(s) to Power Virtual Agents. For the input, create a property called Question, and adjust the description to Please enter the question from the user.
Update the content to the Question input parameter you've just created.
For the output, we can already add a parameter called Answer, and enter the First Content from the dynamic content to it's value. As you can see from the image above, this is the property that contains the actual answer.
Add a Topic
As mentioned, we want to add a button, when the answer isn't correct. Below you can see the current options.
The image above is the output from the Confirmed Failure topic. If you open the solution you've imported earlier, you can see the Adoption Bot listed. If you click it, Power Virtual Agents will launch within Teams. Navigate to Topics and search for Confirmed Failure. It is almost at the bottom. The topic is shown below.
You can see three options. This functionality is similar to what you know as a switch within Power Automate. You can see that the Ask an agent will redirect the user to a different topic. If a question is rephrased, Power Virtual Agent will search through the topics to see which fits best, and feed that to the end user.
The option we want, Ask ChatGPT, is a predefined route, just like the Ask an agent option. As you can see, this option is redirected to the Escalate topic. In order to redirect, the redirection topic must exist. So that is what we are going to do. We will come back to this topic later to actually add it.
Close the topic and create a new one. Under details, you can rename it to Ask ChatGPT. I will start with the end result, and guide you through it. Excuse me for the horrible readability.
We start with the Trigger Phrases. This is used to determine which topic to pic. By adding these trigger phrases, we can directly go to ChatGPT. If you want to guide them through your programmed topics first. you can leave the trigger phrases blank.
Then we go to the second box in this topic. Unlike ChatGPT, Power Virtual Agents is not aware of the full context of your conversation. This is why I am asking again for what the user wants to ask ChatGPT. This means the end-user might need to type their question twice, which is far from ideal. We could change the whole bot by asking a question, saving that as a bot variable, and use it afterward, but that would mean we have to redo the whole chatbot we installed. So for the sake of speed, we just usk the user what they want to ask ChatGPT. This is saved as a variable, which we can use in next steps. By default, the variable is named var. Please rename it to Question.
The third box is just a message to the bot user, to inform them the question is forwarded to ChatGPT. This is because it might take a few seconds to actually retrieve an answer for ChatGPT. For user experience purposes, we let them know it has been forwarded, just so they have a little more patience.
The fourth block is where we actually link Power Virtual Agents to Power Automate. The flow we created earlier should be listed among the options. Just select it. Because we created an input variable, we need to specify it here. You can see I feed the topic variable Question from the second block here. Note that this action creates a topic variable called Answer. This is the output parameter we created in Power Automate.
the fifth block is to let the user know we got an answer. This is just because jotting the answer itself might be a bit confusing for the end-user.
The sixth block is the answer itself. Note that bind the topic variable Answer here. This is similar to Dynamic content in Power Automate. You can find the options by using the drop-down all the way to the right.
The last block is the system End of Conversation redirection. You can see it for yourself what it does. I added it so that in case this still isn't the right answer, users can let us know.
NOTE If you encounter sone issues with saving your topic, try to start a trial vor Power Virtual Agents. This will do the trick.
Add the option Ask ChatGPT
Now that the Ask ChatGPT topic is finished , we only need to add it as an option to the Comfirmed Failure topic, we've seen earlier. This is an easy last step. Open the Comfirmed Failure topic and add Ask ChatGPT as an Option for user. You will see a new branch being added. You only need to redirect it to the topic Ask ChatGPT we've created earlier.
It should look like the image above. Now you only need to publish your bot, and give it a spin. It should handle your questions perfectly.
OpenAI Usage
After you have used some OpenAI functionality, you might want to have a look at the Usage page. You get a few month to play around with a few credits. If you want to make this into an enterprise solution, please read the next section.
Additional Information
Azure OpenAI
In this challenge, we directly used the Large Language Model (LLM) from OpenAI. As you've probably heard, Microsoft is heavily invested in OpenAI. The same LLMs are available through Azure, called Azure OpenAI. Access is limited at the moment (through an application form), which is why I opted to use the LLM from OpenAI itself for demonstration purposes. It won't take long before a broader audience will be able to build solutions directly on Azure using these LLMs. For Enterprise bots, it is highly recommended to use the Azure OpenAI services.
AI Builder
Adding Azure OpenAI functionality to Power Platform solutions will become much easier in the near future. According to this Microsoft article, it will become part of AI Builder, which acts just as a regular connector. At the moment of writing, it is in gated preview. The same applies to Power Apps.
Prompt Engineering
If you are interested in using LLMs like ChatGPT, it is highly recommended to improve the way you write your promts. This is called Prompt Engineering. I recently followed this course, which is in partnership with OpenAI. Microsoft also has a learning path on Azure OpenAI, where the last module focuses on Promt Engineering.
Boosted Conversation
A very exciting feature for PVA is called Boosted Conversation. If you already have a website, FAQ, or some information that you wish to route through a chatbot, this is your feature. You only need to feed the URL, and that's basically it. It will make all the information from that website accessible through the chatbot. It already allows multiple URL entries. At the time of writing it was only working for public sites, although internal sites (e.g. SharePoint) is on the roadmap. It might be already available when you read this.
Key Takeaways
👉🏻 Power Virtual Agents is great for Enterprise specific information
👉🏻 ChatGPT is great for generic information
👉🏻 We can combine these forces, and soon even more easily
👉🏻 For enterprise solutions, Azure OpenAI, or even AI Builder is the way to go
👉🏻 I will study Prompt Engineering