Integrating watsonx Assistant with watsonx.ai foundation models

Obtain the OpenAPI definition of the watsonx.ai generation endpoint

Before you can add the integration for watsonx.ai to a watsonx Assistant virtual assistant, you need to download the OpenAPI definition for the watsonx.ai foundation model inferencing service. An example definition file is available in the assistant toolkit GitHub repository. For this lab, you will use a version modified with the addition of support for the decode_method parameter of the api. Download this version of the watsonx-openapi.json file to your workstation.

Add an empty virtual assistant to the watsonx Assistant instance

For initial experimentation with the integration, use a new virtual assistant. If you have a newly created watsonx Assistant service instance created, it will not have any assistants defined. Create one using the Create Assistant wizard using a name of Watsonx integration and then selecting web as the deployment location. The values you provide in the other parts of the wizard will not matter very much. For a slightly opinionated step-by-step example of how to get a web assistant created, see the VEST watsonx Assistant 101 lab up to the point where you see the home page for the assistant. If you have other assistants defined already in your instance, add one following the documentation

Creating a watsonx.ai API key

Depending on the environment you are using or if you are working within a workshop setting an API key may already be
provided for you to use; ask your instructor or review your environment information to determine if an API key is provided.

Otherwise feel free to follow the directions below to create an API key within your own IBM Cloud account.

If you have previously created an API key for calling watsonx.ai from a notebook or just the REST endpoint with curl you may skip these steps and reuse that API key. Otherwise, continue on.

1. Log in to watsonx.ai and make sure that the same account as the watson machine learning instance for your watsonx.ai project is selected.

2. To generate an API key from your IBM Cloud user account, go to Manage access and users - API Keys and click on Create. In the pop-up, provide a name like watsonx-apikey and a short description, then click Create

   

3. Immediately make a copy of the API key and store it in a secure vault (like 1Password, etc.) as there will be no way to retrieve it again.

Getting your watsonx.ai project id

1. In the watsonx.ai page, select the left 4-bar pull down menu, expand Projects and then click on View all projects.

2. From the list, select the project that your are using for the integration.

3. Click on the Manage tab and check that General is selected. Copy the Project ID to a location where you can access it later. This is not sensitive like the API key so it can be in notepad or something equivalent.

   

Adding the custom extension for watsonx.ai to the Integrations catalog

1. Return to your browser tab with the assistant you created. In the assistant left navigation panel, go down to the lower section and click on Integrations

2. Scroll down to the Extensions section and click on the Build custom extension button:

   

3. Review the getting started guide and click on Next

4. On the basic information tab, name the extension watsonx and provide an optional description. Click Next

5. Either drag and drop or use the file browser to select the watsonx-openapi.json file and then click Next

   

6. Review the information displayed and click on Finish, now the integration catalog shows the integration

   

7. Click on Add in the lower right corner of the new integration tile, confirm Add when prompted

8. Review the Get started information and click on Next

9. In the Authentication panel, select Oauth 2.0, enter in the API key that you created earlier, keep the remaining default values and then click Next

   

10. On the last page review the POST request parameters and response values. The assistant can set parameters in the request before calling the extension and will parse the response to show the user the output. Click on Finish, then click on Close if you are not returned to the Extensions section of the Integrations panel.

Congratulations, you now have a way to make a functional integration from an Action in assistant with watsonx generative AI endpoint.

Create your action

Now it is time to create your action and have it use the integration to watsonx. To get started, you will create an action that uses generative ai to draft a marketing message from your prompt.

1. In the browser tab with watsonx Assistant and click on Actions in the upper left.

2. In the main panel, click on Create action

3. Select Start from Scratch

4. For what the customer says enter Marketing Message and click on Save

   

5. Click on the pencil icon to set a step title. Use Prompt for a prompt as the title.

6. For what the assistant says, you can either make it a short message or be more prescriptive. Note - humans, like generative AI models can often do better when requests are specific. For example:

   Please provide a prompt for the model to use to create the marketing message. Type as much as you like and include lists of details for me to use with the model.
   text
   Copy code to clipboard

   

   Expand Define cutsomer response and specify that the assistant will expect a response from the user as Free Text input. When you click on Free Text, the UI will show User enters free text below the Assistant says panel. When this is shown, click on New Step to configure the next part of the Action.

7. Name the next step Call watsonx.ai endpoint by clicking on the pencil (edit) icon. In the And then section, select Use an extension

   

8. For the extension to use, select watsonx. Then select the Generation operation to show the parameters that can be sent to watsonx.ai on every invocation. For version, choose Enter text and then provide 2023-05-29, For input, choose Action Step Variables and then choose 1. Prompt for a prompt, continuing in the same way, set the model_id to google/flan-t5-xxl and the project_id to your watsonx.ai project id saved before. Don't apply yet

   

9. Expand the optional parameters. For this text generation scenario, use temperature 0.7, set max_net tokens to 200, set min_new_tokens to 50 and set repetition penalty to 2, set decoding_method to sample. Then click on Apply

   

10. Click on New Step

11. In this step, select with conditions . In the condition, click on the first item and select watsonx (step 2)

12. Select Ran successfully

13. Click on Set variable values , then click on Set new value, then click on New session variable

14. Call the variable result and select free text and then click on Apply

15. For the variable assignment To, select Expression and type $ after this is pasted a pulldown will appear. Select watsonx (step 2)

Select body.results, the box will fill in with the assignment for the result variable. Now add the text [0].generated_text to the end of the expression:

click Apply

16. The full expression for the variable should be shown as:

17. In the Assistant says panel, select function and then type in ${result}, or you can select Session variables and then result

The assistant should be ready, time to test it out.

Testing the action

1. Click on the Preview button.

2. After the greeting, type I need to create a marketing message, and send to the chatbot.

3. The chatbot will reply with the prompt for the prompt.

4. Enter the following text into the chatbot

   Generate a 5 sentence marketing message for a company with the given characteristics.
   
   Details
   Characteristics:
   
   Company - Golden Bank
   
   Offer includes - no fees, 2% interest rate, no minimum balance
   
   Tone - informative
   
   Response requested - click the link
   
   End date - July 15
   
   Email
   text
   Copy code to clipboard

5. Send to the chatbot - if all worked, you will get a response back from the model with the message.

   

   You will most likely see something different because we are using a sample (vs. greedy) decoding approach. Don't like the response, re-run the assistant to get another message. Note - with these parameters, the assistant may create information like years for the date (because only July 15th is specified) or phone numbers. In a real use case, a human would use this generated message as a starting point, making edits as necessary and then approving the final version.