With Azure OpenAI we can deploy a ChatGPT model and use it through the same ‘API’ definition as OpenAI has with ChatGPT API. Here i look at some of the differences.
Model versions
At the time of this article (nov 23) Azure OpenAI support ‘GPT 3.5-Turbo’ only. Another interesting limitation is that several regions only support the older ‘0301’ model version and not ‘0613’. The later one is required to do ‘function calls’.
Azure OpenAI vs. OpenAI ChatGPT API
One great feature in Azure OpenAI is how easy it is to add your ‘own data’ using the concept of data sources. The two main ones are ‘Azure Cognitive Search’ and ‘Azure CosmosDB’.
As an example we can use an Azure Search instance to index an intranet, public website or a collection of documents. Then we can include that search index in the Azure OpenAI ‘completions’ API call.
Lets look at an example
{
"temperature": 0,
"max_tokens": 1000,
"top_p": 1.0,
"dataSources": [
{
"type": "AzureCognitiveSearch",
"parameters": {
"endpoint": "<YOUR-AZURE-SEARCH-ENPOINT>",
"key": "<YOUR-AZURE-COGNITIVE-SEARCH-KEY>",
"indexName": "<YOUR-AZURE-COGNITIVE-SEARCH-INDEX-NAME>"
}
}
],
"messages": [
{
"role": "user",
"content": "Can you find the corporate definition of XYZ?"
}
]
}
If we use the excellent Azure.AI.OpenAI SDK we can use it like this
AzureCognitiveSearchChatExtensionConfiguration someChatExtension = new()
{
SearchEndpoint = new Uri("https://your-search-service.search.windows.net"),
IndexName = "someindex01",
};
// There does not seem to be support for Managed Identities as of nov 2023
someChatExtension.SetSearchKey(Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SEACH_API_KEY"));
var chatCompletionsOptions = new ChatCompletionsOptions()
{
DeploymentName = "our-deployed-gpt-35-turbo-model-01",
Messages =
{
new ChatMessage(ChatRole.System, "You are a rude AI assistant that helps people find information from our intranet"),
new ChatMessage(ChatRole.User, "What is the corporate definition of XYZ?"),
},
AzureExtensionsOptions = new ()
{
Extensions = { someChatExtension }
}
};
I think this shows a really simple and powerful way to combine the power of a LLM like ChatGPT with your ‘own data’.