Securing AI Apps on Azure: Using Keyless Auth with Azure AI Services

From live stream:

Ready to go keyless and never worry about compromised keys again? All the Azure AI services support keyless authentication using role-based access control, making it possible for you to authenticate to the services with either your logged in local user identity or your deployed app's managed identity. We'll show you how to use keyless authentication with Azure OpenAI, demonstrating how to set up the access controls in the Portal, with the Azure CLI, or with infrastructure-as-code (Bicep). Then we'll connect to that Azure OpenAI service in our application code, using both the OpenAI SDK and the popular Langchain SDK. Our examples will be in Python, but you can use keyless auth with most modern OpenAI packages.

Transcript

1. Securing AI Apps on Azure: Using Keyless Auth with Azure AI Services Pamela Fox Python Cloud Advocacy @pamelafox Marlene Mhangami Python Cloud Advocacy @marlene_zw
2. Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355
3. Key-based authentication
4. Key-based authentication for OpenAI All services support key based authentication. The OpenAI SDK defaults to key-based, since that's all that OpenAI.com supports: openai_client = openai.OpenAI( api_key=os.getenv("AZURE_OPENAI_KEY") ) azure_client = openai.AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_KEY") )
5. Key-based authentication for Azure services The Azure Python SDK also allows key-based credentials, using the KeyCredential class. from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient search_client = SearchClient( endpoint=os.environ["SEARCH_ENDPOINT"], index_name=os.environ["SEARCH_INDEX"], credential=AzureKeyCredential(os.environ["SEARCH_KEY"]) ) Example with AI Search client:
6. The risks of API keys • API keys can be easily leaked • API keys can be passed around a company (unintentionally) • API keys can be painful to rotate
7. Keyless authentication for local development
8. Keyless auth is based on OAuth2 Instead of a key, an OAuth2 token proves we can use a resource.
9. Steps for keyless authentication during local dev 1. Create the Azure service 2. Give your user permissions to use that service 3. Log in to Azure locally 4. Use the Azure Identity SDK to generate a token/provider and pass that along to the service SDK You Azure service Azure SDK
10. Keyless auth with Azure OpenAI You Azure OpenAI OpenAI SDK Example project: aka.ms/azai/keyless 1. Create the Azure OpenAI service 2. Give your user permissions to use that OpenAI service 3. Log in to Azure locally with the Azure Developer CLI 4. Use the Azure Identity SDK to generate a token provider and pass that along to the OpenAI SDK
11. Give permissions to Azure OpenAI in Azure Portal 1 2 3
12. Give permissions to Azure OpenAI with Azure CLI az role assignment create \ --role "5e0bd9bd-7b93-4f28-af87-19fc36ad61bd" \ --assignee-object-id "$PRINCIPAL_ID" \ --assignee-principal-type User \ --scope /subscriptions/"$SUBSCRIPTION_ID"/resourceGroups/"$RESOURCE_GROUP" az ad signed-in-user show --query id -o tsv Get your principal ID: Assign "Cognitive Services OpenAI User" to role for target resource group:
13. Give permissions to Azure OpenAI with Bicep module openAiRoleUser 'core/security/role.bicep' = { scope: openAiResourceGroup name: 'openai-role-user' params: { principalId: principalId roleDefinitionId: '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' principalType: 'User' } } If you're using the Azure Developer CLI, add this to main.parameters.json: "principalId": { "value": "${AZURE_PRINCIPAL_ID}" }
14. Use Azure Identity with the OpenAI SDK • Use the azure-identity SDK to get a token provider for your identity • Pass the token provider to the OpenAI SDK • Token providers take care of token refresh for you azure_credential = DefaultAzureCredential() token_provider = get_bearer_token_provider(azure_credential, "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider )
15. Keyless authentication in hosted Azure platforms
16. Keyless auth with Azure OpenAI in Azure Container Apps App Identity Azure OpenAI OpenAI SDK Example project: aka.ms/keyless-azure-containerapps 1. Create the Azure OpenAI service 2. Create the Azure Container App 3. Create an identity for the App to use 4. Give your App identity permissions to use the OpenAI service 5. Use the Azure Identity SDK (specifying app identity ID if needed) to generate a token provider and pass that along to the OpenAI SDK
17. App identity options: System vs. User-assigned Azure Container App System identity Azure OpenAI Azure Container App User-assigned identity Azure OpenAI OPTION 1 OPTION 2
18. Assign an identity to Container App in Azure Portal System identity User-assigned identity
19. Assign an identity to Container App with Bicep resource appIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: '${prefix}-id-aca' location: location } resource app 'Microsoft.App/containerApps@2022-03-01' = { name: name location: location identity: { type: 'UserAssigned' userAssignedIdentities: { '${appIdentity.id}': {} } } ... }
20. Give permissions to OpenAI with Bicep roleDefinitionId = '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' resource role 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(subscription().id, resourceGroup().id, principalId, roleDefinitionId) properties: { principalId: appIdentity.properties.principalId principalType: 'ServicePrincipal' roleDefinitionId: resourceId( 'Microsoft.Authorization/roleDefinitions', roleDefinitionId) } } Assign "Cognitive Services OpenAI User" role to the app identity:
21. Use Azure Identity with the OpenAI SDK azure_credential = DefaultAzureCredential( managed_identity_client_id=os.getenv("APP_IDENTITY_ID")) token_provider = get_bearer_token_provider(azure_credential "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider ) If using a user-assigned identity, you either need to set AZURE_CLIENT_ID in environment or pass in the ID to DefaultAzureCredential:
22. Keyless authentication with Langchain
23. Using Langchain with keyless authentication from langchain_openai import AzureChatOpenAI token_provider = azure.identity.get_bearer_token_provider( azure.identity.DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default" ) llm = AzureChatOpenAI( azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_deployment=os.getenv("AZURE_OPENAI_DEPLOYMENT"), openai_api_version=os.getenv("AZURE_OPENAI_VERSION"), azure_ad_token_provider=token_provider, ) Example project: aka.ms/keyless-azure-langchain
24. Next steps
25. Start using keyless auth with Azure OpenAI! Sample Azure platform Language aka.ms/keyless-azure-containerapps Container Apps Python aka.ms/keyless-azure-langchain TBD Python aka.ms/ragchat App Service Python aka.ms/azai/js/code Container Apps JavaScript/TypeScript aka.ms/azai/net/code Container Apps C# https://aka.ms/azure-openai-keyless-guide Learn more (in your favorite language): Start from a working template:
26. Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355