Run the RFP Evaluator on Azure OpenAI (Part 3)

By MUASIF80·June 7, 2026·4 min read·AI Use Cases

In Parts 1–2 we designed and built an LLM-agnostic RFP proposal evaluator that runs on free local Ollama. Now we make it enterprise-grade: in this part we switch LLM_PROVIDER=azure and wire up Azure OpenAI end to end — resource creation, model deployments, authentication, and the Azure-specific errors you will inevitably hit. The graph you built in Part 2 does not change at all; only configuration does. That is the payoff of the agnostic design.

Why Azure OpenAI

Azure OpenAI is the natural choice for organizations already on Azure that need enterprise terms, regional data residency, private networking, and Entra ID (Azure AD) authentication. For evaluating confidential vendor bids, those guarantees matter.

Step 1 — Create the Azure OpenAI resource

az group create --name rg-rfpeval --location eastus

az cognitiveservices account create \
  --name rfpeval-aoai \
  --resource-group rg-rfpeval \
  --location eastus \
  --kind OpenAI \
  --sku S0 \
  --yes

Note that Azure OpenAI capacity and model availability vary by region — if a model is unavailable in your region, pick a supported one (e.g. eastus, swedencentral).

Step 2 — Deploy a chat model and an embeddings model

In Azure, you call deployments, not model names directly. Create one for chat and one for embeddings:

az cognitiveservices account deployment create \
  --name rfpeval-aoai --resource-group rg-rfpeval \
  --deployment-name gpt-4o \
  --model-name gpt-4o --model-version "2024-11-20" \
  --model-format OpenAI --sku-capacity 10 --sku-name Standard

az cognitiveservices account deployment create \
  --name rfpeval-aoai --resource-group rg-rfpeval \
  --deployment-name text-embedding-3-large \
  --model-name text-embedding-3-large --model-version "1" \
  --model-format OpenAI --sku-capacity 10 --sku-name Standard

The deployment name (here gpt-4o) is what your app references — it does not have to equal the model name, and getting this wrong is the #1 source of 404s.

Step 3 — Get the endpoint and key

az cognitiveservices account show \
  --name rfpeval-aoai --resource-group rg-rfpeval \
  --query properties.endpoint -o tsv

az cognitiveservices account keys list \
  --name rfpeval-aoai --resource-group rg-rfpeval \
  --query key1 -o tsv

Step 4 — Configure .env

LLM_PROVIDER=azure
AZURE_OPENAI_ENDPOINT=https://rfpeval-aoai.openai.azure.com/
AZURE_OPENAI_API_KEY=<key1>
AZURE_OPENAI_API_VERSION=2025-04-01-preview
AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-4o
AZURE_OPENAI_EMBEDDING_DEPLOYMENT=text-embedding-3-large

That is the entire change. Recall the factory from Part 2 — when LLM_PROVIDER=azure it returns:

from langchain_openai import AzureChatOpenAI

AzureChatOpenAI(
    azure_endpoint=settings.azure_openai_endpoint,
    api_key=settings.azure_openai_api_key,
    api_version=settings.azure_openai_api_version,
    azure_deployment=settings.azure_openai_chat_deployment,
    temperature=settings.temperature,
)

Step 5 — Run it

# with Docker, pass the .env through:
docker compose run --rm --env-file .env app \
  python -m rfpeval.cli evaluate samples/sample_proposal.md --rfp sample_rfp

# or locally:
python -m rfpeval.cli evaluate samples/sample_proposal.md --rfp sample_rfp

You now get the same evaluation pipeline — proposal ingestion, RFP requirement retrieval, weighted scoring, the human shortlist gate — powered by GPT-4o. Structured outputs (our RequirementAssessment schema) are well supported on Azure OpenAI, so the per-requirement JSON comes back clean.

Authentication: API key vs Microsoft Entra ID

For production, prefer a managed identity: assign the VM the Cognitive Services OpenAI User role and drop the key. AzureChatOpenAI supports token-based auth via azure_ad_token_provider.

Troubleshooting & common errors

What’s next

Azure OpenAI is wired in with a one-line provider switch. In Part 4 we do the same with Google Vertex AI and Gemini — including GCP authentication, which works differently from Azure — again without touching the graph.

Frequently asked questions

Do I reference the model name or the deployment name?

The deployment name. In Azure OpenAI you create a deployment of a model and call that deployment; mismatching it with the model name causes a 404 DeploymentNotFound.

Can I avoid storing an API key?

Yes — use Microsoft Entra ID with a managed identity and the Cognitive Services OpenAI User role. It removes the long-lived secret and gives you RBAC and auditing.

Does switching to Azure change the application code?

No. Only .env changes. The provider factory returns an AzureChatOpenAI model and the LangGraph workflow is untouched.

Conclusion

Switching the RFP evaluator to enterprise Azure OpenAI was a configuration change, not a code change: create the resource, deploy a chat and an embeddings model, set five environment variables, and run. Continue to Part 4: Google Vertex AI.

Independent educational project; not affiliated with any employer; not procurement or legal advice.