Demonstration: Azure Open AI \& AI Foundry Pricing¶
Note
Please note the last updated date, as the values shown in the video might change over time due to price adjustments.
List of References (Click to expand)
Table of Content (Click to expand)
Pricing Models¶
Azure OpenAI Service supports different pricing models, including:
- Standard (On-Demand): Pay only for the tokens processed.
- Provisioned Throughput Units (PTUs): Ensure consistent throughput and minimal latency variance for scalable solutions. Click here to read more about What is provisioned throughput?
- Batch API: For global deployments and some regions, providing completions within 24 hours at a 50% discount on Global Standard Pricing. Click to read more about Getting started with Azure OpenAI batch deployments
| Billing Model | Description | Cost Calculation | Use Cases |
|---|---|---|---|
| Pay-As-You-Go | - Charged based on the number of tokens processed - Suitable for variable or unpredictable usage patterns |
- Cost per unit of tokens, different rates for different model series - Includes both input and output tokens |
- Applications with variable or unpredictable usage patterns - Flexibility in usage |
| Provisioned Throughput Units | - Reserved processing capacity for deployments - Ensures predictable performance and cost - Reserved Capacity: PTU provides reserved processing capacity for your deployments, ensuring predictable performance and cost. - Capacity Planning: It's important to estimate the required PTUs for your workload to optimize performance and cost. The token totals are calculated using the equation below. Click here for more explanation on how to calculate it |
- Hourly rate based on the number of PTUs deployed - Regardless of the number of tokens processed |
- Well-defined, predictable throughput requirements - Consistent traffic - Real-time or latency-sensitive applications - Cost Predictability: PTU offers more predictable costs compared to the pay-as-you-go model, which can vary based on usage. - Performance Guarantees: PTU provides guaranteed throughput and latency constraints. |
| Batch API | - Designed for asynchronous, large-scale batch processing - Processes multiple requests in a single file rather than individually - Offers a 24-hour turnaround - Available with global deployments in select regions, achieving up to a 50% discount on Global Standard Pricing |
- Calculated based on token usage (similar to Pay-As-You-Go) but incorporates a fixed discount (typically 50%) relative to standard pricing - Uses an independent enqueued token quota to ensure online workloads remain unaffected |
- Ideal for non-real-time, high-volume tasks such as bulk content generation, data processing, document summarization, translation, or sentiment analysis - Effective when processing delay is acceptable in exchange for significant cost savings and throughput optimization |
PTU Capacity Planning:
$$ \text{Total Tokens per Minute} = \text{Peak Calls per Minute} \times \Bigl( \text{Tokens in Prompt} + \text{Tokens in Response} \Bigr) $$
Pay-As-You-Go Price Calculation¶
The Pay-As-You-Go model
charges solely based on the cumulative number of tokens processed. To accurately estimate your costs, first determine the total tokens processed per call by adding the tokens from both the prompt and the response. Then, scale that figure by your total number of calls over your desired period.
Note
- Adjust the token counts and cost rate according to the specific model series or deployment region, as pricing may vary.
- For the most up-to-date cost details and additional pricing factors, please review the Azure Pricing Calculator and Azure OpenAI Service pricing page.
Calculation Formula:
$$ \text{Cost per Call} = \left(\frac{\text{Tokens in Prompt} + \text{Tokens in Response}}{1000}\right) \times \text{Cost per 1,000 Tokens} $$
For instance, if each call processes 150 tokens (e.g., 50 tokens in the prompt and 100 tokens in the response) and the rate is defined as X per 1,000 tokens, then the cost for one call becomes:
$$ \text{Cost per Call} = \frac{150}{1000} \times X $$
For multiple calls, simply multiply the cost per call by the total number of API calls:
$$ \text{Total Cost} = N \times \left(\frac{\text{Tokens in Prompt} + \text{Tokens in Response}}{1000}\right) \times \text{Cost per 1,000 Tokens} $$
Find below some examples:
Definitions
- Use Case: The scenario being measured (e.g., Small Chatbot Interaction, Enterprise QA Assistant, Bulk Content Generation).
- Model: The optional model choice (e.g., GPT‑3.5‑turbo, GPT‑4 with 8K/32K context) based on your needs.
- Daily Count: Number of interactions or API calls
per day. - Avg Input Tokens: Average tokens in the prompt
per interaction. - Avg Output Tokens: Average tokens in the completion
per interaction. - Daily Input Tokens: Daily Count × Avg Input Tokens.
- Daily Output Tokens: Daily Count × Avg Output Tokens.
- Daily Total Tokens: Sum of daily input and output tokens.
- Monthly Input Tokens: Daily Input Tokens × 30.
- Monthly Output Tokens: Daily Output Tokens × 30.
- Monthly Total Tokens: Sum of monthly input and output tokens.
| Use Case | Model (Optional) | Daily Count | Avg Input Tokens | Avg Output Tokens | Daily Input Tokens | Daily Output Tokens | Daily Total Tokens | Monthly Input Tokens | Monthly Output Tokens | Monthly Total Tokens |
|---|---|---|---|---|---|---|---|---|---|---|
| Small Chatbot Interaction | (e.g., GPT‑3.5‑turbo or GPT‑4) | 200 | 50 | 150 | 200 × 50 = 10,000 | 200 × 150 = 30,000 | 10,000 + 30,000 = 40,000 | 10,000 × 30 = 300,000 | 30,000 × 30 = 900,000 | ~1,200,000 |
| Enterprise QA Assistant | (e.g., GPT‑4 with 8K or 32K context depending on precision needs) | 500 | 100 | 300 | 500 × 100 = 50,000 | 500 × 300 = 150,000 | 50,000 + 150,000 = 200,000 | 50,000 × 30 = 1,500,000 | 150,000 × 30 = 4,500,000 | ~6,000,000 |
| Bulk Content Generation | (e.g., GPT‑3.5‑turbo or another efficient variant) | 2,000 | 100 | 150 | 2,000 × 100 = 200,000 | 2,000 × 150 = 300,000 | 200,000 + 300,000 = 500,000 | 200,000 × 30 = 6,000,000 | 300,000 × 30 = 9,000,000 | ~15,000,000 |
https://github.com/user-attachments/assets/9ede2347-faaf-42c7-8e4c-64ad7c461938
PTUs and TPM relationship¶
Provisioned Throughput Units (PTUs)
Tokens Per Minute (TPM)
E.g
| PTUs | Calls per Minute | Tokens in Prompt | Tokens in Response | Tokens per Minute (TPM) |
|---|---|---|---|---|
| 1 | 10 | 50 | 100 | 1,500 |
| 2 | 20 | 50 | 100 | 3,000 |
| 5 | 50 | 50 | 100 | 7,500 |
| 10 | 100 | 50 | 100 | 15,000 |
| 20 | 200 | 50 | 100 | 30,000 |
| 50 | 500 | 50 | 100 | 75,000 |
Explanation:
- PTUs: Provisioned Throughput Units represent the capacity of tokens that can be processed per minute.
- Calls per Minute: The number of API calls that can be made per minute.
- Tokens in Prompt: The number of tokens in the input prompt for each call.
- Tokens in Response: The number of tokens in the model's response for each call.
- Tokens per Minute (TPM): The total number of tokens processed per minute, calculated as:
$$ \text{TPM} = \text{Calls per Minute} \times (\text{Tokens in Prompt} + \text{Tokens in Response}) $$
Example Calculation: For 50 PTUs:
- Calls per Minute: Calculate the number of calls per minute:
$$ \text{Calls per Minute} = \text{PTUs} \times \text{Calls per PTU per Minute} $$
$$ \text{Calls per Minute} = 50 \times 10 = 500 $$
- Tokens per Minute: Calculate the total tokens per minute:
$$ \text{TPM} = \text{Calls per Minute} \times (\text{Tokens in Prompt} + \text{Tokens in Response}) $$
$$ \text{TPM} = 500 \times (50 + 100) = 500 \times 150 = 75,000 $$
This means with 50 PTUs, you can process 75,000 tokens per minute.
Provisioned (PTU) Capacity Calculator¶
Improve accuracy of your estimate by adding multiple workloads to your PTU calculation. Each workload will be calculated and displayed as well as the aggregate total if both are running at the same time to your deployment.
https://github.com/user-attachments/assets/27beba15-57d6-4a2b-943e-496829644dbe