How to Monitor OpenAI Usage and Costs with the Usage API and Cost API

Monitoring your OpenAI usage and costs is crucial for optimizing your AI projects. The OpenAI Usage API and Cost API provide detailed insights into your usage patterns and expenses. This guide will walk you through the process of retrieving, parsing, and visualizing this data.

Step 1: Install Required Libraries

Before you start, ensure you have the necessary libraries installed. You can install them using pip:

!pip install requests pandas numpy matplotlib --quiet

Step 2: Import Libraries

Next, import the required libraries in your Python script or Jupyter notebook:

import requests
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import time
import json

Step 3: Set Up API Credentials and Parameters

Set up your OpenAI Admin Key and define the API endpoint and parameters:

OPENAI_ADMIN_KEY = '<PLACEHOLDER>'  # Replace with your actual ADMIN API key
 
headers = {
    "Authorization": f"Bearer {OPENAI_ADMIN_KEY}",
    "Content-Type": "application/json"
}
 
url = "https://api.openai.com/v1/organization/usage/completions"
 
days_ago = 30
start_time = int(time.time()) - (days_ago * 24 * 60 * 60)
 
params = {
    "start_time": start_time,
    "bucket_width": "1d",
    "limit": 7
}

Step 4: Retrieve Usage Data

Use a loop to handle pagination and retrieve all usage data:

all_data = []
page_cursor = None
 
while True:
    if page_cursor:
        params["page"] = page_cursor
 
    response = requests.get(url, headers=headers, params=params)
 
    if response.status_code == 200:
        data_json = response.json()
        all_data.extend(data_json.get("data", []))
 
        page_cursor = data_json.get("next_page")
        if not page_cursor:
            break
    else:
        print(f"Error: {response.status_code}")
        break
 
if all_data:
    print("Data retrieved successfully!")
else:
    print("Issue: No data available to retrieve.")

Step 5: Parse and Visualize Usage Data

Parse the JSON response and create a pandas DataFrame for easier manipulation and analysis:

records = []
 
for bucket in all_data:
    start_time = bucket.get("start_time")
    end_time = bucket.get("end_time")
    for result in bucket.get("results", []):
        records.append({
            "start_time": start_time,
            "end_time": end_time,
            "input_tokens": result.get("input_tokens", 0),
            "output_tokens": result.get("output_tokens", 0),
            "num_model_requests": result.get("num_model_requests", 0)
        })
 
df = pd.DataFrame(records)
df['start_datetime'] = pd.to_datetime(df['start_time'], unit='s')
df['end_datetime'] = pd.to_datetime(df['end_time'], unit='s')
 
df = df[
    [
        "start_datetime", "end_datetime", "start_time", "end_time",
        "input_tokens", "output_tokens", "num_model_requests"
    ]
]
 
df.head()

Step 6: Visualize Token Usage Over Time

Create a bar chart to visualize input and output token usage over time:

if not df.empty:
    plt.figure(figsize=(12, 6))
    
    width = 0.35
    indices = range(len(df))
    
    plt.bar(indices, df['input_tokens'], width=width, label='Input Tokens', alpha=0.7)
    plt.bar([i + width for i in indices], df['output_tokens'], width=width, label='Output Tokens', alpha=0.7)
    
    plt.xlabel('Time Bucket')
    plt.ylabel('Number of Tokens')
    plt.title('Daily Input vs Output Token Usage Last 30 Days')
    plt.xticks([i + width/2 for i in indices], [dt.strftime('%Y-%m-%d') for dt in df['start_datetime']], rotation=45)
    plt.legend()
    plt.tight_layout()
    plt.show()
else:
    print("No data available to plot.")

Step 7: Retrieve and Visualize Cost Data

Use the Costs API to retrieve and visualize cost data:

costs_url = "https://api.openai.com/v1/organization/costs"
all_costs_data = []
page_cursor = None
 
while True:
    costs_params = {
        "start_time": start_time,
        "bucket_width": "1d",
        "limit": 30
    }
 
    if page_cursor:
        costs_params["page"] = page_cursor
 
    costs_response = requests.get(costs_url, headers=headers, params=costs_params)
 
    if costs_response.status_code == 200:
        costs_json = costs_response.json()
        all_costs_data.extend(costs_json.get("data", []))
 
        page_cursor = costs_json.get("next_page")
        if not page_cursor:
            break
    else:
        print(f"Error: {costs_response.status_code}")
        break
 
if all_costs_data:
    print("Costs data retrieved successfully!")
else:
    print("No costs data found.")

Step 8: Parse and Visualize Cost Data

Parse the JSON response and create a pandas DataFrame for cost data:

cost_records = []
 
for bucket in all_costs_data:
    start_time = bucket.get("start_time")
    end_time = bucket.get("end_time")
    for result in bucket.get("results", []):
        cost_records.append({
            "start_time": start_time,
            "end_time": end_time,
            "amount_value": result.get("amount", {}).get("value", 0),
            "currency": result.get("amount", {}).get("currency", "usd"),
            "line_item": result.get("line_item")
        })
 
cost_df = pd.DataFrame(cost_records)
cost_df['start_datetime'] = pd.to_datetime(cost_df['start_time'], unit='s')
cost_df['end_datetime'] = pd.to_datetime(cost_df['end_time'], unit='s')
 
cost_df.head()

Step 9: Visualize Costs by Line Item

Create a bar chart to visualize the total costs aggregated by line item:

if not cost_df.empty:
    cost_df['date'] = cost_df['start_datetime'].dt.date
    cost_per_day = cost_df.groupby('date')['amount_value'].sum().reset_index()
    
    plt.figure(figsize=(12, 6))
    plt.bar(cost_per_day['date'], cost_per_day['amount_value'], width=0.6, color='skyblue', alpha=0.8)
    plt.xlabel('Date')
    plt.ylabel('Total Cost (USD)')
    plt.title('Total Cost per Day (Last 30 Days)')
    plt.xticks(rotation=45, ha='right')
    plt.tight_layout()
    plt.show()
else:
    print("No cost data available to plot.")

Conclusion

By following these steps, you can effectively monitor your OpenAI usage and costs. This data can help you optimize your AI projects and manage your expenses more efficiently.

Reference

Source: OpenAI API Documentation Author: Mitch Welzen