InfraPrism

Custom Tags

Custom tags let you add arbitrary metadata to your LLM calls. Use them to slice and dice your costs by any dimension that matters to your business.

Basic Usage

Add a tags dictionary to any call:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    entity_type="customer",
    entity_id="acme-corp",
    tags={
        "feature": "chatbot",
        "env": "production",
        "version": "2.1.0",
    },
)

Common Tag Patterns

Environment

Track costs by environment:

import os

tags = {
    "env": os.getenv("ENVIRONMENT", "development"),  # production, staging, development
}

Feature/Product

Track costs by feature or product area:

tags = {
    "feature": "document-search",  # or "chatbot", "code-review", "summarizer"
    "product": "enterprise-suite",
}

Version

Track costs across releases:

tags = {
    "version": "2.1.0",
    "prompt_version": "v3",
}

Model Experiments

Compare different approaches:

tags = {
    "experiment": "rag-v2",
    "retrieval_method": "hybrid",
    "chunk_size": "512",
}

User Context

Add non-PII user context:

tags = {
    "user_tier": "premium",      # free, basic, premium, enterprise
    "user_tenure": "new",        # new, returning, long-term
    "plan": "growth",
}

Request Context

Add request-specific context:

tags = {
    "request_source": "api",     # api, web, mobile, slack
    "priority": "high",
    "workflow": "onboarding",
}

Multi-Dimensional Analysis

Combine multiple tags for powerful analysis:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    entity_type="customer",
    entity_id="acme-corp",
    tags={
        "feature": "chatbot",
        "env": "production",
        "tier": "enterprise",
        "region": "us-east",
    },
)

This lets you answer questions like:

  • “What’s the chatbot cost in production for enterprise customers?”
  • “How does US-East compare to EU-West?”
  • “Which features are most expensive for premium users?”

Tag Values

String Values

All tag values are stored as strings:

tags = {
    "feature": "chatbot",
    "retry_count": "3",        # String, not int
    "cached": "true",          # String, not bool
}

Naming Conventions

Use consistent naming:

# Good - consistent snake_case
tags = {
    "user_tier": "premium",
    "request_source": "api",
}

# Avoid - inconsistent naming
tags = {
    "userTier": "premium",
    "request-source": "api",
}

Reserved Tags

The following tag keys are reserved:

  • _infraprism_* - Internal use
  • entity_type - Use the dedicated parameter
  • entity_id - Use the dedicated parameter
  • session_id - Use the dedicated parameter

Filtering in Dashboard

Use tags to filter and group in the dashboard:

  1. Filter by tag - “Show only production traffic”
  2. Group by tag - “Group costs by feature”
  3. Compare tags - “Compare v2.0 vs v2.1 costs”

API Queries

Query by tags using the Analytics API:

response = requests.get(
    "https://api.infraprism.com/v1/analytics/costs",
    headers={"Authorization": "Bearer ip-..."},
    params={
        "entity_type": "customer",
        "start_date": "2025-01-01",
        "tags": "feature:chatbot,env:production",  # Filter by tags
        "group_by": "tags.version",                 # Group by tag
    },
)

Dynamic Tags

Add tags dynamically based on context:

def build_tags(request) -> dict:
    tags = {
        "env": os.getenv("ENVIRONMENT"),
        "version": __version__,
    }

    if hasattr(request, "feature"):
        tags["feature"] = request.feature

    if hasattr(request, "experiment_id"):
        tags["experiment"] = request.experiment_id

    return tags

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    entity_type="customer",
    entity_id=customer_id,
    tags=build_tags(request),
)

Best Practices

Keep Tags Low-Cardinality

Avoid high-cardinality values:

# Good - low cardinality
tags = {"region": "us-east"}

# Avoid - high cardinality
tags = {"user_id": "12345"}  # Use entity_id instead
tags = {"timestamp": "1704067200"}  # Automatically tracked

Don’t Include Sensitive Data

Tags should not contain PII or secrets:

# Good
tags = {"user_tier": "premium"}

# Avoid
tags = {"email": "[email protected]"}  # PII
tags = {"api_key": "sk-..."}          # Secret

Use Consistent Values

Standardize tag values:

# Good - consistent values
tags = {"env": "production"}  # Always "production"

# Avoid - inconsistent values
tags = {"env": "prod"}        # Sometimes "prod"
tags = {"env": "Production"}  # Sometimes "Production"

Limit Tag Count

Keep the number of tags reasonable:

# Good - focused tags
tags = {
    "feature": "chatbot",
    "env": "production",
    "version": "2.0",
}

# Avoid - too many tags
tags = {
    "feature": "chatbot",
    "env": "production",
    "version": "2.0",
    "region": "us-east",
    "datacenter": "dc1",
    "pod": "pod-3",
    # ... 20 more tags
}

Next Steps