Privacy Architecture

InfraPrism was built from day one with privacy as a core architectural principle, not an afterthought. This document explains the design decisions that make privacy-first AI cost tracking possible.

The Problem with Proxies

Most AI observability tools use a proxy architecture:

Your Application ──▶ Proxy Server ──▶ OpenAI/Anthropic
                     (Third Party)

This means:

• All your prompts flow through their servers
• All your responses flow through their servers
• All your data is visible to a third party
• Compliance becomes complex (HIPAA, PCI, GDPR, etc.)

For many organizations, especially in regulated industries, this is unacceptable.

Our Solution: SDK-Only

InfraPrism uses a fundamentally different architecture:

Your Application ──▶ OpenAI/Anthropic (direct connection)
       │
       └──▶ InfraPrism (metadata only)

Your data goes directly to the LLM provider. We only receive metadata.

Architectural Guarantees

1. No Content Transmission

The SDK is designed to never transmit content. This is enforced in code:

# Simplified SDK logic
def track_request(request, response):
    event = {
        "model": request.model,
        "input_tokens": response.usage.prompt_tokens,
        "output_tokens": response.usage.completion_tokens,
        "latency_ms": elapsed_time,
        # Note: NO content fields
    }
    send_to_infraprism(event)

There are no fields for prompt or response content. The data structures don’t support it.

2. Client-Side Token Counting

Token counts come from the LLM provider’s response, not from analyzing your content:

# We use the provider's token count, not our own analysis
input_tokens = response.usage.prompt_tokens
output_tokens = response.usage.completion_tokens

We never need to see your content to count tokens.

3. Client-Side Cost Calculation

Cost is calculated in your environment:

cost = (input_tokens * input_price) + (output_tokens * output_price)

We receive the pre-calculated cost, not the raw data to calculate it.

4. Open Source Verification

Our SDK is open source. You can:

• Read the code
• Audit the data collection
• Build from source
• Run in air-gapped environments

View on GitHub →

What This Means for Compliance

HIPAA

Because we never see Protected Health Information (PHI), we’re compliant by architecture:

• No PHI flows to InfraPrism servers
• No BAA technically required (we don’t handle covered data)
• BAA available for Enterprise customers who want additional assurance

PCI-DSS

For payment card data:

• No cardholder data flows to InfraPrism
• Your PCI scope is not affected by InfraPrism
• Audit and compliance is simplified

GDPR

For personal data protection:

• No personal data in prompts reaches us
• Entity IDs should be internal identifiers, not PII
• EU data residency available for Enterprise

Attorney-Client Privilege

For legal organizations:

• Privileged communications stay within your systems
• No risk of waiving privilege through third-party disclosure
• Full confidentiality maintained

Defense in Depth

Even with our architecture, we implement additional protections:

Encryption

• TLS 1.3 for all data in transit
• AES-256 for all data at rest
• API keys hashed with bcrypt

Access Control

• Role-based access to dashboards
• SSO with SAML/OIDC (Enterprise)
• Audit logging for all access

Data Minimization

• Only collect what’s needed for cost calculation
• Configurable data retention
• Data deletion on request

Comparison: Proxy vs SDK-Only

Aspect	Proxy Architecture	SDK-Only (InfraPrism)
Sees prompts	✅ Yes	❌ No
Sees responses	✅ Yes	❌ No
Added latency	✅ Yes	❌ No
Single point of failure	✅ Yes	❌ No
HIPAA complexity	High	Low
PCI scope impact	Yes	No

Verification

You don’t have to trust us—you can verify:

1. Inspect the SDK

pip download infraprism --no-deps
unzip infraprism-*.whl
# Read the source code

2. Monitor Network Traffic

Use tools like Wireshark or mitmproxy to inspect what the SDK sends:

# All requests go to api.infraprism.com
# Inspect the payload - no content fields

3. Run in Debug Mode

client = InfraPrismOpenAI(infraprism_debug=True)
# Logs all events being sent

FAQ

Q: How do you calculate costs without seeing tokens?

A: We receive token counts from the LLM provider’s response. We never analyze your content.

Q: Can you reconstruct prompts from metadata?

A: No. We only receive token counts, not the tokens themselves. You can’t reverse token counts back to text.

Q: What if I accidentally include PII in tags?

A: Don’t include PII in entity IDs or custom tags. Use internal identifiers instead.

Q: Is the SDK truly open source?

A: Yes, MIT licensed. View the repository →

Next Steps

• Data We Collect - Complete data inventory
• Security - Security practices
• HIPAA Compliance - Healthcare compliance