Protecting AI APIs from Cost Attacks

Protect LLM and AI APIs from runaway inference costs with token-aware rate limiting and quota management.

Securing LLM Endpoints from Cost Attacks

The emergence of generative AI and LLM APIs has introduced a critical security risk: Cost Attacks.

1. The Cost Vulnerability

Traditional API rate limiting counts requests (e.g., 100 requests per minute). However, in generative AI pipelines, this metric is insufficient:

• A single call containing a massive context window prompt can consume 100,000+ LLM tokens.
• At standard provider pricing, a small number of concurrent, high-context requests can run up thousands of dollars in cloud bills within minutes.
• Malicious actors or poorly written client loops can trigger severe billing cost spikes.

2. Token-Cost Rate Limiting

To protect AI endpoints, you must enforce limits based on Token Counts and Monetary Costs:

• Token Budgets: Track cumulative tokens (input + output) consumed by a client in a time window.
• Cost Budgets: Map estimated dollar costs to requests (e.g. GPT-4 calls have a higher weight than GPT-3.5 calls) and enforce maximum dollar budgets per hour or billing cycle.

3. Implementation Patterns

Configure your API middleware to calculate token usage prior to forwarding calls:

`javascript

// Express Middleware Example

const result = await lyaClient.checkWithTokens({

endpoint: '/v1/chat/completions',

tokens: promptTokens + completionTokens,

cost: (promptTokens * 0.0015 + completionTokens * 0.002) / 1000 // Monetized cost

});

if (!result.allowed) {

return res.status(429).json({ error: 'LLM token budget exceeded.' });

}

`