Advanced

Performance Benchmarks

Real-world token savings and compression results

What You'll Learn

  • Real-world RAG pipeline impact: 70% token savings at scale
  • Pattern-specific compression results across all 5 algorithms
  • Quality assurance metrics: 93.51% coverage with 342 tests

Real-World Impact: RAG Pipeline

1,000 Document Entries

JSON Format
~50,000
tokens per query
AXON Format
~15,000
tokens per query

Result: 70% savings = $350 saved per 1,000 queries at typical API rates

Pattern-Specific Results

204x
Repeated Values
99.5% reduction with RLE
4.3x
High-Cardinality Strings
65-75% with dictionary compression
22.7x
Boolean Arrays
95.6% with bit packing
59%
Simple Tables
General token savings vs JSON

Format Comparison

How AXON compares to other formats across different data types:

Data Type JSON CSV AXON
Flat table (100 rows) 4,200 tokens 1,800 tokens 1,700 tokens
Nested objects (50 records) 3,800 tokens N/A 1,200 tokens
Repeated enums (1,000 rows) 12,000 tokens 5,000 tokens 590 tokens
Boolean flags (500 rows) 6,800 tokens 2,200 tokens 300 tokens
Time-series events (200 rows) 8,500 tokens 3,400 tokens 2,100 tokens

Key takeaway: AXON matches or beats CSV on flat data while supporting nested objects that CSV cannot represent at all. The biggest gains come from repetitive and boolean data where specialized compression algorithms apply.

Measure Your Own Savings

Use the built-in tokenStats() function to benchmark AXON against JSON for your specific data:

import { encode, tokenStats } from '@axon-format/core';

const data = {
  orders: [
    { id: 1, customer: "Acme Corp", total: 1250.00, status: "shipped" },
    { id: 2, customer: "Globex", total: 890.50, status: "pending" },
    // ... your real data
  ]
};

const encoded = encode(data);
const stats = tokenStats(data, encoded);

console.log(stats);
// {
//   jsonTokens: 284,
//   axonTokens: 95,
//   savings: "66.5%",
//   ratio: "3.0x"
// }

Or use the CLI to compare files directly:

axon stats data.json data.axon

# Output:
# JSON tokens:  4,218
# AXON tokens:  1,642
# Savings:      61.1% (2.6x smaller)

Cost Calculator

Token savings translate directly to dollar savings. Here is a rough guide based on typical LLM API pricing (input tokens at ~$2.50 per million):

Monthly Queries JSON Cost AXON Cost Saved
10,000 $125 $38 $87/mo
100,000 $1,250 $375 $875/mo
1,000,000 $12,500 $3,750 $8,750/mo

Note: These estimates assume ~5,000 tokens of data per query with 70% AXON savings. Your actual savings depend on data shape and repetition patterns. Use tokenStats() to measure your specific use case.

Quality Assurance

AXON is production-ready with comprehensive testing:

  • 93.51% test coverage
  • 342 passing tests
  • Perfect round-trip encoding/decoding
  • MIT licensed and open source

Next: Best Practices

Learn tips for getting the most out of AXON

Best Practices →