Wattage: A token-spend profiler and cost-regression gate for AI agents faizannraza / wattage Public Notifications You must be signed in to change notification settings Fork 0 Star 0 mainBranchesTagsG
By Coderz Club · 2026-07-27 · Tags: ai, git
Wattage: A token-spend profiler and cost-regression gate for AI agents
faizannraza / wattage Public Notifications You must be signed in to change notification settings Fork 0 Star 0 mainBranchesTagsGo to fileCodeOpen more actions menuFolders and filesNameNameLast commit messageLast commit dateLatest commit History48 Commits48 Commits.github.github actionaction benchmarksbenchmarks docsdocs examplesexamples npmnpm src/wattagesrc/wattage teststests .gitignore.gitignore CHANGELOG.mdCHANGELOG.md CODE_OF_CONDUCT.mdCODE_OF_CONDUCT.md CONTRIBUTING.mdCONTRIBUTING.md LICENSELICENSE README.mdREADME.md SECURITY.mdSECURITY.md mkdocs.ymlmkdocs.yml pyproject.tomlpyproject.toml uv.lockuv.lock View all filesRepository files navigation A Kill-A-Watt meter for your AI agents. Point it at a trace and it tells you exactly where your tokens are being burned and wasted, prices each waste pattern in real dollars, prescribes a fix, and can fail your CI when a change makes your agent measurably more expensive. A real captured agent trace (see provenance) — Wattage catches a stable prompt prefix being re-sent instead of cached, prices the waste, and prescribes the fix. Regenerate this GIF with vhs docs/assets/demo.tape (see the tape file for the exact command). Install and run uvx wattage report trace.json No config file, no API key, fully offline — point it at an OTLP JSON trace export and it prices every call and runs every detector. Don't have a trace yet? Getting your first trace covers both "I already have OTel traces" and "I have zero instrumentation" (a runnable, 5-minute path from nothing to a real, priced report). Or try it right now against the fixture shipped in this repo: git clone https://github.com/faizannraza/wattage cd wattage && uv sync uv run wattage report examples/sample_trace.json ╭──── ⚡ wattage — examples/sample_trace.json ────╮ │ Token Efficiency: A (100) Total cost: $0.0602 │ │ quality: unmeasured │ ╰─────────────────────────────────────────────────╯ Token breakdown ┏━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┃ Category ┃ Tokens ┃ ┡━━━━━━━━━━━━━━━━╇━━━━━━━━┩ │ input │ 18450 │ │ output │ 320 │ │ cache_read │ 0 │ │ cache_creation │ 0 │ │ reasoning │ 0 │ └────────────────┴────────┘ No findings — this trace looks efficient. pricing: 2026-07-18-verified Or get a self-contained, shareable HTML flame graph instead of the terminal view: uv run wattage report examples/sample_trace.json --html report.html The evidence, not a marketing claim Wattage's standout feature is the convergence engine — the nonconvergence detector, which catches an agent thrashing through a loop without making real progress, including patterns a naive exact-match duplicate detector structurally cannot see (a retry with a fresh timestamp each time, an oscillation between two strategies, a "productive-looking" stall where every call is technically unique but nothing is actually learned). Rather than assert that, we built a hand-reviewed set of 10 labeled synthetic loops and benchmarked Wattage's classifier against a real SHA-256 exact-match baseline implementation: Classifier Precision Recall F1 Wattage 1.00 1.00 1.00 SHA-256 exact-match 1.00 0.14 0.25 Reproduce it yourself — no cherry-picking, no hidden setup: uv run python -m benchmarks.harness And on a genuine captured agent trace (not synthetic — see benchmarks/traces/README.md for provenance), Wattage's prefix_churn fix simulation shows a 44.7% cost reduction ($0.000199 → $0.000110) from enabling prompt caching on the stable prefix — small dollar figures because it's a 3-turn demo trace, but the mechanism is identical at production scale. Run it against your own traces for numbers that matter: uv run python -c "from benchmarks.frontier import build_frontier; print(build_frontier())" Full methodology: The Convergence Engine. The badge uv run wattage badge trace.json --out wattage-badge.svg  Wire --badge-out into your CI job (see below) so it regenerates on every merge to your default branch, and the badge in your README stays live. How it works Three surfaces, one normalized data model underneath (sessions → tasks → loops → iterations → calls), built from OpenTelemetry GenAI semantic-convention traces: wattage report — ingests a trace, prices every call against a vendored, dated pricing snapshot, and runs eight detectors: Detector Catches prefix_churn Stable context re-sent instead of cached cache_gap Caching attempted but under-redeemed by later reads verbosity Output far beyond what the step needed redundant_tool_calls The same tool call repeated (exact or fuzzy) nonconvergence Loops that thrash, oscillate, or stall without progress retrieval_thrash Repeated retrieval that never yields relevant results model_mismatch A pricier model doing work a cheaper one could handle reasoning_overspend Heavy reasoning-token spend on a simple step Every finding is priced in real dollars, includes a concrete fix, and is tagged with a quality_risk tier (none / low / review) — a fix that coul
faizannraza / wattage Public Notifications You must be signed in to change notification settings Fork 0 Star 0 mainBranchesTagsGo to fileCodeOpen more actions menuFolders and filesNameNameLast commit messageLast commit dateLatest commit History48 Commits48 Commits.github.github actionaction benchmarksbenchmarks docsdocs examplesexamples npmnpm src/wattagesrc/wattage teststests .gitignore.gitignore CHANGELOG.mdCHANGELOG.md CODE_OF_CONDUCT.mdCODE_OF_CONDUCT.md CONTRIBUTING.mdCONTRIBUTING.md LICENSELICENSE README.mdREADME.md SECURITY.mdSECURITY.md mkdocs.ymlmkdocs.yml pyproject.tomlpyproject.toml uv.lockuv.lock View all filesRepository files navigation A Kill-A-Watt meter for your AI agents. Point it at a trace and it tells you exactly where your tokens are being burned and wasted, prices each waste pattern in real dollars, prescribes a fix, and can fail your CI when a change makes your agent measurably more expensive. A real captured agent trace (see provenance) — Wattage catches a stable prompt prefix being re-sent instead of cached, prices the waste, and prescribes the fix. Regenerate this GIF with vhs docs/assets/demo.tape (see the tape file for the exact command). Install and run uvx wattage report trace.json No config file, no API key, fully offline — point it at an OTLP JSON trace export and it prices every call and runs every detector. Don't have a trace yet? Getting your first trace covers both "I already have OTel traces" and "I have zero instrumentation" (a runnable, 5-minute path from nothing to a real, priced report). Or try it right now against the fixture shipped in this repo: git clone https://github.com/faizannraza/wattage cd wattage && uv sync uv run wattage report examples/sample_trace.json ╭──── ⚡ wattage — examples/sample_trace.json ────╮ │ Token Efficiency: A (100) Total cost: $0.0602 │ │ quality: unmeasured │ ╰─────────────────────────────────────────────────╯ Token breakdown ┏━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┃ Category ┃ Tokens ┃ ┡━━━━━━━━━━━━━━━━╇━━━━━━━━┩ │ input │ 18450 │ │ output │ 320 │ │ cache_read │ 0 │ │ cache_creation │ 0 │ │ reasoning │ 0 │ └────────────────┴────────┘ No findings — this trace looks efficient. pricing: 2026-07-18-verified Or get a self-contained, shareable HTML flame graph instead of the terminal view: uv run wattage report examples/sample_trace.json --html report.html The evidence, not a marketing claim Wattage's standout feature is the convergence engine — the nonconvergence detector, which catches an agent thrashing through a loop without making real progress, including patterns a naive exact-match duplicate detector structurally cannot see (a retry with a fresh timestamp each time, an oscillation between two strategies, a "productive-looking" stall where every call is technically unique but nothing is actually learned). Rather than assert that, we built a hand-reviewed set of 10 labeled synthetic loops and benchmarked Wattage's classifier against a real SHA-256 exact-match baseline implementation: Classifier Precision Recall F1 Wattage 1.00 1.00 1.00 SHA-256 exact-match 1.00 0.14 0.25 Reproduce it yourself — no cherry-picking, no hidden setup: uv run python -m benchmarks.harness And on a genuine captured agent trace (not synthetic — see benchmarks/traces/README.md for provenance), Wattage's prefix_churn fix simulation shows a 44.7% cost reduction ($0.000199 → $0.000110) from enabling prompt caching on the stable prefix — small dollar figures because it's a 3-turn demo trace, but the mechanism is identical at production scale. Run it against your own traces for numbers that matter: uv run python -c "from benchmarks.frontier import build_frontier; print(build_frontier())" Full methodology: The Convergence Engine. The badge uv run wattage badge trace.json --out wattage-badge.svg  Wire --badge-out into your CI job (see below) so it regenerates on every merge to your default branch, and the badge in your README stays live. How it works Three surfaces, one normalized data model underneath (sessions → tasks → loops → iterations → calls), built from OpenTelemetry GenAI semantic-convention traces: wattage report — ingests a trace, prices every call against a vendored, dated pricing snapshot, and runs eight detectors: Detector Catches prefix_churn Stable context re-sent instead of cached cache_gap Caching attempted but under-redeemed by later reads verbosity Output far beyond what the step needed redundant_tool_calls The same tool call repeated (exact or fuzzy) nonconvergence Loops that thrash, oscillate, or stall without progress retrieval_thrash Repeated retrieval that never yields relevant results model_mismatch A pricier model doing work a cheaper one could handle reasoning_overspend Heavy reasoning-token spend on a simple step Every finding is priced in real dollars, includes a concrete fix, and is tagged with a quality_risk tier (none / low / review) — a fix that coul