The challenge
Large-language-model services execute many requests together. Continuous batching, shared caches, dynamic scheduling, and overlapping work improve service performance, but they make energy a property of the execution as a whole. A request may fill idle capacity, extend a GPU’s active tail, or displace cached work needed later. Aggregate counters can measure the system, yet they often erase the relationships needed to explain which decisions changed its energy use.
The work
JouleLedger proposes a request-preserving representation that connects synchronized GPU-energy measurements with the requests, scheduler steps, cache relationships, and shared work that produced them.
The research is deliberately staged. The first study asks whether shared-execution information improves after-execution estimates relative to strong request-only and aggregate baselines. A separate later study would ask whether the representation can predict the total energy difference between feasible serving actions before they are taken. Accounting success would not, by itself, establish a successful controller.
Why it matters
If validated, JouleLedger could make inference energy more observable: helping researchers diagnose regressions, report workload consequences more defensibly, and identify which serving decisions have energy differences large enough to act on. It could also provide an evidence-based energy signal to systems that already choose among replicas, batches, caches, and hardware settings.
The project does not yet claim energy savings. Its value begins with making the chain from physical measurement to request-level explanation inspectable—and requiring every later control claim to earn its own experimental support.