apidays New York 2025 - Why I Built Another Carbon Measurement Tool for LLMs by Pascal Joly (IT Climate Ed)

Why I Built Another Carbon Measurement Tool for LLMs (And What I Learned Along the Way)
Pascal Joly, Sustainability Consultant and Instructor at IT Climate Ed

apidays New York 2025
API Management for Surfing the Next Innovation Waves: GenAI and Open Banking
May 14 & 15, 2025

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Why I Built a Carbon Measurement Tool for LLMs (And What I Learned Along the Way) Pascal Joly, IT Climate Ed, LLC
2. “it's super fun seeing people love images in ChatGPT. but our GPUs are melting.” Sam Altman, March 2025
3. The Ghibli image craze March 2025: ChatGPT image generator creates viral trend 5M images generated/hour Purely recreational = 1 AI image 1 phone charge Courtesy of X, @Zeneca- Ghibli-Style AI Art Misses What Made Miyazaki Matter
4. Energy demand from data centers is set to more than double by 2030 to 945TW IEA report, April 2025
5. My journey to Sustainable AI
6. Searching for answers Will AI benefits outweigh its environmental impact? How much of the data center expected growth can be attributed to AI? How models compare in terms of energy efficiency for training and inference? How much carbon emissions do I emit from my chatbot conversations? How much does LLM energy intensity vary by task type and complexity? Will gains in model energy efficiency solve the impending energy crisis?
7. Training vs. Inference: Energy Consumption Inference / Usage 65% Training 35% Source: Sustainable AI course, Terra.do
8. Energy impact of LLM conversations 400M Chatgpt Users 2.9 Wh Per Query (*) 33,000 US Homes (Daily Power Equivalent) (*) Source: IEA Energy report, 2025
9. What if we could change how people use LLMs? Wasteful Prompt Hi There! I was wondering if you could please tell me about renewable energy sources. Thank you so much! Efficient Prompt List renewable energy sources Extra tokens: ~40% wasted energy Minimal tokens: optimal energy use
10. Estimating AI emissions: tool ecosystem 👤 Role 🎯 Goal 🛠 Recommended Tool(s) Product Manager Model selection (training phase) ML CO2 Calculator, Green Algorithm Product Manager Model selection (inference phase) EcologITs, AI Energy Score Sustainability Analyst (Enterprise) Carbon footprint of current projects Cloud Dashboards, Carbon Accounting Platforms Developer Evaluate/improve model performance CarbonTracker, CodeCarbon End User Estimate my inference emissions (chatGPT, claude…) ??
11. Introducing AIWattch End User-focused ⏱ Real-time feedback on LLM interactions Open source and collaborative Private and Secure Educational Support most popular LLMs
12. Install AI Wattch from the Chrome Web Store
13. AI Wattch in Action: Demo
14. Admin parameters
15. GitHub Repository
16. AI Emissions Estimator - System Flow ChatGPT Interface DOM Monitor Token Estimator Char/token: 4 Emissions Calculator Real-time Display Admin Dashboard PUE: 1.125 Grid: 383 gCO2e/kWh Input: 0.002 Wh/token Output: 0.01 Wh/token
17. Calculation 1.Number of input and output tokens: #Char input x Token/Char = Ti #Char reply x Token/Char = Tr 2.Total Energy during conv exchange: PUE x ((Ti x (EnergyF Ti) + Tr x (EnergyF Tr)) = Conv Energy (kWh) 3.CO2 emissions: Conv Energy x CO2i = CO2 emitted (g) 2300.56
18. My LLM-assisted Product Development Journey Frame the problem and brainstorm possible solutions. 󰣗 Prototype with Vibe coding…. Debugging (!) PRD and project proposal Hired a developer On boarded the project on Github for open source contributions
19. Challenges and Lessons Learned 🔒 Transparency Gap Limited visibility into non-open source models' energy consumption 📍Data center Uncertainty Unknown carbon intensity and PUE ⏱ Real Time complexity Estimating query on the fly: response time Browser Integration DOM updates, dynamic web interfaces 🔄 Task Variability Different AI workloads have vastly different energy profiles 💡Key Learning Set the right level of expectations for accuracy. The goal is estimations.
20. Impact: the Scale of Opportunity 🧠 Efficient Prompting If just 1% (4M) adopt energy-efficient prompting (40%): → Saves ~1.6 GWh / year →Avoids 730 tons CO₂/year ➡ It’s Bigger Than ChatGPT Also used by millions: Claude · Gemini · LLaMA · Copilot · Mistral · DeepSeek… It’s Bigger Than Energy An average 100MW datacenter: 2M ltrs water/day (*) source Source: IEA AI and Energy report, April 2025
21. Impact: transparency and standardization 📣 Awareness → Transparency • Creates user demand for AI energy labels • Pressure vendors for disclosure & accountability 🏢 From Habits to Standards • Teams: Best practices • Organization: Internal policies • Enterprise: Industry-wide standards
22. Roadmap • Time based calculation • Actionable tips • Multimodal Feature backlog: https://github.com/AIWattch/browser-extension/issues Near Term Medium Term Long Term • API integration for token counting • Reasoning Model • Additional LLM and browser support • Teams • Summary dashboard • Gamified approach
23. AIWach in the tool ecosystem 👤 Role 🎯 Goal 🛠 Recommended Tool(s) Product Manager Model selection (training phase) ML CO2 Calculator, Green Algorithm Product Manager Model selection (inference phase) EcologITs, AI Energy Score Sustainability Analyst (Enterprise) Carbon footprint of current projects Cloud Dashboards, Carbon Accounting Platforms Developer Evaluate/improve model performance CarbonTracker, CodeCarbon End User Estimate inference emissions (ChatGPT, Claude…) AIWattch
24. What can YOU do? Join the Movement Contribute: https://github.com/AIWattch/browser-extension Try AIWattch and provide feedback Take a course Bring changes from within
25. Q&A
26. Thanks! Contribute to AIWattch 👉