Tokentoll

What's New in v0.3.0

Dramatically improved real-world detection coverage -- tokentoll now catches LLM calls in complex codebases like aider, AutoGPT, ChatDev, instructor, and gpt-engineer.

Detection Improvements

• Lazy/nested imports -- detects from openai import OpenAI inside functions, not just at module level
• Re-exported modules -- handles from myapp.llm import litellm proxy patterns
• Factory function clients -- matches call chains like get_client().chat.completions.create() without needing to track client = OpenAI()
• Source text fallback -- can_handle checks source for call patterns even when no SDK import is found
• New call types -- audio transcription, speech, image generation

Performance

• 50x speedup on large codebases -- quick-reject string pre-filter skips irrelevant files before AST parsing
• 0.8s to scan Django (2903 files), down from 41s
• Zero false positives verified against Django, Flask, FastAPI

Variable Resolution

• Multi-pass constant propagation with fixed-point iteration
• Resolves os.getenv("MODEL", "gpt-4o"), os.environ.get(...) fallback defaults
• Function parameter defaults, kwargs splatting, class attributes, constructor arg propagation
• posonlyargs fix for Python 3.8+ compatibility

Stats

• 51 tests, all passing
• 5 SDK detectors: OpenAI, Anthropic, Google GenAI, LiteLLM, LangChain
• Zero runtime dependencies