AI Agent Aggregates Stolen LLM Access

A SANS ISC honeypot caught a coding agent hunting, stealing, and reselling LLM API access through a self-built aggregation gateway.

SANS Internet Storm Center researchers used an AI honeypot emulating an OpenAI-compatible inference endpoint to capture the operations of an attacker running a semi-autonomous coding agent for LLM-access theft and resale — an evolution of the LLMjacking technique previously documented by Sysdig. The agent's requests to the honeypot leaked roughly 43KB of its own operational material, including an AGENTS.md control file, an offensive playbook, reconnaissance scripts, harvested API keys, target lists, and working history, because the client embedded session context and instructions directly in its model requests.

The captured workflow followed a consistent pipeline: locate LLM resale gateways via FOFA queries (e.g., for V2Board panels and subscription-userinfo headers); acquire access through open registration with free starting balances, default credentials, group_id-based authorization weaknesses, exposed endpoints like /api/auth-files, and automated trial-account creation using disposable email and CAPTCHA-solving services; validate stolen keys by testing inference quality (including a factorial-computation logic test to distinguish real models from canned/fake responses); and finally aggregate validated credentials into a self-hosted New-API gateway. In one capture the operator loaded 379 upstream endpoints as channels, pruned 341 that failed validation, and mapped the surviving 38 onto five popular model names with round-robin failover — effectively building working infrastructure to resell stolen inference capacity. When gateway rate limits interfered, the agent directly edited the SQLite backend to clear session rows and inject an admin token.

The significant point for defenders is the feedback loop: the same agent used to find and steal LLM access also builds and operates the infrastructure that re-serves it, creating a partially self-expanding supply chain (human-steered, not fully autonomous). Operators of LLM gateways should review their exposure to the same weaknesses the attacker's playbook targeted: open registration with starting balances, client-supplied authorization fields, exposed account-management endpoints, default credentials, unauthenticated model/account disclosure, and excessive default billing limits — and should assume these checks can now be run continuously by automated agents. Organizations using free or cheap third-party LLM proxies should also consider that their own coding agents may leak AGENTS.md content, project instructions, source code, command output, and other operational context to an untrusted or malicious endpoint.

Source reporting: https://isc.sans.edu/diary/rss/33332