Perplexity releases numbat AI agent observability tool

SANS ISC reviews numbat, Perplexity AI's open-source tool for monitoring and enforcing controls on AI agents and MCP servers to detect risky behaviors like network scanning.

This SANS Internet Storm Center diary is a hands-on review of numbat, an open-source AI agent observability tool released by Perplexity AI, rather than a vulnerability or threat report. The author frames the tool as a response to a broader industry problem: unmanaged sprawl of AI agents and MCP (Model Context Protocol) server connections in enterprises, which often operate with excessive privilege, opaque execution paths, and poor identity controls. This lack of governance creates risk of lateral movement, shadow collaboration between agents, and unauthorized data exfiltration — concerns amplified by a referenced but unspecified recent OpenAI/Hugging Face incident.

numbat is a Go binary (cross-platform: macOS, Linux, Windows) that hooks into desktop, CLI, IDE, and gateway AI agents (e.g., Claude, Gemini) to capture events, findings, and enforcement decisions via local hooks/plugins, OTLP/HTTP logs, and on-disk session artifacts written to ~/.numbat/records.ndjson. It ships with a categorized detection rule catalog (secrets, exfiltration, integrity, execution, reconnaissance, privilege, lateral movement, impact, source control, tampering, persistence, sequences) that can operate in monitor-only or enforcement mode. The author demonstrates the recon.network_sweep detector (mapped to MITRE ATT&CK T1046 - Network Service Discovery) by prompting Claude Code to run an nmap scan, showing that numbat successfully logs the finding. The tool also supports incident response workflows, including case creation, SHA256-hashed evidence manifests, and timeline generation grouped by agent/session.

For defenders, this is not a threat advisory but a tooling recommendation: enterprises deploying AI coding agents or MCP-connected tooling should consider agent-level observability solutions like numbat to gain visibility into agent actions, detect anomalous or adversarial behaviors (e.g., agents performing reconnaissance, privilege escalation, or data exfiltration attempts), and support IR investigations with tamper-evident evidence packaging. The piece also highlights MDM-style fleet deployment options for managed enforcement across an organization's AI agent estate.

Source reporting: https://isc.sans.edu/diary/rss/33312