AI Risk Audit and Control across Agents, Foundation Models, and Data
USC Professor · Founder of FORTIS Labs · Open-source author
Homepage · Research · Open Source · FORTIS Lab · Community · Contact
Note
Assistant Professor at USC Computer Science and PI of the FORTIS Lab. I research, build, and open-source AI risk audit and control: methods, benchmarks, and tools for inspecting and intervening on AI systems across the deployment stack. Creator of PyOD, the canonical Python anomaly-detection library (9.8k★; named by OpenAI, Apache Beam, PostHog, MLflow, and Genentech). Across my open source: ~24k GitHub stars and 42M+ downloads. Co-authored work (TrustLLM) is cited in a U.S. Senate HSGAC report, NIST AI 100-2e2025, and the International AI Safety Report 2026. ~12k Google Scholar citations.
Important
Founder of FORTIS Labs: a venture building auditable AI agents. Open-source infrastructure to capture, verify, and roll back agent decisions, anchored by auditable and drawing on a decade of anomaly-detection research. Introductions from investors and design partners welcome at hello@fortislabs.ai.
AI systems are deployed faster than they can be verified. Foundation models and autonomous agents now make consequential decisions, execute code, and interact with external services, often without systematic inspection of what they do or why. My research builds the methods, benchmarks, and open-source tools for AI risk audit and control.
Methodologically, this work extends my prior research on anomaly and outlier detection (the basis of the PyOD ecosystem) from data distributions to foundation-model behavior and agent decision traces, where unsafe, anomalous, or out-of-policy actions must be detected and reconstructed before and after deployment.
Three layers of the deployment stack:
- 🤖 Agent Layer: Risk Audit and Runtime Control. Auditability frameworks (Auditable Agents,
auditable), runtime control that intercepts tool calls before they fire (Aegis), static over-privilege scanning (agent-audit), post-run failure localization in multi-agent systems (GRADE), and agent-specific failure modes (over-privilege, cross-user contamination, the autonomy tax of defense training). - 🧠 Foundation-Model Layer: Trust and Robustness. Jailbreak detection for vision-language models, causal analysis of hallucination, query-agnostic attacks on retrieval-augmented generation, and LLM-as-anomaly-detector benchmarks.
- 📊 Data Layer: Anomaly and Out-of-Distribution Detection. PyOD ecosystem, ADBench, automatic OOD detector selection, modality-specific OOD methods, and few-shot cross-domain OOD detection.
~24k GitHub stars and 42M+ downloads across projects. Featured below (full list on the homepage):
| Project | Layer | What It Does |
|---|---|---|
| PyOD | Data | Canonical anomaly detection: 60+ detectors across tabular, time-series, graph, and image data. (9.8k★) |
| auditable | Agent | System of record for AI-agent decisions: capture, replay against live state, roll back. |
| agent-audit | Agent | Static security and over-privilege scanner for AI-agent code. |
| anywhere-agents | Tooling | One config for Claude Code and Codex across every project and session. (171★) |
| agent-style | Tooling | 21 writing rules loaded into AI agents at generation time. (432★) |
Tip
External adoption of PyOD. Named by OpenAI as expected operational tooling, shipped as a first-class ModelHandler in Apache Beam (Apache Software Foundation), running the live-traffic alerting subsystem in PostHog, the canonical anomaly-detection flavor in MLflow community-flavor docs, and embedded in Genentech (Roche) drug-discovery validators. 5,493 public repositories and 139 packages depend on PyOD (May 2026 snapshot). The U.S. DoD CDAO lists PyOD; ESA OPS-SAT flies it for spacecraft anomaly detection.
Other Notable Projects
- PyGOD (1.5k★): graph outlier detection, sister project to PyOD.
- GRADE: typed two-layer graph of agent execution and dependency; localizes the faulting step in multi-agent runs.
- Aegis (contributor): pre-execution firewall that gates each agent tool call.
- TrustLLM (625★, co-author): LLM trustworthiness benchmark cited by NIST AI 100-2e2025, the FLI AI Safety Index, U.S. Senate HSGAC, and DoD CDAO.
- AD-AGENT (99★): LLM-driven multi-agent anomaly detection platform.
- ADBench (1k★): NeurIPS 2022 official anomaly detection benchmark.
- Anomaly-Detection-Resources (9.3k★): curated resource hub for anomaly detection.
- CS-Paper-Checklist (1.6k★): practical sanity checklist for CS paper writing.
- agent-config: personal working repo and canonical source for
anywhere-agents.
I lead the FORTIS Lab at USC, working on AI risk audit and control, anomaly detection, and trustworthy AI systems, with a team of PhD, master's, and undergraduate researchers.
Founder & Maintainer of 博士栈 · CSPhD.org, a non-profit community for CS / AI / EE / Stats PhDs. Since 2022 it has grown to several thousand members and helped many land PhD positions, internships, and full-time roles, with a searchable board of 540+ PhD, RA, and postdoc openings.
- 🌐 Homepage · Google Scholar · LinkedIn
- ✉️
yue.z [AT] usc.edu