LiteLLM supply chain attack: what dev teams must do

What happened

On March 24, 2026, someone published malicious versions of LiteLLM (1.82.7 and 1.82.8) directly to PyPI, bypassing the project's normal release process. The compromised packages contained a credential stealer that harvested SSH keys, cloud provider credentials, Kubernetes configs, API keys, and more, then exfiltrated them to a domain that looked like it belonged to LiteLLM but didn't.

The malicious packages were live for roughly 40 minutes before PyPI quarantined them. That doesn't sound like much. But LiteLLM gets installed around 15-20 million times per week, which works out to about 1,700 installs per minute. During that window, over 119,000 downloads happened. An estimated 40-50% of those were unpinned installs pulling the latest version automatically.

The attack originated from a compromised Trivy dependency in LiteLLM's CI/CD security scanning workflow. A group called TeamPCP is believed to be responsible, and there are indications they've partnered with the Lapsus$ extortion group to monetize stolen data. At RSA Conference last week, incident response researchers confirmed knowledge of over 1,000 impacted SaaS environments, and threat hunters estimate data was exfiltrated from 500,000 machines.

This week, the first downstream victim went public. An AI recruiting startup confirmed it was "one of thousands of companies" affected, and the Lapsus$ group is now auctioning what they claim is 4TB of stolen data, including source code, credentials, and VPN configurations.

Why this one matters more than the usual CVE

We see supply chain advisories often enough that they start to blur together. This one is different for a few reasons.

First, the attack vector. The compromise didn't start with LiteLLM itself. It started with Trivy, an open-source vulnerability scanner that many teams run inside their CI/CD pipelines. Think about that for a second: the tool you're using to scan for vulnerabilities was the entry point. Your security tooling compromised your security. That's not just ironic, it's a reminder that CI/CD pipelines are a high-value target and every tool running in them needs the same scrutiny you'd give to production dependencies.

Second, the position of LiteLLM in the stack. It's a unified interface for calling LLMs from multiple providers. That means it typically has access to API keys for every AI provider you use, plus whatever environment variables and cloud credentials exist in the same context. Compromising a package in that position gives attackers a shortcut to everything.

Third, the downstream blast radius. LiteLLM isn't just installed directly. It's a transitive dependency pulled in by AI agent frameworks, MCP servers, and LLM orchestration tools. Teams that have never heard of LiteLLM may still have been running it in their CI jobs.

What we keep seeing in our own work

During penetration testing engagements for gaming studios, one of the first things we look at is the CI/CD pipeline. It's consistently the softest target. Teams invest heavily in firewalls, WAFs, and runtime monitoring, but the build pipeline often runs with overly broad permissions, pulls dependencies without pinning, and logs secrets in ways that would make you wince.

We've seen similar patterns when building cloud-native apps for enterprise clients with strict compliance requirements. Swiss regulatory environments, for example, require you to demonstrate control over your software supply chain. That's not just a checkbox exercise. It means knowing exactly what version of every dependency you're running, when it was published, and what checksums match. The teams that already had this discipline in place were the ones least exposed to this attack.

In our native iOS and Android projects, we've historically been a bit more insulated from this kind of thing because the Apple and Google ecosystems have more centralized build tooling. But the pattern is shifting. More mobile teams are integrating AI code generation tools and LLM-based features into their apps, which means Python-based tooling is creeping into build pipelines that never had it before. If you're running any AI/ML preprocessing as part of your mobile build, check your dependency tree.

What you should actually do

Here are concrete steps. Some of these you can do today.

Pin your dependencies. Not just in requirements.txt, but everywhere: Dockerfiles, CI config, GitHub Actions. Use hash-pinning where your package manager supports it. The fact that 40-50% of LiteLLM installs were fetching latest on every run is the root cause of why a 40-minute window affected so many environments.

Check if you were exposed. If you installed or upgraded LiteLLM via pip on March 24 between 10:39 and 16:00 UTC, check for versions 1.82.7 or 1.82.8. Run pip show litellm in every environment. Look for ~/.config/sysmon/sysmon.py and suspicious pods in Kubernetes matching node-setup-*. LiteLLM and several security firms have published scanner scripts. Use them.

Rotate credentials aggressively. If you find any sign of compromise, assume every credential on that machine is burned: SSH keys, cloud tokens, database passwords, API keys in .env files. Simply removing the package is not enough, because the malware was designed to establish persistence and may have already deployed additional payloads.

Use dependency cooldowns. PyPI is shipping "relative dependency cooldowns" in pip v26.1 (expected this month), which lets you configure pip to only install packages that have been published for a minimum time period. This won't protect you from everything, but it would have prevented automatic installation of a package that had been live for 40 minutes. You can already set absolute cooldowns in pip v26.0 with the --uploaded-prior-to flag.

Audit your CI/CD tool permissions. Your vulnerability scanner, your linter, your test runner, they all run with whatever permissions your pipeline has. If your pipeline can push to production, so can a compromised scanner. Apply least-privilege to every step.

The uncomfortable truth about AI tooling and supply chains

Here's what keeps nagging at me about this one. The AI development stack is young, fast-moving, and held together by a relatively small number of open-source packages that everyone depends on. LiteLLM, LangChain, various agent frameworks. These projects move fast, ship often, and are maintained by small teams. That's not a criticism. It's just the reality of where we are.

The 2026 State of DevOps Modernization Report found that nearly a quarter of deployments require remediation, with remediation times averaging over 7.5 hours. Add AI-generated code and AI-managed dependencies to that mix, and you get a supply chain that's expanding faster than most teams can audit.

From our PHP and Docker work with enterprise clients, we know that supply chain discipline is boring. It's unglamorous. Nobody wants to spend a sprint tightening dependency pins and reviewing CI permissions. But the teams that do it are the ones that sleep through incidents like this instead of scrambling to rotate every credential they own.

If any of this sounds uncomfortably familiar, or if you're not sure whether your CI/CD pipeline would survive this kind of attack, let's talk. We do security reviews that specifically cover build pipelines and dependency management, not just the production perimeter, and we bring the development experience to understand what's actually practical for your team to implement.