DevOps’ Big Challenge: Limiting Risk Without Impacting Velocity

By Asaf Karas, CTO for JFrog Security

Businesses leverage enterprise applications to build a competitive edge and move quickly. These applications need to be built, secured, deployed, and updated on a daily basis in most cases. This is where the power of DevOps comes in to prioritize swift software delivery.

Businesses’ heavy reliance on software applications has spurred increased use of open-source software (OSS) libraries and packages, which can help accelerate software delivery. While the benefits of open-source software are undeniable, its use also comes with challenges, particularly where security is concerned. OSS accounts for roughly 70 to 90 percent of all enterprise software, which is largely why OSS has become a primary target for cyber threats, where malicious actors target vulnerabilities within software supply chains.

This means developers are now every enterprise’s first line of defense against a software supply chain attack. Ideally, DevOps teams need to preemptively shield against high-risk packages before they infiltrate their organization’s infrastructure, which calls for the early-stage analysis of open-source packages. The idea is to obstruct risky or malicious components at the point of request or update so they don’t infiltrate the software ecosystem.

Rising Threat: Software Package Hijacking

Software package hijacking is a rising and significant cybersecurity threat where legitimate software packages are injected with malicious code. Although challenging to execute, the method is highly effective due to the widespread use of these packages, creating a high infection rate. Once a package hijacking incident is identified, package maintainers or public repository administrators work to remove the malicious version and publish a clean one, making the infected version inaccessible.

There are two main types of software package hijacking:

1. External Package Hijacking. Software package hijacking is typically carried out using unauthorized access to maintainers’ and developers’ accounts or by discreetly injecting hidden malicious code into legitimate code contributions to open-source projects. A relatively recent example of this was seen in the PyTorch library, a renowned Python machine-learning framework with more than 180 million downloads.

In December 2022, PyTorch experienced a dependency hijacking attack directly targeting the machine learning (ML) developer community. The attacker successfully acquired PyTorch maintainer credentials and introduced a malicious dependency named “torchtriton” into the project. The malicious package garnered more than 3,000 downloads within just five days.

The payload concealed within torchtriton exfiltrated sensitive information, including Secure Shell (SSH) keys and environment variables, sending them to the attacker’s server.

2. Self Package Hijacking (Protestware). Software package hijacking isn’t limited to external malicious actors; developers and project maintainers themselves sometimes engage in this activity as a form of protest or advocacy for their beliefs. This form of hijacking, often referred to as “protestware,” is a concerning trend because it can be difficult for an organization to recognize it before it’s too late.

Take, for instance, “faker” and “colors,” two npm packages highly favored by Node.js developers. The “colors” package enables developers to apply styles, fonts, and colors to the Node.js console, while “faker” aids in generating data for testing purposes during development.

The same individual authored both packages which became quite popular, rising to millions of weekly downloads. In January 2022, the author intentionally sabotaged the packages to protest against large corporations that didn’t contribute to the open-source community. They sabotaged both packages by injecting an infinite loop into the code, effectively rendering thousands of projects that depended on these packages inoperable. It took two days to detect this malicious modification following the release of the tainted versions.

By making this single alteration to the package code, a multitude of companies were impacted by the malicious code and faced significant disruptions to their products and development workflows.

Defense Mechanism: Curate Before You Code

It’s not surprising that these hijacking methods have become prominent in recent years — up to 96 percent of applications contain at least one open-source component. As developers collaborate on software production, there is one word they should become familiar with when it comes to securing the software development pipeline: curation.

At a high level, the word “curation” is defined as the act of thoughtfully selecting and organizing items — a process typically associated with articles, images, music, and so on. In this case, however, the items being curated are open-source software components in a way that acts as an automated lock to safeguard the gateway of the software pipeline. It entails filtering, tracking, and managing software packages based on preset policies to ensure the use of reliable components across the development lifecycle.

Curating software components streamlines development by guaranteeing the safety, reliability, and current status of packages. The idea is to protect against both known and unknown risks through a comprehensive approach that strengthens the organization’s software supply chain by establishing a trusted source of packages.

As the concept of DevSecOps (development + security + operations) gains prominence, curation serves as the initial defense, preventing package-related risks early in the software development process to improve alignment within the organization and enhance the overall developer experience. Effectively curating software packages within the supply chain provides peace of mind by offering secure building blocks for development teams.

Ultimately, software package hijacking is a growing concern in the cybersecurity landscape as external actors and —in some cases, even developers themselves — attack software packages to execute malicious code. Vigilance through proactive curation of software packages, improved security measures across the software supply chain, and rapid incident response are all essential to safeguarding the integrity of software packages upon which countless developers and organizations rely.

About the Author

Asaf Karas is Chief Technology Officer for JFrog Security. A seasoned security expert, Karas has extensive experience in reverse engineering, device debugging, network forensics, malware analysis, big data, and anomaly detection. Prior to JFrog, Karas served as CTO of Vdoo, which delivered an integrated security platform designed for connected, IoT, and embedded devices. Vdoo was acquired by JFrog in June 2021. Karas also spent several years working with international military organizations. Asaf spent almost 15 years leading security research at the Israeli Defense Forces where he served as a branch leader for over 100 cyber specialists. Asaf can be reached online at (LinkedIn, @JFrogSecurity) and at our company website https://jfrog.com/