XZ Utils Supply Chain Attack: A Threat Actor Spent Two Years to Implement a Linux Backdoor

A threat actor quietly spent the last two years integrating themself in the core team of maintainers of XZ Utils, a free software command-line data compressor widely used in Linux systems. The attacker slowly managed to integrate a backdoor in the software that was designed to interfere with SSHD and allow remote code execution via an SSH login certificate. The backdoor was discovered a few days before being released on several Linux systems worldwide.

The threat actor is suspected to be a developer with or using the name Jian Tan. Several security experts believe this supply chain attack might be state sponsored.

What is XZ Utils, and what is the XZ backdoor?

XZ Utils and its underlying library liblzma is a free software tool that implements both XZ and LZMA, which are two compression/decompression algorithms widely used in Unix-based systems, including Linux systems. XZ Utils is used by many operations on those systems for compressing and decompressing data.

The CVE-2024-3094 backdoor found in XZ Utils was implemented to interfere with authentication in SSHD, the OpenSSH server software that handles SSH connections. The backdoor enabled an attacker to execute remote code via an SSH login certificate. Only XZ Utils versions 5.6.0 and 5.6.1 are impacted.

How the XZ backdoor was implemented cautiously for more than years

On March 29, 2024, Microsoft software engineer Andres Freund reported the discovery of the backdoor. He found it when he became interested in odd behavior of a Debian sid installation, such as SSH logins taking a lot of CPU and Valgrind errors and decided to analyze the symptoms in depth. Freund explained that the discovery of the backdoor in XZ was luck, as it “really required a lot of coincidences.”

Yet it appears that the implementation of the backdoor has been a very quiet process that took about two years. In 2021, a developer named Jian Tan, username JiaT75, appeared out of the blue to start working on the XZ Utils code, which is not unusual because developers of free software often work together on updating code. Tan contributed frequently to the XZ project since late 2021, slowly building trust in the community.

In May 2022, an unknown user using the fake name Dennis Ens complained on the XZ mailing list that the software update was not satisfying. Another unknown user, Jigar Kumar, came into the discussion two times to pressure the main developer of XZ Utils, Lasse Collin, to add a maintainer to the project. “Progress will not happen until there is new maintainer,” Jigar Kumar wrote. “Why wait until 5.4.0 to change maintainer? Why delay what your repo needs?”

Meanwhile, Collin expressed that “Jia Tan has helped me off-list with XZ Utils and he might have a bigger role in the future at least with XZ Utils. It’s clear that my resources are too limited (thus the many emails waiting for replies) so something has to change in the long term.” (Collin wrote Jia in his message while other messages reference Jian. To add to the confusion, Jian’s nickname is JiaT75.)

In the months that followed, Tan became increasingly involved in XZ Utils and became co-maintainer of the project. In February 2024, Tan issued commits for versions 5.6.0 and 5.6.1 of XZ Utils, both of which contained the backdoor.

It is also interesting to note that in July 2023, Tan requested to disable ifunc (GNU indirect function) on oss-fuzz, a public tool made to detect software vulnerabilities. That operation was probably done to allow the backdoor in XZ to stay undetected once it was released, as the backdoor makes use of that function to achieve its goals.

Finally, several persons responsible for different Linux distributions have been contacted by the attacker to include the backdoored versions of XZ Utils in their own distributions. Richard WM Jones from RedHat wrote about it on a forum: “Very annoying – the apparent author of the backdoor was in communication with me over several weeks trying to get xz 5.6.x added to Fedora 40 & 41 because of it’s ‘great new features’. We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor he had added). We had to race last night to fix the problem after an inadvertent break of the embargo. He has been part of the xz project for 2 years, adding all sorts of binary test files, and to be honest with this level of sophistication I would be suspicious of even older versions of xz until proven otherwise”. Tan also tried to have it included in Ubuntu.

XZ backdoor: A highly technical attack

In addition to the highly elaborated social engineering covered previously in this article, the backdoor itself is very complex.

Microsoft’s senior threat researcher Thomas Roccia designed and published an infographic to show the whole operation leading to CVE-2024-3094 (Figure A).

Figure A

The backdoor is composed of several parts that have been included over multiple commits on the XZ Utils GitHub, described in depth by Freund.

Gynvael Coldwind, managing director of HexArcana Cybersecurity GmbH,a cybersecurity company providing consulting and courses services, wrote in a detailed analysis of the backdoor that “someone put a lot of effort for this to be pretty innocent looking and decently hidden. From binary test files used to store payload, to file carving, substitution ciphers, and an RC4 variant implemented in AWK all done with just standard command line tools. And all this in 3 stages of execution, and with an ‘extension’ system to future-proof things and not have to change the binary test files again.”

DOWNLOAD: Open source quick glossary from TechRepublic Premium

Martin Zugec, technical solutions director at Bitdefender, said in a statement provided to TechRepublic that “this appears to be a meticulously planned, multi-year attack, possibly backed by a state actor. Considering the massive efforts invested and the low prevalence of vulnerable systems we’re seeing, the threat actors responsible must be extremely unhappy right now that their new weapon was discovered before it could be widely deployed.”

Which operating systems are impacted by the XZ backdoor?

Thanks to Freund’s discovery, the attack was stopped before being spread on a wider scale. The cybersecurity company Tenable exposed the following operating systems known to be affected by the XZ backdoor:

• Fedora Rawhide.
• Fedora 40 Beta.
• Fedora 41.
• Debian testing, unstable and experimental distributions versions 5.5.1alpha-01 to 5.6.1-1.
• openSUSE Tumbleweed.
• openSUSE MicroOS.
• Kali Linux.
• Arch Linux.

In a blog post, Red Hat reported that no versions of Red Hat Enterprise Linux are affected by CVE-2024-3094.

Debian indicated that no stable version of the distribution are affected, and Ubuntu posted that no released versions of Ubuntu were affected.

MacOS homebrew package manager reverted XZ from 5.6.x to 5.4.6, an older yet safe version. Bo Anderson, maintainer and Homebrew technical steering committee member, declared that Homebrew does not “… believe Homebrew’s builds were compromised (the backdoor only applied to deb and rpm builds) but 5.6.x is being treated as no longer trustworthy and as a precaution we are forcing downgrades to 5.4.6.”

How to mitigate and protect from this XZ backdoor threat

More systems might be affected, especially those on which developers compiled the vulnerable versions of XZ. Security company Binarly offers an online detection tool that could be used to test systems to see if they are affected by the XZ backdoor.

The version of XZ should be carefully checked, as versions 5.6.0 and 5.6.1 contain the backdoor. It is advised to revert to a previous known safe version of XZ Utils, such as 5.4.

Software supply chain attacks are increasing

As previously reported on TechRepublic, software supply chain attacks are increasingly being used by threat actors.

Yet usual software supply chain attacks mostly consist of managing to compromise a key account in the process of the development of software, and use the account to push malicious content to legitimate software, which often gets detected quite rapidly. In the XZ Utils case, it is very different because the threat actor carefully managed to gain the trust of legitimate developers and become one of the maintainers of the tool, allowing him to slowly push different vulnerable parts of code into the software without being noticed.

Software supply chain attacks are not the only increasing threats; other supply chain attacks based on IT products are also increasing.

Therefore, companies should ensure that third parties are taken into consideration in their attack surface monitoring.

Disclosure: I work for Trend Micro, but the views expressed in this article are mine.