Risky Biz News: Supply chain attack in Linuxland

In other news: AT&T confirms 2019 data breach; Canonical switches to manual reviews after flood of scam apps; HP leaves Russia.

Risky Biz News: Supply chain attack in Linuxland

This newsletter is brought to you by Resourcely, the company that can help you manage Terraform securely. You can subscribe to an audio version of this newsletter as a podcast by searching for "Risky Business News" in your podcatcher or subscribing via this RSS feed. On Apple Podcasts:

‎Risky Business News: Risky Biz News: Epic supply chain attack on Linux SSH on Apple Podcasts
‎Show Risky Business News, Ep Risky Biz News: Epic supply chain attack on Linux SSH - 31 Mar 2024

This newsletter goes out three days after this incident came to light, so it's gonna cover what happened using a summary-like tone, focusing on aggregating links, conclusions, and hot-takes. At this point, you either know what happened, or you need a crash course in all that took place over the Easter weekend.


What happened: A backdoor mechanism was discovered in XZ Utils, a library that supports lossless compression. The library is extremely popular and is used with most major Linux distros and with a ton of Linux and macOS apps.

What does the backdoor do: In simple terms, it intercepts SSH RSA key decryption operations, which are redirected to the backdoor code. This allows the attacker to pass special arguments during an SSH auth operation and execute code on remote systems if they use a backdoored XZ Utils version.

What versions are impacted: The backdoor code was found in XZ Utils versions 5.6.0 and 5.6.1. The backdoor was live for about a month before detection.

What is the downstream impact: Right now, it was confirmed that the backdoored XZ Utils versions were included in Fedora (40 Beta and Rawhide), Debian (Sid), openSUSE  (Tumbleweed and MicroOS), Kali Linux (Rolling), Gentoo, and Arch Linux (Rolling). Other distros have not confirmed they were infected, but this Repology page says the backdoored versions made it into a whole lot of other distros, such as Alpine Linux, Gentoo, Slackware, PCLinuxOS, and more. Most of the affected distros were dev/unstable versions, so the code didn't make it into many production systems—unless some crazy developers run production systems on unstable distros. The macOS Homebrew package manager and the OpenWRT router firmware also included a backdoored XZ Utils version.

Who discovered the backdoor: Andres Freund, a software engineer at Microsoft and a developer for the PostgresSQL project. Here is Freund's report—the email he sent on the Linux security mailing list.

How did Freund discover the backdoor: Freund noticed an uptick in CPU usage by the SSH daemon, causing delays in SSH performance and login times. While he investigated the performance issues, he discovered malicious code in XZ Utils.

Mastodon post from Kevin Beaumont

Who is the attacker: The main culprit behind the backdoor has been identified as a developer going by the pseudonym of Jia Tan or JiaT75. Software engineer Evan Boehs has studied the attacker's past commits and published a timeline of Jia Tan's actions, including commits to other open-source projects, such as libarchive (used in Windows 👀).

What's so special about the attack: As Boehs points out, all of this started in April 2022. The attacker contributed code to XZ Utils, built his reputation and trust, and used sockpuppet accounts to pressure the project's author (Lasse Collin) to add them as one of the project's maintainers. Collin was burnt out and dealing with mental health issues at the time and welcomed the help since they were a single developer on a highly popular project.

Mailing list entry from Lasse Collin
Mastodon post from Dominic White

But this wasn't it. Besides sockpuppet accounts to pressure Collin, Jia Tan worked hard for his backdoor in other places as well. He apparently made a pull request on Google's OSS-Fuzz project to blind it to its backdoor, and pressured Fedora and Ubuntu devs to update to the backdoored versions.

HackerNews post from one of the Fedora developers

How did nobody else spot the backdoor: While Jia Tan added some code to the XZ Utils project, the bulk was never added to the project's GitHub repo. The actual backdoor resided in the XZ Utils tarballs. These are TAR archives that are auto-generated whenever developers released a new version of the XZ Utils library. Third-party software projects don't (usually) pull the source code and compile the whole XZ Utils project. They just pull the tarball. Jia Tan modified lines in the tarball configs to load the backdoor, which was hidden in binary test data files. Developers usually audit the source code, (erroneously) thinking that the tarballs perfectly reflect the code. In reality, you can tamper with tarballs and other release archive files quite easily—and other threat actors have done so in the past. Below is a graph of the backdoor's multi-part components by Thomas Roccia.

The multiple parts of the XZ backdoor

What's the connection between compression and SSH: The actual backdoor resides in the liblzma component of XZ Utils. Liblzma is used by systemd. Some Linux distros use modified versions of SSH to support systemd notifications. The quote is from a Sam James analysis linked below.

"We know that the combination of systemd and patched openssh are vulnerable but pending further analysis of the payload, we cannot be certain that other configurations aren't."

Attribution: None so far, but the way this backdoor works is weird and intriguingly complex, showing this is not the work of an amateur. As Michał Zalewski points out:

"If this timeline is correct, it’s not the modus operandi of a hobbyist. In today’s world, if you have the technical chops and the patience to pull this off, you can easily land a job that would set you for life without risking any prison time. It's true that we also have some brilliant folks with sociopathic tendencies and poor impulse control — but almost by definition, such “black hat” groups seek instant gratification and don’t plan heists years in advance. In other words, all signs point to this being a professional, for-pay operation — and it wouldn’t be surprising if it was paid for by a foreign government."

Zalewski also has what he described as a "zinger closing paragraph" where he suggests that this may not be an accidental discovery at all.

"In fact, here’s an interesting thought: perhaps they have known for a while. Would we be able to tell the difference between a carefully-timed disclosure — presumably engineered to conceal “methods and sources” — and a serendipitous discovery?"

Technical write-ups: Besides Andres Freund's original report, others have also taken a look at the backdoor. Check technical write-ups from Jonathan Schleifer MidarSam James, SMX, and Gynvael Coldwind (quoted below).

"Someone put a lot of effort for this to be pretty innocent looking and decently hidden. From binary test files used to store payload, to file carving, substitution ciphers, and an RC4 variant implemented in AWK all done with just standard command line tools. And all this in 3 stages of execution, and with an "extension" system to future-proof things and not have to change the binary test files again."

Developer response: Lasse Collin has confirmed the incident here. The XZ Utils project was down at the time of this newsletter because it was primarily hosted via GitHub. GitHub pulled the project for security reasons to prevent its code and tarballs from being pulled into other projects. It also suspended Collin and all maintainer accounts, probably not knowing if they were compromised or not. Better to be safe than sorry.

Other responses: CISA has published an advisory.

IOCs and tooling: Right now, there are no IOCs since this is a gated exploit. However, here's a honeypot that stands up hosts with vulnerable SSH & XZ versions, ready to be hackermanned! There's also a Shell script to check if individual systems are impacted.

Fallout: Some are calling for the removal of auto-generated tarballs and other archive files from project releases.

Does this have a CVE: Yes. CVE-2024-3094, with a CVSSv3 security score of 10/10.

Memes, jokes, and hot takes: Because we all love clever memes and trolling.

via Joe Cooper
via Marius Genheimer
via r000t
via Mainframed767
via @jordantwittt
via vx-underground
via qwertyoruiopz
via Mike Sheward
voa Pancake
via Ian Campbell

Breaches, hacks, and security incidents

AT&T confirms 2019 breach: AT&T has acknowledged that hackers have stolen the information of 73 million customers. The company says the data originates from a 2019 security breach. AT&T says the data contains information on 7.6 million current customers and 65.4 million former users. The company said it found no evidence of a breach of its systems, suggesting the incident might have impacted a third party. AT&T confirmed the breach after a threat actor began selling the data on a hacking forum in mid-March. It repeatedly denied the authenticity of the data until the Friday before the Easter holiday. AT&T is currently resetting user account passcodes. [Additional coverage in TechCrunch]

ONR cyberattack investigation: The British nuclear energy watchdog has started legal proceedings against the Sellafield nuclear material processing site. The UK Office for Nuclear Regulation (ONR) has charged the site's operators with cybersecurity violations between 2019 and 2023. The Sellafield plant suffered a ransomware attack in December of last year.

MOVEit hacks SEC inquiry: Progress Software says it's still cooperating with the SEC on an investigation into the hacks of its MOVEit servers last year. The company's CEO claims the SEC's inquiry didn't suggest the company was at fault. [Additional coverage in CybersecurityDive]

Hot Topic cred stuffing: Clothing store retail chain Hot Topic says its online store rewards program was the victim of a credential stuffing attack in November of last year.

Claro cyber-attack: The Trigona ransomware gang has (finally) taken credit for the cyberattack on Claro, one of the largest telcos from Central and South America.

Prisma Finance crypto-heist: A threat actor has stolen $11.6 million worth of assets from cryptocurrency platform Prisma Finance. The attacker claimed to be a security researcher and described the hack as a "whitehat rescue." Unlike in similar cases, the attacker has dragged out negotiations and has not returned the stolen funds. [Additional coverage in CoinTelegraph]

General tech and privacy

RCS coming to the iPhone: Google has added (and then removed) a section on the Android website claiming that RCS was coming to iPhone devices in (the Northern) fall of 2024. [Additional coverage in 9to5Google]

Signal cloud backups: Secure instant messaging service Signal is working on a new feature that will allow users to create cloud backups and migrate chats and media between devices. According to reports, this may be a dual free-paid feature.

New Telegram privacy feature: Instant messaging service Telegram has rolled out a new privacy feature that allows users to restrict who can message them. The feature will first roll out to users in Russia, Belarus, and Ukraine. Telegram rolled out the feature after Russian users reported receiving messages from strangers calling for new terrorist attacks.

HP leaves Russia: HP has ceased all operations and has exited the Russian market. The company stopped all shipments to Russia in February 2022 after its invasion of Ukraine. It began winding down operations in May of the same year and had planned for a final exit in May 2024. HP pulled out of Russia last week, two months ahead of its planned departure. The move has surprised Russian companies, which cannot update drivers or contact support.

Ad-blocker usage: Ghostery has published an interesting study on who typically uses ad-blockers. Glad to see the infosec field is at 76%. If you know, you know!

Graph showing statistics of ad-blocker usage from the Ghostery report

Government, politics, and policy

German E2EE draft bill: The Federal Ministry for Digital and Transport Affairs (BMDV) has submitted a draft bill to the government that aims to make end-to-end encryption (E2EE) mandatory for messengers and cloud services. It's just a draft bill. Don't get too excited!

NVD consortium plan gets criticized: NIST's plan to create a larger consortium to manage the NVD instead of its beleaguered staff is getting pushback from the industry because it's taking too much to pull together, leaving the US' vulnerability database increasingly out of date and behind the curve. [New coverage in CyberScoop and our original coverage on the topic]

ESG leaders remain anonymous: The US government has refused to identify the leader of its joint election security task force. Officials say they want to shield its task force from threats and harassment. The task force is named the Election Security Group (ESG) and operates under US Cyber Command and the National Security Agency. [Additional coverage in The Record]

DOD industrial base cybersecurity strategy: The US Department of Defense has published its first strategy for improving cybersecurity practices across the US defense industrial base.

Chart showing the major components of the DOD cyber strategy document

In this Risky Business News sponsor interview, Catalin Cimpanu talks with Resourcely co-founder and CEO Travis McPeak about how the DevOps ecosystem has evolved and ushered the need for DevSecOps and how the company provides and manages its secure-by-default templates.

‎Risky Business News: Sponsored: Resourcely on how it manages its secure templates on Apple Podcasts
‎Show Risky Business News, Ep Sponsored: Resourcely on how it manages its secure templates - 31 Mar 2024

Cybercrime and threat intel

Threat/trend reports: CognyteDr.WebFS-ISACIntel471PatchStack+SucuriS2W TalonSurfShark, and Zscaler have recently published reports covering infosec industry threats and trends. The dual PatchStack and Sucuri report is interesting because it looks at abandomed WordPress themes and plugins. Researchers say they spotted 827 abandoned WordPress themes and plugins throughout 2023. The WordPress team removed 481 of these, as they contained vulnerabilities, but not the rest.

New npm malware: Eight malicious npm packages were discovered last week. Check out GitHub's security advisory portal for additional details.

Indians repatriated from Cambodia scam centers: The Indian government says it worked with Cambodian authorities to rescue and repatriate Indian nationals trapped and forced to work in cybercrime compounds. Officials say they repatriated 75 nationals over the past three months, bringing the total to 250. Cambodia joins Myanmar, Indonesia, and the Philippines in cracking down on cybercrime compounds, freeing, and repatriating trapped workers.

PyPI attack: Last week, we reported about a malware submission wave that hit PyPI, forcing the portal to disable new user account creation and package uploads. CheckmarxCheck Point, and Phylum have now published IOCs from the incident.

SapphireStealer campaign targeting Russians: Russian security firm FACCT has published details on a phishing campaign that used lures related to the recent Russian presidential elections to infect victims with the SapphireStealer malware. The code for this stealer has been available online since March 2022, per the company.

Infostealer campaigns targeting macOS: Security firm Jamf has published a report on two infostealer campaigns (Atomic Stealer & Meethub) targeting macOS users.

Canonical switches to manual reviews: Canonical has switched to doing manual reviews for all apps submitted to the Ubuntu OS app store. The company changed its procedures after multiple publishers attempted to upload malicious crypto-wallet applications to the store over the past few weeks. Canonical intends to create a separate app submission policy for cryptocurrency wallets going forward. It previously rolled out verified accounts as a way for users to spot impersonators.

Malware technical reports

IDAT Loader: Rapid7 has published a report on the IDAT Loader malware, and a recent campaign that delivers the Brute Ratel framework and a version of the SecTop RAT.

Chart showing the infection chain of IDAT Loader

Setting up infrastructure is complex, and improper configuration is a bane to developers, security teams, and DevOps/platform teams everywhere.

❌ Wasted development time
😢 Platform teams stuck in ops mode, support developers
🔓 Unnecessary incidents

See why your team is wasting thousands of hours and millions of dollars, and how Resourcely can help you avoid misconfiguration.

APTs, cyber-espionage, and info-ops

MuddyWater: Malwation says it's tracking MuddyWater APT campaigns targeting Africa, Israel, and Turkiye.

DinodasRAT: Check Point has published a report on DinodasRAT, a backdoor also known as XDealer, used in Operation Jacana, a cyber-espionage campaign that targeted the Guyana government. ESET previously attributed the attacks to a Chinese threat actor. Kaspersky also published a report on the same malware last week.

Cloud Werewolf: Russian security firm BI.ZONE has discovered a new campaign from Cloud Werewolf, a state-sponsored group attacking Russian and Belarusian government organizations.

BI.ZONE taxonomy: Per this January 2024 article spotted by Oleg Shakirov, this is the threat actor naming scheme (taxonomy) used by Russian security firm BI.ZONE.

  • Werewolves - state-sponsored groups;
  • Wolves - financially motivated groups;
  • Hyenas - hacktivist groups.

New ACTINIUM infrastructure: Embee Research used passive DNS data to discover new server infrastructure associated with ACTINIUM, an APT group more commonly known as Gamaredon.

Bellingcat campaign: Security researcher IntelCorgi has published an analysis of the malspam campaign that targeted Bellingcat journalists at the end of last year.

"I agree with the assessment made by the Cluster25 (and other threat researchers) that this campaign is attributable to an undefined Russia-nexus threat actor. The assessment is made with medium confidence based on the consistent and precise targeting of organizations which have spoken out against Russia or regularly report on malicious Russian activity."

Russia's EU propaganda network: Ukrainian news outlet InformNapalm has tracked down and plotted the connections of a vast network of pro-Kremlin propaganda news sites active in the EU. We'll just drop here the outlet's closing paragraph.

"Experts and EU states should monitor the activities of such sites, and readers should question such content because efforts that favor Russia in the information field in times of geopolitical confrontation play into the hands of the Putin regime and are harmful to Europe. After all, Russia has demonstrated that it is waging not only a war on the battlefield to destroy Ukraine, but also an information war to break up the United Europe."
Huge chart showing the connections between hundreds of Russian propaganda sites

Vulnerabilities, security research, and bug bounty

TeamCity security update: JetBrains has published a security update for the TeamCity server. The update fixes 26 issues. JetBrains has published details on only seven of the 26. [h/t Simon Tsui]

Fortra security update: Fortra has released a security update for its Robot Schedule Enterprise Agent for Windows.

Splunk security update: The Splunk team has released security updates to patch four vulnerabilities.

Unconfirmed Signal zero-day: There's some discussions online about a potential zero-day zero-click exploit in the Signal VoIP stack.

SCCM exploitation technique: GuidePoint Security has published new research on an undocumented way to compromise an account and elevate privileges inside an SCCM (System Center Configuration Manager) – aka Microsoft Endpoint Configuration Manager (MECM) – network.

Bref security audit: Security firm Shielder has published a security audit of Bref, an open-source project that helps you go serverless on AWS with PHP. The audit found five security flaws. Four were patched.

Infosec industry

New tool—Vulnerability Lookup: CIRCL (Computer Incident Response Center Luxembourg) has launched a new tool named Vulnerability Lookup. The tool aggregates vulnerability feeds and allows users to sort them by update time.

New tool—Cleanowners: GitHub's dev team has released a new tool, a GitHub action named Cleanowners that suggests which non-organization members should be removed from CODEOWNERS files.

New tool—Ruzzy: Security firm Trail of Bits has open-sourced a tool named Ruzzy, a fuzzer for pure Ruby code and Ruby C extensions.

New tool—CcmPwn: Google's Mandiant division has open-sourced a new tool named CcmPwn that can hijack Windows sessions via CcmExec, a service native to SCCM Windows client.

JSAC2024 slides: Slides JSAC2024 security conference, which took place in January, are available on JPCERT's site.

RIP Ross Anderson: Ross Anderson, a widely-recognized professor of security engineering from the University of Cambridge, has passed.

Risky Business Podcasts

Listen to Resourcely CEO Travis McPeak talk with Risky Business Snake Oilers host Patrick Gray about the company's automagic Terraform cloud-provisioning technology.

‎Risky Business: Snake Oilers: Resourcely, Panther and Island on Apple Podcasts
‎Show Risky Business, Ep Snake Oilers: Resourcely, Panther and Island - 3 May 2023