Log4j Zero-Day Vulnerability Response
Last Updated: January 7, 2022
- Change Log
- Executive Summary
- Immediate Actions
- CIS and MS/EI-ISAC Products and Log4j
- Appendix A: Detecting Vulnerable Software
- Appendix B: Manual Mitigation
- Appendix C: Detecting Exploitation
- Appendix D: Engaging MS-ISAC CIRT
- Additional Actions
|01/07/22||Updates to Executive Summary including recommended actions and FTC Log4Shell remediation warning|
|12/30/21||Added CVE-2021-44832 to Executive Summary and a note on Next-Generation Firewalls to Additional Actions|
|12/23/21||Added CIS and MS/EI-ISAC Products and Log4j and indexed table of contents|
|12/22/21||Added Change Log and updated Playbook graphic based on member feedback|
|12/21/21||Added CIS Products and Log4j Vulnerability link|
|12/20/21||Minor updates to Recommendations|
|12/18/21||Rewrote Executive Overview to include new CVEs and updates related to version 2.17.0|
|12/17/21||Minor updates to Executive Overview and Recommendations; added Yahoo tool|
|12/16/21||Added tools to Appendix A and added new sources|
|12/15/21||Updated content and sources; included note that initial patch for CVE-2021-44228 was incomplete and workarounds are ineffective; included CVE-2021-45056|
|12/14/21||Page created with Exec Summary, Recommendations, Playbook, and sources|
On December 9, 2021, security researchers discovered a flaw in the code of a software library used for logging. The software library, Log4j, is built on a popular coding language, Java, that has widespread use in other software and applications used worldwide. This flaw in Log4j is estimated to be present in over 100 million instances globally. This vulnerability and associated attacks against it are being characterized as Log4Shell in the cybersecurity community.
The flaw, also known as a vulnerability by the security community, was rated a 10 out of 10 on the Common Vulnerability Scoring System, or CVSS, due to the potential impact that it can have if leveraged by attackers. Details of the vulnerability can be found in the National Vulnerability Database (NVD) under the heading CVE-2021-44228.
As of Dec. 14, researchers discovered that the fix developed for CVE-2021-44228 was incomplete and the vendor, Apache, released a new fix. On Dec. 17, two new issues were confirmed and the next day, Apache released another fix. We expect this cycle of vulnerability-fix vulnerability-fix will continue as attackers and researchers continue to focus on Log4j.
To simplify things, the current list of vulnerabilities and recommended fixes is listed here:
- CVE-2021-44228 (CVSS score: 10.0) - A remote code execution vulnerability affecting Log4j versions from 2.0-beta9 to 2.14.1 (Fixed in version 2.15.0)
- CVE-2021-45046 (CVSS score: 9.0) - An information leak and remote code execution vulnerability affecting Log4j versions from 2.0-beta9 to 2.15.0, excluding 2.12.2 (Fixed in version 2.16.0)
- CVE-2021-45105 (CVSS score: 7.5) - A denial-of-service vulnerability affecting Log4j versions from 2.0-beta9 to 2.16.0 (Fixed in version 2.17.0)
- CVE-2021-4104 (CVSS score: 8.1) - An untrusted deserialization flaw affecting Log4j version 1.2 (No fix available; Upgrade to version 2.17.0)
- CVE-2021-44832 (CVSS score: 6.6) - Remote code execution vulnerability affecting Log4j2 versions 2.0-beta7 through 2.17.0, excluding security fixes for 2.3.2 and 2.12.4. (Fixed by upgrading to 2.3.2 (for Java 6), 2.12.4 (for Java 7), or 2.17.1 (for Java 8 and later).
We recommend following the advice of Apache, which recommends updating to the latest version of Log4j immediately.
Please note that all 1.x versions of Log4j are considered “End of Life,” which means they are no longer supported by Apache. While the vulnerabilities (CVEs) listed above are all associated directly with Log4Shell, there are several other known vulnerabilities in 1.x versions of Log4j that can leave users open to a wide range of attacks. It is for this reason that we recommend all Log4j users update to the latest 2.x version available immediately.
When the initial vulnerability was made public, it was described as a zero-day (or 0day), which means it was being targeted and potentially acted upon prior to the software developers knowing that it existed. In other words, the developers had zero days to implement a fix for the vulnerability before it may have been used in an attack.
Due to the widespread prevalence of Log4j, the high impact of an attack against it, and evidence that malicious actors are actively targeting organizations with vulnerable versions of Log4j, CIS is encouraging all organizations to mitigate risk as soon as possible. This page is intended to help all organizations, regardless of technical maturity, to find resources for mitigating risks associated with Log4j.
Security patches are available and CIS encourages all organizations to implement these updates as quickly as possible. Additionally, any organization that relies on outside providers for services that may use Log4j should work with those providers to ensure these third-party relationships do not open them up to undue risk.
It is important to note that simply updating Log4j may not resolve issues if an organization is already compromised. In other words, updating to the newest version of any software will not remove accesses gained by adversaries or additional malicious capabilities dropped in victim environments. Therefore, CIS recommends vigilance in investigating activity in your environment, looking for evidence of unauthorized access, and acting in accordance with incident response best practices to reduce exposure. State, Local, Tribal, and Territorial (SLTT) governments and affiliated organizations, including Elections organizations, may contact the CIS SOC for assistance if you believe you were impacted or compromised.
Threat-focused security organizations have observed state actors begin to leverage the vulnerability in new attacks. CIS recommends heightened vigilance in monitoring networks for abnormal behaviors and invoking immediate response actions as necessary.
The Federal Trade Commission (FTC) recently warned U.S. companies under it's purview to remediate Log4Shell vulnerabilities and that it intends to use the commission's full legal authorities to pursue organizations failing to reduce consumer exposure to these vulnerabilities.
MS-ISAC and EI-ISAC members are encouraged to reach out to the Security Operations Center (SOC) as soon as possible if you suspect you have been impacted by an attack against this vulnerability.
The MS-ISAC is currently working with members who have been impacted. The scope of this vulnerability is more widespread than any other case in recent history. We aim to support all SLTTs and respectfully ask for your patience as we work through this together.
Additionally, we have the ability to scan your external facing infrastructure with up-to-date signatures from leading vulnerability vendors Tenable and Qualys. If you are a member or qualify to join (see here for MS-ISAC, and here for EI-ISAC membership information), please inquire with our SOC about a scan.
The Department of Homeland Security (DHS) Cybersecurity and Infrastructure Security Agency (CISA) will be standing up a single source landing page for guidance, remediation support, and reporting related to this vulnerability.
First and foremost, we encourage all organizations to immediately patch any instances of Log4j to the latest supported version available. Apache continues to post new versions on their security update page. It may be helpful to review Apache’s notes on this page.
If you are unable to fully update products that rely on Log4j due to those products being maintained by a third party, reach out to your third-party contacts as soon as possible to get updated guidance. They may have released updates without notification.
If you or any third-party providers are unable to update software in a timely fashion, CIS recommends uninstalling or disabling Log4j until updates can be applied. This is especially critical for any Internet-facing products using Log4j.
The MS-ISAC created this flow chart for members and others to follow for identifying and mitigating risk related to this vulnerability:
Note: Appendices are available below this graphic.
CIS and MS/EI-ISAC Products and Log4j
The list of technology products impacted by the Log4j vulnerability continues to grow, given the ubiquity of the code.
CIS has made available a list of our products that have been impacted by the vulnerability as well as the mitigations applied by CIS. The steps for users to update to the latest version(s) are being tracked and updated in near real-time. You can access the most up to date information in our knowledge base here.
Separately, our Security Operations Center (SOC) and engineering teams have evaluated the software products offered to MS-ISAC and EI-ISAC members and have determined that the services are unaffected by the Log4j vulnerability, at this point in time. These services include, but are not limited to:
- Albert Network Monitoring
- CIS Endpoint Security Services (ESS) and Endpoint Detection and Response (EDR)
- Malicious Domain Blocking and Reporting (MDBR)
CIS will continue to provide updates on this page and in our WorkBench knowledge base when there is new information to share, or there are recommended actions for our members to take.
Appendix A: Detecting Vulnerable Software
Section 1: Affected Vendors
Section 2: Source Code Scanning
- CERT CC (Carnegie Mellon)
- Grype – Searches libraries installed on a system and displays vulnerabilities present
- Syft – Searches for installed code and libraries and displays their versions
Section 3: Active/Dynamic Testing
- CISA Log4j Scanner
- NMAP Scripting Engine
- CyberReason – (also performs mitigations; however, we recommend exercising caution with automated mitigations)
- Huntress Tester
- Yahoo Check for Log4j (somewhat intrusive and should be executed with care)
- Tenable Nessus Plugin (Note: to run this plugin, an entity must already run Nessus; Tenable is encouraging non-customer organizations to download a community license for Nessus scanner)
Appendix B: Manual Mitigation
NOTE: There is strong evidence that the workarounds are not effective. Therefore, we encourage everyone to update to the latest version from Apache.
According to Section 3 of the Trusted Sec playbook, “If it is not possible to upgrade, there are several mitigation tactics that can be performed.”
For Log4j versions >= 2.10, set the log4j2.formatMsgNoLookups system property to true on both client- and server-side components.
This can be done in multiple ways:
- Add -Dlog4j2.formatMsgNoLookups=true to the startup scripts of Java programs; or
- Set the following environment variable: LOG4J_FORMAT_MSG_NO_LOOKUPS=”true”
For Log4j versions from 2.0-beta9 through 2.10.0, remove the JndiLookup class from the classpath. For example:
- zip -q -d log4j-core-*.jar
Additionally, they recommend the following actions:
- Isolate systems into their own restricted DMZs or VLANs
- Block all outbound network connections from servers or limit outbound network connections to trusted hosts and network ports
- Turn on any endpoint or network security signatures for Log4j exploitation
- Monitor networks and servers closely for suspicious or unexpected behavior
TrustedSec further recommends testing and evaluation “after ALL mitigation steps above have been completed to ensure that vulnerabilities do not persist.”
Appendix C: Detecting Exploitation
Appendix D: Engaging MS-ISAC CIRT
If you require CIRT assistance, please request support through the SOC. The CIRT will reach out and will require information from your organization. Please be prepared to provide as many of the following items as possible:
- A list of known affected systems
- A list of patched systems
- A Kape capture of the known affected system or systems (pre-patched state if possible)
- Download the contents of the Kape tool file from our Sharefile
- Place folder contents into a folder on the affected machine(s)
- Run the Kape executable
- Output will be found in same folder where Kape tool was placed
- Provide output to CIRT analysts via Sharefile folder which will be provided by CIRT once case is acknowledged and a specific Sharefile folder for your incident has been created
- Any relevant logs collected, to include, but not exclusive to:
- Network logs
- Endpoint Detection logs
- IDS/IPS logs
- Suspicious outbound connections
- Apache logs
We recommend the following actions in addition to the immediate actions above:
- If you use an outside IT or cybersecurity provider (e.g., internal SOC, Managed Security Service Provider [MSSP], etc.) ensure they are aware of, monitoring, and taking appropriate actions on any alerts associated with the presence of Log4j in your environment
- Ask them specifically for evidence of outbound traffic as a result of a Log4j exploitation attempt
- Install or modify an existing Web Application Firewall (WAF) with rules that automatically update with the latest information Note: Palo Alto recommends adding a Next-Generation Firewall (e.g., a firewall powered by Machine Learning algorithms) for the best protection against these kinds of attacks. WAFs alone can be noisy and may miss attacks that deviate from the rules, while Next-Gen Firewalls block activity based on deviations from normal or expected behavior.
- CISA will be maintaining a repository of information and affected software here
- Another useful and trusted repository is being maintained by the Dutch National Cyber Security Center
- Other sources of information are listed in the Resources section below; however, CIS cannot vouch for the consistency and reliability of data in all resources shared
- All affected organizations are encouraged to report compromises to CISA and the FBI. For MS-ISAC and EI-ISAC members, we recommend notifying the SOC and seeking assistance through our incident response team, which is a no-cost service available to all SLTT and Elections organizations in the U.S.
- DHS CISA Apache Log4j Vulnerability Guidance
- DHS CISA Github Repository
- Tanium Blog: What is Log4j and How Do I Protect My Organization?
- Tanium Blog: How Tanium Can Help with CVE-2021-44228: Log4Shell
- National Vulnerability Database: CVE-2021-44228
- Apache Log4j Security Update Page
- Microsoft Blog: Guidance for Preventing, Detecting, and Hunting for CVE-2021-44228 Log4j 2 Exploitation
- Cisco Talos Intelligence Group - Threat Advisory: Critical Apache Log4j vulnerability being exploited in the wild
- Palo Alto Networks blog: Apache log4j Vulnerability CVE-2021-4428: Analysis and Mitigations
- Dutch National Cyber Security Center Github Repository
- Swiss Government CERT Blog on Log4j (Good visual)
- CrowdStrike blog: Log4j2 Vulnerability Analysis and Mitigation Recommendations
- TrustedSec: Log4j Detection and Response Playbook
- F5 Labs: Explaining the Widespread Log4j Vulnerability
- Tenable blog: CVE-2021-44228: Proof-of-Concept for Critical Apache Log4j Remote Code Execution Vulnerability Available (Log4Shell)
- Curated Intelligence Trust Group Github Repository
- SwitHak Blue Team Cheat Sheet for Log4j
- Symantec Enterprise blog: Apache Log4j Zero-Day Being Exploited in the Wild content
- Tech Solvency: The Story So Far
- TryHackMe: Solar, exploiting log4j
- Picus Security: Simulating and Preventing CVE-2021-44228 Apache Log4j RCE Exploits
- National Vulnerability Database: CVE-2021-45046
- Carnegie Mellon University CERT Coordination Center
- RecordedFuture: Log4Shell Attacks Expand to State Actors
- Log4Shell Vulnerabilities in VMware Horizon Targeted to Install Web Shells