How to Respond to Apache Log4j using Cisco Secure Analytics – Cisco Blogs

IT and Security professionals worldwide are working to assess and mitigate their exposure to Apache Log4j vulnerability (CVE-2021-44228). The following guide has been put together for current Secure Network Analytics and Secure Cloud Analytics customers, providing suggested ways to leverage your deployment to assist in your detection and response efforts. To learn more about Cisco’s response, hear from experts, and understand how you can use other Cisco Secure product, please visit Cisco Secure Alert on Apache Log4j page.

Customers can research any prior interactions with known indicators such as IP addresses tracked by Cisco Talos intelligence group and should create Custom Security Events and Watchlists to identify any future communication with the known indicators. Customers should keep a close eye on any issued detections that would indicate an attack might be underway, since the activity following this exploit can vary greatly. Potentially related detections include Suspected Cryptocurrency Activity, Watchlist Observations, Unusual Geographic Access, Lateral Movement, Data Hoarding, etc.

Vulnerability Description

Apache Log4j is a java-based logging framework library.  The JNDI (Java Naming and Directory Interface) component in Apace Log4j versions 2.0-beta9 through 2.14.1 improperly handles log messages.  Certain user-supplied log messages are improperly executed prior to being written to log files.  Unauthenticated remote attackers can leverage specially crafted LDAP log messages to download and execute arbitrary code with elevated privileges.  Please note that due to the widespread use of this library that other vectors besides LDAP are possible depending on the implementation.

Exploitation

This vulnerability is being exploited in the wild, first detected on December 9, 2021.  Public proof of concept code is also available from multiple sources, which can be easily weaponized.

Monitoring Indicators of Compromise

Cisco Talos has published a series of Indicators of Compromise (IOC’s) including IP addresses of hosts serving malicious payloads in their blog located at: https://blog.talosintelligence.com/2021/12/apache-log4j-rce-vulnerability.html

Please check the Talos blog regularly for updates.

There are several methods to detect evidence of exploitation in your environment using Secure Analytics products.  We’ll start with Secure Network Analytics, followed by the techniques you can use for Secure Cloud Analytics.

Secure Network Analytics:

Search for Past Evidence of Exploitation using Secure Network Analytics – Method One

Users should perform a Flow Search going back at least 7 days for the IP’s provided by Talos. Exploitation was first detected on December 9, 2021, but it is possible that activity was happening prior to this date. Consider searching back further than 7 days.

1. From the Manager’s web UI click on the Analyze menu then select Flow Search.
2. Select “Last 7 Days” for the Time Range.
3. Select “Inside Hosts” as the Subject host group.
4. Enter the Talos IP’s as Peer Host IP Addresses.
5. Click on Search to return any matches between inside hosts and the Talos IP’s.

The Flow Search criteria should look like the following:

Search for Past Evidence of Exploitation using Secure Network Analytics – Method Two

Users should perform a Host Search for the IP’s provided by Talos. The Host Search will report if Secure Network Analytics has ever seen an IP address, and if so, when it was first seen, last seen, the total bytes, and by which Flow Collector.

1. From the Manager’s web UI click on the Analyze menu then select Host Search.
2. Enter the Talos IP’s in the IP Address field.
3. Click on Search to run the Host Search and view the results. Ideally, the report will read “Never” next to each IP for the First Sent and Last Sent columns and “None” for the Total Bytes column.

The Host Search criteria should look like the following:

Detect Future Malicious Communications using Secure Network Analytics

Users should create an outside host group containing the Talos IP’s.  A Custom Security Event (CSE) can be built to look for traffic to this outside host group.  This CSE will fire on future communications to these IP’s.

1. From the Manager’s web UI click on Configure then select Host Group Management.
2. Click on the ellipses (…) to the right of the Outside Hosts host group and select Add Host Group. (Users may wish to nest this new host group under another parent depending on their host group structure.)
3. Enter “Log4j Talos IP Watchlist” (or similar) as the Host Group Name field.
4. Enter the Talos IP’s in the IP Addresses And Ranges field.
5. Click on Save to create the new host group.

The new host group criteria should look like the following:

 

5. To create the CSE click on the Configure menu and select Policy Management.
6. Click on Create New Policy near the top-right of the page and select Custom Security Event.
7. Enter “.CSE: Log4j Talos IP Watchlist Traffic” (or similar) into the Name field.
8. Click on the plus (+) sign under the Find field and create the following criteria:
   1. Subject Host Groups: Inside Hosts
   2. Peer Host Groups: Log4j Talos IP Watchlist
9. Toggle the Status to On.
10. Click on Save to create the CSE, which will then fire any time traffic is seen between inside hosts and the Talos IP’s in our watchlist host group. Take note of the description that is built inside the CSE describing when it will fire: When any host within Inside Hosts communicates with any host within Log4j Talos IP Watchlist, an alarm is raised.

The CSE criteria should look like the following:

 

Detection with the Flow Sensor Payload Data using Secure Network Analytics

Customers with Flow Sensors can search payload data looking for “ldap://” going back at least 7 days in a flow search.  Exploitation was first detected on December 9, 2021, but it is possible that activity was happening prior to this date.  Consider searching back further than 7 days.

1. From the Manager’s web UI click on the Analyze menu then select Flow Search.
2. Select “Last 7 Days” for the Time Range.
3. Select “Inside Hosts” as the Subject host group.
4. Select “Outside Hosts” as the Peer host group.
5. At the bottom-center of the flow search criteria, expand the Advanced Connection Options.
6. In the Payload field in the Advanced Connection Options and enter the following: ldap://
7. Click on Search to return any matches for that payload. Please note that legitimate uses of LDAP will appear depending on your environment’s implementation.  Look for any unusual requests to servers that are not domain controllers or LDAP servers.  You may wish to exclude these hosts in revised Flow Searches.  Alternatively, set both the Subject and Peer host groups to Inside Hosts to look for internal exploitation.

The Flow Search criteria should look like the following:

 

Search for Abnormally Large LDAP Queries using Secure Network Analytics

Users should perform a flow search going back at least 7 days looking for abnormally large LDAP queries between affected servers and outside hosts.  Exploitation was first detected on December 9, 2021, but it is possible that activity was happening prior to this date.  Please note that this vulnerability exists in a library that the attack vectors may vary greatly depending on the implementation.  Customers should adjust flow search criteria to match the ports, protocols, and applications that match their exact implementation.  You may also consider making an Inside Hosts host group of known vulnerable servers and focus on that host group.

1. From the Manager’s web UI click on the Analyze menu then select Flow Search.
2. Select “Last 7 Days” for the Time Range.
3. Select “Inside Hosts” as the Subject host group.
4. Select “Outside Hosts” as the Peer host group.
5. Under the center Connections box, click on Select under Applications. The Applications Selector will appear on the left-side of the page.
6. Either search for or scroll down and select the following applications on the Include tab (this is the default tab):
   1. LDAP
   2. LDAP (unclassified)
   3. LDAPS
   4. LDAPS (unclassified)

The selections should look like the following in the Applications Selector:

 

1. Click on Apply in the bottom-right corner of the Applications Selector to return to the Flow Search.
2. At the bottom-left of the flow search criteria, expand the Advanced Subject Options.
3. In the Subject Bytes field in the Advanced Subject Options enter the following: >100
4. Click on the radio button labeled Client under Orientation at the bottom of the Advanced Subject Options.
5. Click on Search to display any abnormally large LDAP queries from a vulnerable server reaching out to download a malicious payload. Depending on how your environment is configured you may find legitimate large LDAP queries with certain hosts.  You may wish to exclude these hosts in revised Flow Searches.  Alternatively, set both the Subject and Peer host groups to Inside Hosts to look for internal exploitation.

The Flow Search criteria should look like the following:

Detect Future Abnormally Large LDAP Queries using Secure Network Analytics

A Custom Security Event (CSE) can be built to automatically detect abnormally large LDAP queries.  Please note that this vulnerability exists in a library that the attack vectors may vary greatly depending on the implementation.  Customers should adjust flow search criteria to match the ports, protocols, and applications that match their exact implementation.  Users will want to use the refined the search criteria used in the previous Flow Searches and make those criteria match the CSE.  For example, excluding servers which legitimately perform large LDAP queries on a regular basis to avoid generating a lot of noise.  You may also consider making an Inside Hosts host group of known vulnerable servers and focus on that host group.

5. From the Manager’s web UI click on Configure then select Policy Management.
6. Click on Create New Policy near the top-right of the page and select Custom Security Event.
7. Enter “.CSE: Log4j Abnormally Large LDAP Queries” (or similar) into the Name field.
8. Click on the plus (+) sign under the Find field and create the following criteria:
   1. Subject Host Groups: Inside Hosts
   2. Peer Host Groups: Outside Hosts
   3. Subject Applications to Include: LDAP, LDAP (unclassified), LDAPS, LDAPS (unclassified)
   4. Subject Bytes: >100
   5. Subject Orientation: Client
9. Toggle the Status to On.
10. Click on Save to create the CSE, which will then fire any time abnormally large LDAP requests are made from a vulnerable server reaching out to download a malicious payload. Take note of the description that is built inside the CSE describing when it will fire: When any host within Inside Hosts, acting as a client; using any disallowed application; with a total payload of >100 bytes communicates with any host within Outside Hosts, an alarm is raised.

The CSE criteria should look like the following:

Secure Cloud Analytics

Search for Past Evidence of Exploitation using Secure Cloud Analytics

Users should search the Event Viewer going back at least 7 days for the IP’s provided by Talos.  Exploitation was first detected on December 9, 2021, but it is possible that activity was happening prior to this date.  Consider searching back further than 7 days.

1. From the Secure Cloud Analytics portal click on the Investigate menu then select Event Viewer.
2. Make sure the Event Viewer is in inline mode by setting the toggle in the top-right of the screen to inline.
3. Change the Start Date to one week ago.
4. Under the Connected_IP field, click on the blue icon to display the filter conditions. Select the third option which reads “In list.”
5. Paste the Talos IP’s in the field under Connected_IP and then click away from the field to accept the list. The query will immediately start running.

The Event Viewer criteria should look like the following:

Detect Future Malicious Communications using Cisco Secure Cloud Analytics
The Talos IP’s are being regularly updated in our threat intelligence feed. Ensure you have the alerting enabled.

1. From the Secure Cloud Analytics portal click on Settings then select Alerts.

2. From the Alert Priorities page, search for “Talos” in the Alert Type field.

3. Set the priority to High and ensure the Alert is “Enabled”

Your alert list should look like the following:

4. We recommend reviewing all alert types, priorities, and whether they are enabled to ensure the detections can be triggered. Because the post-exploit activity could vary greatly, increasing alert priority and sensitivity for many of the tactics and techniques is highly recommended. Delete “Talos” form the search field to return to the complete list of detections for review.

The list of available detections should look like this:

 

Review Watchlist Observations to ensure any traffic to the Talos IP’s is being investigated in Secure Cloud Analytics

The “Talos Intelligence Watchlist Hit” alert described above is only triggered when a significant amount of traffic is exchanged with the IPs. We suggest you review any interactions with the Talos IPs through our “Watchlist Interaction” observation.

1. From the Secure Cloud Analytics portal click on Monitor then select Observations.
2. Select “Selected Observation” from the left panel
3. Choose “Watchlist Interaction” from the Observation Type Field
4. Set the Time Range to start on December 10^th, when the IPs were first added to the Talos watchlists.

Your observation list should look like this:

5. You can investigate internal devices by clicking the down arrow next to device ID and review details on the Device, Alerts associated with the device, and Observations associated with the device.

6. You can investigate the external IP by clicking the down arrow next to the IP address and pivot to a variety of intelligence resources

Impact

The impact of the vulnerability allows attackers to execute arbitrary code from their hosted payload on a vulnerable server. In the event of successful exploitation, Secure Network Analytics and Secure Cloud Analytics will continue to monitor networks for anomalous and malicious activity.  You will have visibility on attacker actions taken, so be on the lookout for an uptick in suspicious behavior from any affected servers.  For example, Cisco Talos has observed attackers exploiting this vulnerability to deploy cryptominers.

CVSS Scoring

Solution

Apache has released an updated version of Log4j and a workaround to address this vulnerability.  Affected users of Log4j should upgrade to version 2.15.0 or apply the mitigation described in Apache’s advisory located at: https://logging.apache.org/log4j/2.x/security.html

References

---

We’d love to hear what you think. Ask a Question, Comment Below, and Stay Connected with Cisco Secure on social!

Cisco Secure Social Channels

Instagram
Facebook
Twitter
LinkedIn

Share: