Finding 0-days with Jackalope

Overview

On March 21^st, 2021, the McAfee Enterprise Advanced Threat Research (ATR) team released several vulnerabilities it discovered in the Netop Vision Pro Education software, a popular schooling software used by more than 9,000 school systems around the world. Netop was very responsive and released several updates to address many of the critical findings, creating a more secure product for our educators and children to use. During any vulnerability research project, as we continue to gain a deeper understanding on how a product works, additional threat vectors become apparent which may lead to additional findings; this proved once again true during the Netop research. In this blog we will highlight an additional finding: CVE-2021-36134, a vulnerability in the processing of JPEG images, on the Netop Vison Pro version 9.7.2 software. The main emphasis will focus on the process and techniques used during blackbox fuzzing of a Windows DLL.

Background

Fuzzing can be a challenging exercise and just knowing where to start can be cause for confusion. There are many different fuzzers on the market, many of them primarily designed to handle open-source projects on Linux. In late 2020 Google’s Project Zero team released a new fuzzer named Jackalope. Jackalope is a coverage-guided fuzzer, meaning it keeps track of code paths during testing and uses that information to guide its future mutations. Jackalope leverages a library called TinyInst for its code coverage and allows for command line parameters related to code coverage to be passed directly to TinyInst. What caught my attention about Jackalope was that it was designed with a blackbox, Windows and MacOS first mentality. It was built to fill a gap in Windows blackbox fuzzing, which has existed for some time and therefore warranted further investigation. During the time of Jackalope’s release, we were working on the Netop Vision Pro research which runs primarily on Windows, so it was logical to test Jackalope to see if we could discover any new vulnerabilities on Netop Vision Pro.

Setup

The Jackalope documentation does a great job of explaining the setup and build process to get started. For this setup we set up shop on a Windows 10 fully patched system and compiled Jackalope from the GitHub repo using Visual Studio 2019. In a short amount of time, it was time to test the setup. The repo provides a test binary which can be built with the source and therefore is the best place to understand how the fuzzer works. There are just under a few hundred lines of code, but how it works can be summed up in just a few lines, as seen in Figure 1.

Figure 1 test.cpp

Examining the test code, it becomes apparent the test binary simply crashes if it finds the word “test” in memory. It causes a crash by attempting to write the value “1” at an invalid memory address, “NULL”.  Therefore, to ensure the fuzzer is working properly we need to create a small input corpus. This can be done by creating an “in” directory and placing a couple of text files within it, one containing the word “test”. We are not looking for crashes or new vulnerabilities during this test, but simply making sure our setup is functioning as expected. The test run can be seen in Figure 2, where the command to execute the test case was taken from the Jackalope documentation.

Figure 2 Testing fuzzer

Target Selection

When selecting an overall target function, it’s first important to look at how an application takes input alongside with how the fuzzer can tailor that input. Jackalope is designed to provide either a file or a chunk of shared memory to the target. This gives a lot of flexibility since almost anything can be set up as shared memory including network packet payloads. The trick becomes how to pass the file or shared memory to the target. In larger applications on Windows, a typical approach is to determine what functionality you want to fuzz, find a DLL that exports a function within the target code path, and pass that function the input. The closer the exported function is to the end of the desired code path to fuzz, the less headaches, and better results you will have trying to exercise the desired code.

Through the research done on Netop, we had a deep insight into how the system functions and the very large number of DLLs that it contains along with the numerous amounts of exported functions. After review, the function MeImgLoadJpeg which is exported in MeImg.dll stuck out as a good place to get started.

Figure 3 MeImgLoadJpeg Header

What makes this a good candidate for fuzzing? First, how, and when this function is executed is important. This function gets executed on both the student and teacher machines whenever a JPEG image is loaded into the system. For students this is when an image is pushed over the network to them; for example: when a teacher uses the blank screen feature on a student. On the teacher’s machine this function is called when a teacher loads an image to send to the student. The key components here are that it is potentially executed often, input can come from a local file or a network file and it affects both components of our system.

Second, when investigating this function further, the parameters are fuzzing friendly. Through light reversing, it can easily be seen that it takes a file path and opens the file directly within the function. This makes fuzzing it with Jackalope very simple since it supports file fuzzing and we won’t need to open or manipulate the test file in memory. Also, very few parameters are passed, one of which (BITMAPINFOHEADER), is well documented by Microsoft, making it simpler to construct valid calls. This also is true for the return parameter, HBITMAP. This will make it easy to determine success and failure conditions. Lastly, the fuzzable component of this function is a JPEG file. JPEG is a well-documented format and a well-fuzzed format, making test corpus generation and potentially crash analysis simpler.

Writing the Test Harness

In most fuzzing setups, a custom program is necessary to setup the required structure and complete any initialization required by the target function. This is commonly referred to as the test harness. It is required any time your target for fuzzing is not the main binary or executable, which tends to be the case most of the time. For example, if you want to fuzz a small executable like the “file” command on Linux, you don’t always require a test harness, since the binary takes its input (a file) directly from the command line and there is little to no setup required to get to your desired state. However, in many cases, especially on Windows, it is common to be looking to fuzz part of the code that is often not as directly accessible or requires setup before it can be passed the fuzzed data. This is where a test harness comes into play. Using the Jackalope “test.cpp” file provided in the GitHub repo, it is easy to see an example of what is needed when writing a test harness. The harness needs to configure the incoming test case as ether a file or shared memory input, set up parameters for the target function, call the function, and, if needed, create a crash to indicate a found crash to Jackalope.

To get started we first must load the DLL which contains our target function. In Windows this is usually performed with a call to “LoadLibrary”. Since our entire purpose is to fuzz a function within this DLL, if it fails to load, we should just exit.

Figure 4 LoadLibrary

Now that the DLL is loaded into memory, we need to obtain the address of our target function. This is commonly done through “GetProcAddress”.

Figure 5 GetProcAddress

The next step, setting up the parameters for the target function, is arguably the most crucial and can be the most difficult step in building a test harness. The best trick to get this right is to find examples of your target function being called in the real application and then mimic this setup in your test harness.  In Netop, this function is only called by one other function. Figure 6 shows a slightly cleaned up version of a portion of the IDA decompilation of the function which calls MeImgLoadJpeg.

Figure 6 IDA Pro Decomplication of call to MeImgLoadJpeg

We can learn some key points from this call that are important to keep consistent if we want to find a useful crash. We know the first parameter (a2) is simply a file path. In our code, we do need to ensure our file path is in the format of a wide-character, since this is the typical format for a Windows file path. The second parameter (v8) is a Windows BITMAPINFOHEADER object. We can see from this code all the members of the BITMAPHEADER object are being set to 0 using a “memset”, except the “biSize”, which is being set to “40”. Since this is the only time this function is called in the Netop application, if we want to find a bug that has a chance to be leveraged through Netop, we need to follow this format. Why the value is set to 40 is less relevant for our purposes within the test harness; however, it may require investigation depending on any crashes found. The same principle holds true for the 3rd and final parameter. We see here it is hardcoded to zero, so we want to do the same. Could we test other values? Of course, but if Netop is hardcoded to zero we would never actually be able to pass anything else outside of our exercise. Using our additional understanding from Figure 6, we put the below code in Figure 7 into our harness.

Figure 7 Target Function Setup

With our parameters configured, we now need to simply call the function we want to fuzz. Where is the fuzzed data? In this case, our fuzzed data will be the jpeg file. The fuzzer will be passing a file path of a mutated jpeg file.

Figure 8 Calling MeImgLoadJpeg

This next step is highly dependent on the target function. If the target function will not fail in a manner that will crash your harness (throw an unhandled exception), then you need to create a crash for a failed test case. This can be seen in Figure 1 test.cpp. In this case, the target function has error handling and we are interested in any case which causes an unhandled exception within our DLL. If the DLL throws an unhandled exception, it will crash our test harness. As a result, we only need to check the return value for our own purposes to confirm things are working properly. This is good for initially testing, but we will want to remove any unnecessary code for our actual fuzz run. A non-null return value means the jpeg image was parsed and null means a handled error occurred, which is uninteresting for our case.

Figure 9 Checking the return value

With all this framework in place, we can run our harness with a valid image and confirm we get the expected result.

Figure 10 Test run

Performance Considerations

Although the above test harness code will successfully execute the target function and fully function within our fuzzer, we can make a few targeted changes to increase performance and results. One of the slowest operations in an application is printing to the screen, and this is true when fuzzing. Error checking is extremely helpful for development; however, printing “Result was not null” or the inverse every time will reduce our executions per second and doesn’t add anything to our fuzzer. Additionally, it is important not to introduce any extra code in our “fuzz” function which could potentially introduce additional code paths. This could cause the mutators to think that it found a new path of interesting code, when in fact it’s only the harness. As a result, you want your “fuzz” function to be the absolute bare minimum required to execute your code and perform other setup actions outside this function.

Selecting a Test Corpus

Now that we have a working test harness, we need to create a test corpus or input files for the fuzzer. The importance of this step for fuzzing cannot be overstated. The test cases produced by a fuzzer or only as good as the ones provided. In most cases, you are looking to create a set of minimal test inputs (and minimal size) that generate maximal code coverage.

Selecting or building a test corpus can be a complicated process. One of the advantages to using a known and popular format like JPEG is that there are many open-source corpora. Strongcourage’s corpus on GitHub is a great repo since it is many corpora combined and is the testing corpus that was used to find CVE-2021-36134. Jackalope does not recursively traverse directories when reading the input directory and does not throw an error in this regard, therefore it is important to make sure your corpus directory is only one level deep.

Results

Using this basic outlined method and running Jackalope in the same fashion as the example test binary, a write access violation crash is found in the MeImgLoadJpeg in just a few minutes. This write access violation bug is filed as CVE-2021-36134.

This violation occurs because of memory being allocated for the destination of a memory copy based on the default three color components of a JPEG image, instead of using the value provided in the input file’s JPEG header. The copy is using the values provided by the file to determine the address of where to copy the image in memory. A write access violation occurs when a malformed image reports having four color components instead of the default three and the memory allocated is not the same size as the actual image.

Given the extensive prior reports and full system vulnerabilities we submitted to NetOp before, we decided not to take the analysis further and determine if the bug was truly exploitable. The code path can be leveraged over the network, by utilizing the teacher’s blank student screen feature. It is worth noting this code runs on both the student and the teacher, so the teacher would be unable to load this image to send to a student without crashing their own system. An attacker could leverage the previously discovered and disclosed vulnerabilities to emulate a teacher and send this image to a student regardless.  Since the destination of the memory copy is being calculated based on image width and number of color components, it is plausible for an attacker to control where the “write” takes place; however, they would need to use an address space that could be calculated without invalidating the image further. In addition, the code is writing pixels from the image which is also under the attacker’s control. As a result, this could lead to a partial arbitrary write.

Conclusion

Fuzzing can be a fantastic tool for discovering new vulnerabilities in software. Although having source code can enhance fuzzing, it should not be considered a barrier of entry. Many tools and techniques exist which can be used to successfully fuzz blackbox targets and in turn help enhance the security of the industry.

One goal of the McAfee Enterprise Advanced Threat Research team is to identify and illuminate a broad spectrum of threats in today’s complex and constantly evolving landscape. Leveraging Google Project Zero team’s Jackalope and blackbox fuzzing techniques a JPEG parsing vulnerability, CVE-2021-36134 was discovered in Netop Vision Pro version 9.7.2. As per McAfee Enterprise’s vulnerability public disclosure policy, the ATR team informed Netop on June 25^th, 2021, and worked directly with the Netop team. This partnership resulted in the vendor working towards effective mitigations of the vulnerability detailed in this blog.