Micropatching Remote Code Execution in WebEx Browser Extension (CVE-2017-3823)

This is a short story of writing a micropatch for a widely-used web browser extension.

[Updated on 1/27/17 after Cisco has, again commendably quickly, issued a new WebEx browser extension version to fix a bypass in version 1.0.5.]

Two days ago, Tavis Ormandy of Google's Project Zero published details of a remote code execution bug he had found in Cisco WebEx Browser Extension that allowed any web page to launch arbitrary executable on visitor's computer. A thing like this is a big deal: WebEx has ~20 million users and many of them are enterprise and government users, or in other words, interesting targets for professional criminals and spies.

According to Tavis, Cisco was impressively quick to issue an updated version 1.0.5 after receiving his report. The automatically installed update restricts the affected functionality so that it can only be used by web sites originating via HTTPS from domains webex.com and webex.com.cn. For all other web sites, an attempt to exploit the vulnerability would result in a warning dialog like this:

I tried to get Tavis's exploit to work with the updated extension, but clicking OK on the above dialog didn't work. Delivering the exploit from a fake local www.webex.com server didn't - as expected - show the above warning, but also didn't result in the exploit working. My theory is that Cisco also added some quick check to break this particular exploit, which was a smart thing to do, since cross-site scripting issues are surely about to start springing up on webex.com servers (one is already there). If my theory is correct, someone will find (and possibly publish) a way to get around this quick check and combine it with a cross-site scripting issue somewhere in the webex.com domain. By the way, Pentest-Tools.com finds 544 hosts in the webex.com domain, so the attack surface seems sizeable.

The show is not over either, it seems, as Tavis has apparently found and reported some residual remote code execution issues with the latest WebEx update.

[Update 1/27/17: Tavis's report is now public. Apparently WebEx developers have implemented the DLL white-listing strategy in version 1.0.5 that we mentioned in the original blog post, but Tavis has found a way around it by misusing the installation functionality for replacing one of the white-listed DLLs with Cisco-signed MSVCR100.DLL. Cisco has since published WebEx update 1.0.7 that fixes this bypass.]

So we have a publicly known remote code execution bug with a working POC on millions of computers, an unknown number of not-yet-updated WebEx extensions, an update that seems to be exploitable with cross-site scripting, and a still-secret residual remote code execution bug in the latest update. One could say there's room for improvement.


Writing a Micropatch

The way this WebEx vulnerability is currently fixed doesn't seem good enough to me. (To be fair, it probably also doesn't look good enough to Cisco but they had to do something quick and are probably working on a better long-term solution.) The risk of resurrecting Tavis's exploit with some modification and cross-site scripting, or of simply tricking users to click OK, made me wonder if we could do a better fix with 0patch.

My initial thought was that we couldn't, as browser extensions are essentially JavaScript code and we currently can't patch that. But then my man Edwin van Andel of Zerocopter kind of dared me to 0patch this bug so I dropped everything else and fired up my 0patch virtual machines. Surprisingly, it turned out that most of WebEx is actually native code - the browser code is merely in charge of launching ciscowebexstart.exe and passing the meeting parameters to it. It is this executable that then does the heavy lifting.

Taviso's exploit shows that a WebEx message, which is used for launching a meeting, can also be used for executing any exported function with arbitrary arguments from any DLL in the WebEx folder. We don't know, perhaps even from anywhere on the computer or even from a remote network share.

So what if we patched WebEx so that it would only be possible to call a limited set of functions from a limited set of DLLs, namely those that are needed for WebEx's normal operations. Looking at what happens when a meeting is launched, my debugger and I could only find calls to functions from atmccli.DLL.Well, atmccli.DLL only exports four functions, and they all seem to be in use so why don't we simply limit loading of DLLs to atmccli.DLL?

The challenge at this point was to find the place in the code where the DLL name provided in the (potentially malicious) WebEx message is used for actually loading a DLL, likely using LoadLibrary/Ex. If we found that place, we could perhaps replace the DLL name with "atmccli.DLL".

I started by monitoring ciscowebexstart.exe with Process Monitor to see the call stack where Tavis's calc.exe gets launched. This is what I found.

Apparently, it is atgpcext.dll that calls wsystem from MSVCR100.DLL. So I opened it up in IDA and looked around the addresses in the above call stack. Fortunately, a lot of logging strings are sprinkled around the DLL, making it easier to understand what the code is supposed to be doing.

Subsequent hours comprised of typical connect-the-dots reversing steps, switching from IDA to WinDbg to Process Monitor and back in attempting to understand the process from point A (web site sending WebEx a message) to point B (the chosen DLL function gets invoked).

Eventually the dots were connected and I ended up with a decent patch candidate:

MODULE_PATH "C:\ProgramData\WebEx\WebEx\T30_MC\atgpcext.dll"
PATCH_ID 10000
PATCH_FORMAT_VER 2
VULN_ID 2099
PLATFORM win32

patchlet_start
PATCHLET_ID 1
PATCHLET_TYPE 2

PATCHLET_OFFSET 0x5da9f
JUMPOVERBYTES 5 ; We jump over the original "call sub_21CBA"
N_ORIGINALBYTES 5

code_start

    call over
    dw __utf16__("atmccli.dll"),0
over:
    pop eax

code_end
patchlet_end

Let's see what this patch does. It gets applied to WebEx's atgpcext.dll at offset 0x5da9f, and replaces the call to function which returns a pointer to the DLL name from the message with a pointer to our own fixed string, "atmccli.dll". This replacing is caused by JUMPOVERBYTES 5, which tells 0patch agent not to continue execution of the original code after returning from the patch code, but to jump over (well, obviously) 5 bytes of that code, effectively landing just behind the "call sub_21CBA" 5-byte instruction.

But wait, what is this trickery in the patch code? Do we really make a call to some label, then pop eax from the stack? Yes we do. This is an efficient, and very hacker-worthy solution for getting a pointer to a string if you don't have some extra space to store said string. The call namely pushes the address of the "following instruction" to stack, but what follows is not an instruction but the string. The pop then takes the address of the string from stack, and that's all the patch needs to do. (Note that this is an old trick; if you have any references to its past use, do let me know so I can credit the author.)

Now, the result of this patch getting applied is that whichever DLL one instructs WebEx to use, WebEx will always use its own atmccli.DLL. This obviously breaks Tavis's exploit, as he is trying to load MSVCR100.DLL, and we replace that with atmccli.DLL; there is no _wsystem exported function in atmccli.DLL, so the operation silently fails.

Let's take a look at a video showing how this micropatch blocks the exploit while keeping WebEx functional.

So How Good Is This Micropatch?

Generally speaking, there can be two problems with a security patch: it can fail to adequately remove the security issue, and/or it can cause functional problems.

1. Adequate removal of the security issue: Frankly, it's hard to say without digging deep into what the four exported functions of atmccli.DLL are capable of. Perhaps one could find that InitControl provides a way to launch arbitrary executables, which would make the patch ineffective. At this point, only WebEx developers know the answer to this, which just underlines my belief that 0patches like these should best be written by original developers - and someday hopefully will be.
   
   In comparison to Cisco's official fix (update 1.0.5), this patch prevents the exploit even if it originated from one of the webex.com servers, e.g., via cross-site scripting.
   
   [Update 1/27/17: After the report about exploiting the WebEx version 1.0.5 was made public, we reproduced the second exploit and discovered that our original micropatch - the one for version 1.0.1 - also blocked said exploit without any modification, which we think is kind of cool. This video shows it best.]
   
   
2. Functional problems: In an attempt to simulate a vendor's quick response to a published vulnerability in their product, we simply tested the main WebEx functionality: launching and running a meeting. This seemed to work, but again, WebEx developers would quickly notice if this patch broke some functionality as they are deeply familiar with its working. If they said, for instance, that a few other DLLs are also invoked under normal WebEx operations, we could improve the patch to white-list DLL-function pairs, and still keep things tiny.

The entire process of creating this patch took about 8 hours, including finding and setting up the vulnerable extension, getting the exploit to work, reversing WebEx, trying out a couple of inadequate patch candidates, and wrestling with our not-quite-yet-production-quality 0patch Builder that we use for compiling patches.

It is my assessment that with some help of WebEx developers this micropatch - or likely a much better one - could be made in two hours tops. Their knowledge would significantly shorten the reversing time and would allow for finding the optimal patching strategy that would reliably close the hole without impacting functionality.

And this is how we see the future of vulnerability patching.


@mkolsek
@0patch