Saturday, August 30, 2008

Fixing attachment download with Active Sync / Air Sync in Windows Mobile 6.1 and Exchange Server

I recently bought myself a new smartphone - an XDA Diamond - which runs Windows Mobile 6.1.

After setting up Outlook Mobile to sync with an Exchange server I noticed that attachments in email would not download to the device. I searched google because I thought there must be many people having this problem but actually found only two of them.

I thought that there must be a difference in the way Active Sync works between Windows Mobile 6.0 and 6.1 because syncing with my old MDA Varion III worked without any problems. So I analyzed the Active Sync protocol and found out that there actually is a difference regarding the way URLs are used.

Let me show you what I mean.

This is a request from my MDA Vario III as recorded in the HTTP server log:


2008-07-19 13:44:11 W3SVC1 EMELIE 192.168.0.100 POST
/Microsoft-Server-ActiveSync User=yyy&DeviceId=yyy
&DeviceType=PocketPC&Cmd=GetAttachment&
AttachmentName=Inbox/attachment%20name.EML
/1_multipart_xF8FF_2_NL_0808.gif&Log=V4TNASNC:0A0C0D0FS:
0A0C0D0SP:1C2I739S232418R0S0L0H0P 443 HOME\yyy
80.187.122.111 HTTP/1.1 MSFT-PPC/5.1.2200 - -
emelie.xxx.com 200 0 0 176995 546 1312

And this is the same request from my XDA Diamond:

2008-07-21 21:24:24 W3SVC1 EMELIE 192.168.0.100 POST
/Microsoft-Server-ActiveSync User=yyy&DeviceId=yyy
&DeviceType=PocketPC&Cmd=GetAttachment&
amp;AttachmentName=Inbox%2Fattachment%20name.EML
%2F1_multipart_xF8FF_2_NL_0808.gif&Log=V4TNASNC:0A0C0D0FS:
0A0C0D0SP:3C2I739S1902R0S0L0H0P 443 HOME\yyy
91.89.177.111 HTTP/1.1 MSFT-PPC/5.2.1301 - -
emelie.xxx.com 500 0 0 278 539 15

The difference is that the request URL sent from the WM6.1 device contains URL-encoded URLs. The slash (/) is encoded as %2F. The log entry also shows that the HTTP status is 500 (Internal Server Error) for the URL-encoded request.

So what's the problem? I don't see why it should not be legal to URL-encode the request like it is done in WM6.1. If it were Microsoft would probably not have implemented it this way:) and a lot of people would have the same problem
So, I figured it must be something special on the IIS (the MS HTTP server) that causes this. This was confirmed by this post. It looked like the Apache Jakarta ISAPI Redirector filter (JK) would cause the problem. So I removed it from the Web site and suddenly I was able to download attachments with WM6.1!

This was just a workaround of course since the ISAPI filter was installed on purpose. So I needed to find out what was causing the problem inside JK. The first thing I tried was looking for a newer version of JK. I downloaded version 1.2.26 but this did not improve things. After searching around a bit I dsicovered that there was a second Apache project called JK2 (version 2). I got that one and tried to configure it without any success. Later, I realized that the JK2 project is no longer being developed and JK (the first version) is still being developed. So using JK2 was not an option for me.

Back to JK, I turned on tracing for the ISAPI filter and noticed the following entry in the trace file for the Active Sync request responsible for the downloading of the attachement:

[Fri Aug 29 23:59:33.326 2008] [1004:4836] [debug]
jk_isapi_plugin.c (1199): Filter started
[Fri Aug 29 23:59:33.326 2008] [1004:4836] [emerg]
jk_isapi_plugin.c (1234): [/exchange/user@home.xxx.com
/Inbox/attachment name.EML/1_multipart_xF8FF_2_image.jpg]
contains forbidden escape sequences.
So it turned out that JK actually does not like the way the HTTP request is encoded.
In order to fix this I downloaded the source code for JK and searched for the place where this message is written to the log. It is inside the callback method that is being called on the ISAPI filter for each request:

DWORD WINAPI HttpFilterProc(PHTTP_FILTER_CONTEXT pfc,
DWORD dwNotificationType, LPVOID pvNotification)
{
...
rc = unescape_url(uri);
if (rc == BAD_REQUEST) {
jk_log(logger, JK_LOG_ERROR,
"[%s] contains one or more invalid escape sequences.",
uri);
write_error_response(pfc, "400 Bad Request",
HTML_ERROR_400);
return SF_STATUS_REQ_FINISHED;
}
else if (rc == BAD_PATH) {
jk_log(logger, JK_LOG_EMERG,
"[%s] contains forbidden escape sequences.",
uri);
write_error_response(pfc, "404 Not Found",
HTML_ERROR_404);
return SF_STATUS_REQ_FINISHED;
}
...
}


As you can see this happens when the function unescape_url returns BAD_PATH which happens when the decoded character is a slash:

static int unescape_url(char *url)
{
register int x, y, badesc, badpath;

badesc = 0;
badpath = 0;
for (x = 0, y = 0; url[y]; ++x, ++y) {
if (url[y] != '%')
url[x] = url[y];
else {
if (!isxdigit(url[y + 1]) || !isxdigit(url[y + 2])) {
badesc = 1;
url[x] = '%';
}
else {
url[x] = x2c(&url[y + 1]);
y += 2;
if (url[x] == '/' || url[x] == '\0')
badpath = 1;
}
}
}
url[x] = '\0';
if (badesc)
return BAD_REQUEST;
else if (badpath)
    return BAD_PATH;
else
return 0;
}

static char x2c(const char *what)
{
register char digit;

digit =
((what[0] >= 'A') ? ((what[0] & 0xdf) - 'A') + 10 : (what[0] - '0'));
digit *= 16;
digit +=
(what[1] >= 'A' ? ((what[1] & 0xdf) - 'A') + 10 : (what[1] - '0'));
return (digit);
}




What to do now? One possibility would be to change the code and rebuild JK from scratch. To do this I would need Visual C++ and probably some time to get it all worked out. Another way to fix this is to patch the JK DLL file so that HttpFilterProc does not enter the path of execution that causes the problem. This is exactly what I did.

I disassembled the DDL and found the place to patch it:

.text:6A6BAB0B loc_6A6BAB0B:                           ; CODE XREF: HttpFilterProc+22A j
.text:6A6BAB0B                 lea     edx, [esp+39A8h+Str1]
.text:6A6BAB12                 push    edx
.text:6A6BAB13                 call    sub_6A6BB490
.text:6A6BAB18                 add     esp, 4
.text:6A6BAB1B                 cmp     eax, 0FFFFFFFFh
.text:6A6BAB1E                 jnz     short loc_6A6BAB47
.text:6A6BAB20                 mov     ecx, Time
.text:6A6BAB26                 lea     eax, [esp+39A8h+Str1]
.text:6A6BAB2D                 push    eax
.text:6A6BAB2E                 push    offset aSContainsOneOrMoreInvalidEscapeSequences_ ; "[%s] contains one or more invalid escap"...
.text:6A6BAB33                 push    4
.text:6A6BAB35                 push    0
.text:6A6BAB37                 push    4CAh
.text:6A6BAB3C                 push    offset aJk_isapi_plugin_c ; "jk_isapi_plugin.c"
.text:6A6BAB41                 push    ecx
.text:6A6BAB42                 jmp     loc_6A6BB000
.text:6A6BAB47 ; ---------------------------------------------------------------------------
.text:6A6BAB47
.text:6A6BAB47 loc_6A6BAB47:                           ; CODE XREF: HttpFilterProc+26E j
.text:6A6BAB47                 cmp     eax, 0FFFFFFFEh
.text:6A6BAB4A                 jmp     short loc_6A6BAB97
.text:6A6BAB4C ; ---------------------------------------------------------------------------
.text:6A6BAB4C                 mov     eax, Time
.text:6A6BAB51                 lea     edx, [esp+39A8h+Str1]
.text:6A6BAB58                 push    edx             ; ArgList
.text:6A6BAB59                 push    offset aSContainsForbiddenEscapeSequences_ ; "[%s] contains forbidden escape sequence"...
.text:6A6BAB5E                 push    5               ; int
.text:6A6BAB60                 push    0               ; Source
.text:6A6BAB62                 push    4D2h            ; int
.text:6A6BAB67                 push    offset aJk_isapi_plugin_c ; "jk_isapi_plugin.c"
.text:6A6BAB6C                 push    eax             ; Time
.text:6A6BAB6D                 call    sub_6A6CEDF0
.text:6A6BAB72                 push    offset aDoctypeHtmlPublicW3cDtdHtml4_0TransitionalEn ; 
I replaced the original conditional jump instruction with an unconditional one (jmp) so that the problematic code is never executed.

I hear that this is a problem for people who have a Citrix License Server installed on their IIS along with an Exchange (Front-End) server because Citrix uses Apache Tomcat and thus requires the functionallity of JK. Of course all other installation that use JK to redirect to an Apache Tomcat engine are affected.

I will see if I find time to discuss this on the JK developers mailing list because I believe the best way to solve this is to change the source code and release a new version of JK. Of course, it needs to be discussed why the developers chose to reject request URls with encoded slashes inside first. Unfortunaltley, the code does not give any hints.

So what can you do until this is hopefully fixed for real?

Well, you could get the source code yourself, make the necessary change and rebuild the DLL or just patch it like I did.

For the DLL version 1.2.26 (32bit) the offset to patch and the bytes to change are as follows:
isapi_redirect.dll
0000AB4A: 75 EB
You can also download the patched version that I made here.

Wow, this post got pretty long for the first one...

Please leave a comment if you find this useful.

References:
Newgroup Discussion
Thread on German PPC Forum PPC-Welt

Information in this post or the patched DLL must not be used commercially without mentioning this blog post.
Post a Comment