mpAjax (multi-part Ajax responses)

http://test.getify.com/mpAjax

Questions or Thoughts? Contact me @getify on twitter. If you like this project, you also might like to check out jXHR (JSON-P XHR), LABjs (Loading And Blocking Javascript) and flXHR (easy cross-domain Ajax).

Current version: 0.3.4 mpAjax-0.3.4.zip. This project is now hosted over at jQuery Plugins.




Release Notes

0.3.4 Cleaned up some code, now hosted on github/mpAjax.

0.3.2 Fixed a bug where jQuery automagically translating the 'data' parameter to an object (XMLDOM) would throw a JS error.

0.3.1 Fixed a bug where not specifying a "success" handler would cause a JS error.

0.3 This release is a significant change over 0.2.1. It now features full integration with the "ajaxSuccess" event binding functionality of jQuery. This means you can either specify your multi-part handlers per request (via "success"), or bind one or more global event handler(s) for the "ajaxSuccess" event.

The demo has now been updated to allow you to choose one of three types of handler specification (callback, single event handler, multiple event handlers). The most exciting part of the 0.3 release is the ability to split up your handlers into separate event bindings bound to different DOM objects. In the demo, for instance, it shows binding the "text/html" handler to the actual DIV that the HTML will go into. And the "application/json" handler is bound to the DIV where it's data will be dropped. Notice in the demo that the "this" binding for the "separate_event_handleList()" and "separate_event_handleData()" functions is bound to the respective DOM elements that the event was bound on. This makes code more semantic, simple, and sensible. It also gives you much more flexibility.

It should be noted that only the "success" handler and the "ajaxSuccess" event are supported for mpAjax functionality. The other callbacks/events, such as "complete"/"ajaxComplete", simply pass you the unparsed data as before.


The concept here is that this plugin extends the core jQuery ajax() functionality (useful for all xhr methods, get(), post(), etc) to allow it to be smart in parsing a response and doing some additional logic if it finds multiple parts (think, email MIME multipart delimiters) in the response. However, it's important to note that the code maintains backward compatibility, so even if you have the plugin in your page and you make an Ajax request that comes back with a normal response, the code will behave EXACTLY as normal. The only special functionality jumps in if the response from the server has the response formatted as multi-part by convention. So you can safely use the plugin in a hybrid page where you do some of each type of call/response.

I have noticed that there are a lot of times in web app development that I need to make an Ajax call to the server and have it return multiple types of "data" in response. For instance, as shown in the demo, I might make a search/filter query to a DB, and get back the HTML table with data filled in already. But I also might need to pass back some meta data about the result set, like page count or total records.

There are a couple of ways this is normally done. One way is to send back the response as all HTML, with the meta data hidden in input[type=hidden] fields, or some other sort of way embedded in the HTML. This is inefficient, and also kind of unclean in that your Javascript then has to go and inspect the response or the DOM to get the values it needs. It's poor separation of asset types.

The other way is to make a JSON packet, with the values filled in as JSON properties, then escape and pack up your HTML string into another property of the JSON object. This is functional, but not efficient. For one, HTML tends to have a lot of " characters, which each one needs to be \ escaped to be valid JSON. This means that the response is bigger by one character for each " character (and actually, even each \ that happens to appear, though those are less common) that needs to be returned to the browser. I did some tests with Digg.com and Yahoo.com to see just how many " marks tend to appear in HTML, and it turns out a pretty high percentage. For Digg, it would be an extra 4.5k of download just for all those escape characters. For Yahoo, it would be 1.7k! Granted, you don't usually return an entire huge page via Ajax, but the point is, the proportion of " characters in HTML is non-trivial, so the mixed HTML/JSON packets with escaping are bigger and less efficient.

Also, the server does have to do all that escaping at some level when it packages up the packets. That takes some processing time. I did some benchmarking with PHP's built in json_encode(), and found it to have an impact on how quickly it encoded a big block of HTML that had only single quote ' chars in it, requiring no escaping, versus the same block of HTML with " characters that did have to be escaped. So, there's inefficiency in this part of the process, too.

So, I figured that if I could return the HTML natively, without any escaping, in the return stream, and the JSON natively, without any conversion or escaping, in the same stream, and then parse the results as separate content-types on the client, I'd squeeze out some more performance and reduce transmission size. Also, it will make the return stream more cleanly formed, making debugging easier.

Thus, was born my idea for using the same standard used in email MIME multipart delimiters to "separate" different types of data in an Ajax return call, and let the jQuery ajax code parse the response and treat each part as a separate "response" (meaning, each one could have its own handler to handle that part properly). So, the HTML block is passed to a handler that knows to drop it into an "innerHTML" property somewhere in the DOM. And the JSON block is passed to a handler that knows to eval() it or better yet, pass it to the JSON.parse() library for save eval()ing.

A "multi-part" response might look like this:

!!!!!!=_NextPart_411426488
Content-Type: text/html
 
<div>Hello, I'm some HTML markup, and I can use " characters natively with no problems.</div>
 

!!!!!!=_NextPart_2106560908
Content-Type: application/json
 
{"greeting":"Hello","name":"I'm JSON, natively without any transformations or escaping!"}

Theoretically, any type of textual response can be handled like this: HTML, CSS, JSON, etc. If Javascript can take that block of response text and do something meaningful dynamically with it, then it's a candidate for being returned as a multi-part response part. In fact, by convention, each "part" gets a delimiter that has a unique identifier, and also a content-type header declaration. This means you could have a response with several blocks of HTML to be used in different places, and also several blocks of JSON or CSS to also be used as appropriate.

And the plugin accomplishes all this by being very unobtrusive to how you normally write your jQuery ajax scripts. For instance:

Old:

 jQuery.post(
    "http://test.getify.com/mpAjax/search.php",
    "json="+JSON.stringify(requestbody),
    handleResponse
    "text/plain"
 );

New:

 
 jQuery.post(
    "http://test.getify.com/mpAjax/search.php",
    "json="+JSON.stringify(requestbody),
    {
     "text/html":handleList,
     "application/json":handleData
    },
    "text/plain"
 );

Notice all I do is extend how you can specify the handlers for a response. You can still have a single function, and it will either be called once for a single part response, or be called multiple times in succession, once for each part of a multi-part response. In the multipart type call, you actually get an object passed to the function (instead of just a string of data), and the object has the delimiter header (with ID in it) from that part, and also it's declared headers, like content-type or content-length. So, your single handler can inspect those properties and know what to do with each "part".

Or, better, you can specify an object literal that has a single handler for each content-type, as shown above. Whenever a "text/html" block is encountered in the response, a rich data object for that "part" will be sent to the handler you specified for that type of response. Same for "application/json".

Lastly, if you just want to pass an array of handlers, they will be called in order, each one once, as the "parts" of the response are parsed. So, if you know you will always have two html blocks and one JSON block for a particular response, you can pass an array with 3 handler callbacks and they'll be called appropriately, again with the appropriate rich data object.

So, the point here is you can return multiple types of "data" in your single response, and have your Javascript code treat them like separate responses. This makes your response data cleaner, smaller, and overall more efficient, than other options available. And it plays nicely with regular single-part responses, so there's no pain in integrating it where you need it, to get more performance from your Ajax.

The URL above has the working demo. But also, the VERY simple PHP I used to construct the multi-part response by convention is shown here:

http://test.getify.com/mpAjax/mpAjax-php.txt

Lastly, let me say that this is not explicitly a jQuery only thing. I just wrote the first version in jQuery since I know it best. But I intend to port it to the other frameworks. And it should be pretty easy for others to do as desired. It can even be adapted to just work with native bare XHR without any framework needed, if so desired.