Couple of weeks ago, I wrote a clean, fast networking toolkit for iOS and Mac written for the LLVM Compiler 3.0 with ARC.
Reception was very good that it was the “most-watched” repository on Github last week. Early adopters have sent me innumerable emails on how fast their network operations are, and how responsive their app is after integrating MKNetworkKit.

 

MKNetworkKit is faster (and it makes your app feel smoother and faster with seamless transparent caching) in reality. There are two things that makes MKNetworkKit faster.
First, it’s ARC based and the awesome LLVM 3.0 compiler brings in a ton of performance benefits. Using the @autoreleasepool block instead of NSAutoReleasePool in MKNetworkOperation alone should improve the performance by a huge factor (Apple claims it to be 6 times faster). If you still haven’t migrated to ARC because you want to support those iOS 3.x users, read this post by Matt Gemmell first.

Second important performance benefit is seamless and transparent caching. By transparent caching, I mean, caching that involves zero overhead from the developer (That’s you).

Let me, in this post, brief you about how this seamless caching might help you. First, an example. You are refreshing a twitter stream. A GET request for that looks similar to

GET http://twitter.com/mugunthkumar/statuses

Caching your responses, the wrong way

Some developers cache responses using Core Data. STOP IT

Using Core Data for caching is like using the military to kill bedbugs in your bedroom. DON’T DO IT.

The problem when you implement Core Data is, you should program your view controllers to work with two kinds of data structures. One, JSON/XML from server and the other, Core Data NSManagedObjects. Some “clever” developers “take it to the next level” by just programming their view controllers to work with only Core Data. The design goes like this. The network layer talks to the server and updates Core Data Store. Once the store is updated, send a NSNotification to “refresh” view controllers.
This may sound clever to many. But it’s completely WRONG. I REPEAT. COMPLETELY WRONG. Core Data (or even structured SQL storage) is not meant for that. Core Data is a structured storage for persisting/serializing Object graphs in your application. Don’t use it for storing objects that have a low life time. Thumbnail images, list of tweets etc., just don’t belong there. Secondly, this would require you to read and write to flash memory on the iDevice which have a limited read/write cycle. Finally, reading and writing to disk (in this case, Core Data) is a very ugly way to transfer data from one class to another (in this case, from your Network Layer to View Controllers)

Say hello to MKNetworkKit


MKNetworkKit calls the SAME completion handler with cached data if you are making the call for the second time. When the network connectivity is proper, MKNetworkKit calls your completion handler twice. First with the cached data and again after fetching the latest data from server. As such, you can design your view controllers to work with just ONE kind of data, data from server. If you have an app that doesn’t cache and doesn’t work offline, just replace your networking library with MKNetworkKit and you get caching for free. Not even a single line of code change is required on your view controllers. Caching is transparent. Your view controllers needs to work with the same data structure, no matter whether the data is from cache or server.

Behind the scenes

MKNetworkKit caching is super light-weight and it caches your responses in a NSMutableDictionary. Doesn’t that lead to high memory usage? Yes. of-course, but MKNetworkKit is clever. It observes the UIApplicationDidReceiveMemoryWarningNotification and flushes the dictionary to disk. The next time you make a request to the cached URL, the cached data is brought back to memory. So in-effect, it has a in-memory cache and disk cache and uses a least recently used algorithm to flush items to disk. The MKNetworkEngine class has methods that can be over-ridden to control the cache cost.

Just override the method

-(int) cacheMemoryCost;

and return a higher/lower value based on your application’s requirements.

MKNetworkKit caching works like a intermediate proxy server by intercepting the response headers. So, when a second GET request to the same URL is made, MKNetworkKit behaves the following way.

1) For GET requests that sent a ETag in the header previously (for example, requests to Amazon S3 servers), MKNetworkKit sends a second request as a HEAD request (even if you specify as GET) with IF-NONE-MATCHheader. Most servers that send ETag (like Amazon S3) responds to IF-NONE-MATCH by sending the real data only when it has been changed. Otherwise, they return a 304 Not Modified which MKNetworkKit safely ignores. So data transfer is really not huge. Even this request is honored only after 1 day. So, repeated requests to the same thumbnail image doesn’t even hit the server.

2) For any GET request that contained a Cache-Control:max-age=in the header, MKNetworkKit honors that and makes the second request only after the expiry date. For dynamically generated requests, you should set Cache-Control on server. This not only helps MKNetworkKit, but most proxy servers along the way.

3) For requests that contained a Last-Modified field in the header, MKNetworkKit sends a “IF-MODIFIED-SINCE” HEAD request. This again works like the IF-NONE-MATCH. The server, either sends the complete data or a 304 Not Modified.

4) For servers that implement Cache-Control: no-cache, MKNetworkKit goes one step ahead and makes a second request only after a minute (60 sec). So repeated refreshes to often refreshed page will not exactly hit the server. For example, refreshing foursquare check-ins. It’s rare for these information to change within the next 60 sec.


5) For performance reasons, when the response type is an image, and the response headers doesn’t have any cache control information, MKNetworkKit assumes an expiry date of 7 days.

The last two are not an RFC standard. But these optimizations make network operations fast on a mobile devices even if the server isn’t implemented in an optimized way. All POST, PUT, HEAD and DELETE requests are ignored and not cached.

And, of course, you can customize all these behaviors by editing the values in MKNetworkKit.h

Freezing Operations

Another very important feature of MKNetworkKit is operation freezing. Imagine that, your customer take an awesome photo with your own “Instagram-killer” app, but since he is at a remote exotic place, 3G connectivity is poor and uploading the photo fails. Without MKNetworkKit, you should remember the photo related meta data, store the photo’s NSData in a file and upload the operation the next time the app is launched. Painful!
With MKNetworkKit, you mark the photo upload operation is “freezable”. This means, in the event of network failure, your operations are automatically serialized (frozen beneath snow) and restored the next time the app launches, all for free.

MKNetworkOperation *op = [myEngine operationWithPath:@"/imageUpload" params:paramsDict httpMethod:@"POST"];
[op setFreezable:YES]; // ONE EXTRA LINE
[myEngine enqueOperation:op];

You can mark any POST/PUT/DELETE operation as freezable. GET operations are not freezable and MKNetworkKit ignores your call to setFreezable: if your operation is a “GET” operation.

Advanced Tips

Friction-free authentication

MKNetworkOperation *op = [myEngine operationWithPath:@"/letmein"];
[op setUserName:@"mugunth" password:@"mYsEkReTpAsSwOrD"]; // ONE EXTRA LINE
[op setUserName:@"mugunth" password:@"mYsEkReTpAsSwOrD" basicAuth:YES]; // OR THIS LINE
[myEngine enqueOperation:op];

When your server uses HTTP Basic auth or Digest auth or even Windows NTLM authentication, all you need to do is to set your username and password and MKNetworkKit auto-magically authenticates your request!. For NTLM authentication, just ensure your username is “domain\username”. There is no separate method setDomain: or something.

You can also set basicAuth:YES to send the credentials even before receiving the authentication challenge. However, this works only if your server uses HTTP Basic Authentication.

Custom Authentication

If your server uses HTML Form authentication, you can override the authHandler block and provide custom auth mechanisms.

MKNetworkOperation *op = [myEngine operationWithPath:@"/letmein"];
op.authHandler = ^(NSURLAuthenticationChallenge *cred) { 
// show a web view controller or do whatever you want and finally, when you have a credentials ready, just call 
[challenge.sender useCredential:credential forAuthenticationChallenge:challenge];
};
[myEngine enqueOperation:op];

Overriding MKNetworkEngine

Tweaking the URL building methods with MKNetworkEngine is easy. Let me start by giving you an example. Imagine that you have a server which authenticates a logged in user and after authentication, it expects a Authorization header to be set with an access token.
You need to have a factory method that creates URLs customized to your web service. This could include adding a authorization header in our case. You can do this by overriding the method,

-(void) enqueueOperation:(MKNetworkOperation*) operation;

For example, in the above case, you can check your engine subclass for an access token and add it to the operation’s header in this method (if you have one).

Overriding MKNetworkOperation

Sometimes, you might do custom error handling based on the response from your server. For example, your server might send a valid HTTP response (status code 200) which could be an internal error, like “Invalid User”. Business logic errors are better handled with your own application level error codes. You overriding MKNetworkOperation to customize the success/failure reporting. I would highly recommend doing this along with designing your server to send error codes for error conditions.

The following methods are to be overridden for customizing error handling.

-(void) operationSucceeded;
-(void) operationFailedWithError:(NSError*) error;

For example, in the previous case, you can override operationSucceeded, inspect the response and if the response was indeed a business logic error (“Invalid User”), you can call the [super operationFailedWithError:[Your custom error class for "Invalid User"]];
Otherwise, call [super operationSucceeded].

You can also override operationFailedWithError to introspect the actual cause of the error, if your server sends any error related information in the dictionary.

The reason for overriding at this level is to minimize error handling at view controller layer. So your presentation layer, UIViewController subclasses will be cleaner to read.

If you have subclassed the MKNetworkOperation and want the factory method in your engine subclass, operationWithURLString:params:httpMethod to prepare an operation of your subclass. To register your operation subclass with the engine, you can call the registerOperationSubclass: method.

-(void) registerOperationSubclass:(Class) aClass;

Your code fragment might look like,

[self.myEngine registerOperationSubclass:[MyAppNetworkOperation class]];

Image Cache with MKNetworkEngine

MKNetworkEngine has a handy method to load images from a server. It’s built-in cache mechanism ensures, your images are cached without using any third-party image cache libraries. That means your code base is even leaner.
You can use the following method of MKNetworkEngine to load images.

-(void) imageAtURL:(NSURL *)url onCompletion:(MKNKImageBlock) imageFetchedBlock;

MKNKImageBlock is defined like this.

typedef void (^MKNKImageBlock) (UIImage* fetchedImage, NSURL* url, BOOL isInCache);

So your calling code will look like,

    [ApplicationDelegate.myAppEngine imageAtURL:[NSURL URLWithString:@"http://example.com/image.png"] onCompletion:^(UIImage *fetchedImage, NSURL *fetchedURL, BOOL isInCache) {
        self.imageView.image = fetchedImage;
    }];

If you are loading images onto a imageView inside a tableview cell, you can check if the completed URL is same as the passed URL

    [ApplicationDelegate.myAppEngine imageAtURL:[NSURL URLWithString:@"http://example.com/image.png"] onCompletion:^(UIImage *fetchedImage, NSURL *fetchedURL, BOOL isInCache) {
	if(originalURL == fetchedURL)
        cell.imageView.image = fetchedImage;
    }];

This check is necessary as tableview cells might be recycled and a cell might end up getting images from multiple network operations.

You might also want to read my other elaborate post on Image Caching with MKNetworkKit

Lastly, if you are fading in your images like those fancy apps, add your fade-in logic if the image is loaded from server and not from cache.

Source Code

MKNetworkKit on Github

MKNetworkKit might look simple, but it’s incredibly powerful. Use it in your apps and tell me how it scores. My next blog post would be on how to write a better RESTful server that serves a mobile device. Watch this space, subscribe to my feed or follow me on twitter.

Interested in learning in-depth features of Objective-C and iOS? You should get my book.

Amazon - http://mk.sg/ios5book

iBooks - http://mk.sg/ibook

Wiley – http://mk.sg/book

Book Depository – http://mk.sg/bdbook

Mugunth

Follow me on Twitter

  • http://twitter.com/fbartho Frederic Barthelemy

    Would you mind expanding on your “Authorization Header” example? Oauth is a very common operation, and having that implemented at the network layer would be glorious. What would be needed to make something like that happen for say Twitter access?

  • Justin Copeland

    I am having trouble getting the image caching with imageAtURL to work in a tableView. The images don’t get displayed until after they scroll out of view and then scroll back in. If I try and add a [tableView reloadRowsAtIndexPaths:[NSArray arrayWithObject:indexPath] withRowAnimation:UITableViewRowAnimationFade]; to the completion callback I get a bunch of index out of range errors and other weirdness. Any thoughts?

  • Nini_mimi2

    Awsome Framework!

    One question though…

    I am trying to POST with JSON in the body of the operation. In ASIHTTPrequest I would use [request appendPostData:(NSData*)]

    how can I do it with MKNetworkKit?

    • Joebob

      Same issue here. I don’t see a way to do this in MKNK.

    • Chris

      I’ve got the same issue. If your json is flat (no nested elements) you can do it my passing a NSDictionary as the param. But it does not seem to get encoded correctly if you have nested elements.

    • Sage

      Have you’ll solved this? if so would you please share your solution

    • lyon

      I have the same question, Is this solved now?  

      • MugunthKumar

        No. I’m still working on this.

        • Emmanuel

          Hi Mugunth, first of all, congrats on this amazing library, I’ve been playing around with it for a day and I have to say, it’s very impressive what you did here, anyway, I’m in the same boat than Nini and lyon, is there a way to set the post body of the request? I’m trying to post some JSON through [ MKNetworkOperation addData: forKey: ] and my server is reporting the posted key as a file? Thanks in advance!

  • Maxim Golunov

    Freezing operations is an awesome idea, however I found it troublesome to use in my case. I enqueue the operations that can be “frozen” and wait for completion block or error block to be called to give feedback to the user. If network is not reachable, the operation is frozen, however neither completion nor error blocks are not called. Ever. I can see that when connection is alive again the operation is restored and runs successfully, with no feedback though. I understand, that operation blocks are not serialized with freezable operations. Do you plan doing anything about it, or would that remain your framework limitation?

    • Anonymous

      As of now, it’s a framework limitation since blocks cannot be serialized. I’m looking for alternative ways to implement that though

  • Brian Christo

    Hi

    I am calling multiple file downloads using for loop….

    But then I have a cancel option. If i click the cancel button, all the download operations has to be stopped or cancelled. 
    But in my case, the operations are not cancelled. The download continues and the connection:didReceiveData in MKNetworkOperation.m is called until all those download completes…

    I have attached the screenshots of my code..

    I am always getting the log as “self.downloadOperation is not Cancelled”.. and the download continues… I want to cancel all existing download operations totally….

    Please have a look and help me please…

    Brian.

  • http://www.facebook.com/people/Stefan-Emanuelsson/1367400037 Stefan Emanuelsson

    Thanks for making this framework available, i nearly peed my pants when i read of all the features. I’ve converted to an ARC-guy now thanks to this, and will look forward to removing tons of code (especially for loading images). Removing code is my favorite form of refactoring, so this made my day :)

  • http://twitter.com/EvGeniyLell EvGeniyLell

    How can I repeat the MKNetworkOperation several times during after failure?[operation onCompletion:^(MKNetworkOperation *operation) {DLog(@"%@", operation);} onError:^(NSError *error) {DLog(@"%@", error);[self enqueueOperation:operation];// not work}];

  • Billo

    does multipart post work with the framework

    • MugunthKumar

      Yes it should

  • Carl

    Is the JSON issue sorted?

  • Testing

    test

  • Gilad

    Thanks for building this framework!
    Question: I’m trying to upload several images from the camera rolls, and after enqueing them using 
    addData, I often get a memory warning and my app gets killed. I’d like to be able to do AddALAsset (or, better yet, AddALAsset: withRepresentation:) so I don’t blow up memory just for enqueing the image.  Before I start coding, I wanted to check if you have any ideas, suggestions or already have something in the works.

    • http://www.facebook.com/surajthomask Suraj Thomas K

      Hi Gilad, I’m facing a similar problem. I think, there is some memory thrashing happening with addData.

  • Tony

    HI there, I absolutely love this framework, one problem I am having is testing it though? As the calls can not be made synchronous I am running into a problem using unit testing with this framework, does anyone have suggestions?

  • http://www.facebook.com/surajthomask Suraj Thomas K

    Hi Mugunth,

    there is a memory leak when uploading files using mknetworkkit. Like, suppose, you are sending a file with 2MB size, then when I call addData method, it’ll create an additional allocation of 2.5MB, other than the original file in memory. When sending chunks of data from a large file, it is causing crash.
    May be this problem is only with me. I’m using classes that have ARC disabled along with MKNetworkKit. You can find the leak, if you just try uploading a file and examine it in Instruments->Allocations.

    By the way, this is a solid good framework and thank you for sharing this awesome framework with online community.

    Regards,
    Suraj

  • motionpotion

    Does this framework support kerberos authentication?

  • http://twitter.com/ryoushi77 ryoushi77

    Hello,

    I´m using MKNetworkKit for enabling downloading of bigger files for my iPhone application. I have one question, because I could not find anything that would hinted me in the direction. Is it possible to stop and download and resume it afterwards?

    Regards

    Peter

  • John_The_Coder

    Hello,

    Right now I am facing a problem, while nesting multiple requests. i.e. Send request 1 and process result of request 1 and invoke the next request from the request1 completion block. But at some point I saw that the MKNetworkOperation becomes Nil. Its because the [NSURL UrlWithString:@"url"] is failing randomly. I have searched through net and couldn’t able to find the answer. Also I am calling all these methods from the same thread. Is there anything else you can help me with?

    Thnx,
    John