Dispatch HTTP Critique

Dispatch is a library for asynchronous HTTP interaction. It provides a Scala vocabulary for Java’s async-http-client. It is apparently popular and crops up as a transitive dependency of several other popular Scala products. So it can’t be all bad.

However, a recent project of ours had an unhappy experience with it.

Disclaimer

I’ll say at the outset that there is an alternative HTTP client of my own. So I’m biased. Nevertheless, I’ll try to be dispassionate, keep this brief and stick to the facts as we found them in our particular experience.

Problem 1: Complex API

The first problem we faced is that the API is complex. A lot of effort has been put into achieving an idiomatic Scala API, but in the process there are parts that have become obtuse and confusing. So confusing that a “periodic table of elements” is needed to interpret it.

Problem 2: Stateful Behaviour

OK I admit my own prior lack of understanding. I couldn’t grok the stateful behaviour of the API with respect to cookies etc and gave up. Sometimes weird things happened. But a hard-to-understand API is unlikely to be a good API. And Dispatch is seriously hard to understand.

Problem 2: Threads Whether You Want Them

Getting an asynchronous API to work inevitably means some other service thread(s) to do the background work. The threading model is not documented clearly and could easily become a source of frustration, when an existing system already has its own threads. For example, in our application, Tomcat has 25 service threads, each clamouring for the use of Dispatch.

Problem 4: Unhappy Failure Modes

Dispatch makes the response status hard to obtain. Also, it sometimes fails with confusing error messages.

Problem 5: Performance

The whole raison d’etre for an asynchronous API is to avoid blocking and thereby improve performance. As with all performance tweaks, an evidence-based approach is vital. So we did some measurements.

On our production server, Dispatch had variable and sometimes dreadful performance. We compared it with exactly the same application & configuration except where the HTTP calls had been converted to blocking calls using Java URLConnection. In our app, each incoming request gave rise to many downstream HTTP requests, which were made by Dispatch in the first case and by URLConnection in the second.

The performance improved from 6s per incoming request using Dispatch, to better than 200 to 250ms per incoming request using URLConnection. That was more than 25-fold improvement!

We were using Dispatch intensively - and it let us down badly. The advice here must be: measure up whether Dispatch works in your app before you commit to using it.

More generally, the lesson of this is that non-blocking APIs only improve performance in certain circumstances. They are not a panacea.

I suspect that Dispatch had three possible things that went wrong:

  • Not using HTTP keep-alive connections to their full potential. URLConnection races through successive requests with minimal overhead.
  • Extra thread scheduling overhead to handle the non-blocking behaviour.
  • Thread starvation - the wrong number of service threads might mean some of our 25 Tomcat threads actually end up waiting for the non-blocking I/O. Ironic.

Summary

OK, I’m sorry this comes over as rather negative, more negative than I intended. But I’ve just written down what happened to us. For the time being, I plan to avoid using Dispatch though.

 
comments powered by Disqus