Models for handling multiple concurrent connections

When I first learned networking in Java, it took me a while to wrap my head around the fact that there was no select(). All of my previous TCP programming experience was based on select-driven event loops in C, Python, or Perl. The select-loop seemed like a very natural way to handle multiple connections. The first server and the first C program of significant complexity that I attempted to modify was the LambdaMOO server — its relatively easily understood main loop formed my introduction to how one would build a server that handled multiple connections.

I played a little with toy fork-based servers but most of the things I wanted to play with such as MUD servers and HTTP applications required too much shared state. Forking and sharing lots of state between connections is more complicated than a single process with multiplexed I/O. All of the servers I wrote for work or play were select-based in C, Python, or Perl.

Around the same time I was also playing with GUI frameworks. Working with them felt naturally similar to the servers, since most GUI frameworks are also event-driven. A main loop receives event notifications and processes them. Here was when I first played with having the main loop spin off threads to do more processing but all of the interaction with the GUI still has to be in the main thread so the worker threads needed to dispatch events for the main loop to handle them.

This background makes a server with mostly event-driven processing and (if necessary) a few worker threads seem very natural. (See also: Twisted).

Java initially felt unnatural for doing nework server development — all these threads and no way to do multiplexed i/o. I got used to it but I missed the simplicity of a single I/O thread.

I’ve been considering options for implementing a service in Java for work that will manage outbound and inbound connections to interoperate with other, remote, services. Part of what it needs to do involves managing lots of TCP connections. I’ve been looking at some layers on top of NIO (for example EIO.

I think what I’m going to do is write the service directly on top of NIO. Libraries of code are great. Sometimes, though, the abstraction leaks in a way that the library provides no way to recover from. Or the library abstracts away some API complexity that was necessary to do something useful and now that your app is written against the library instead of the direct API you’re stuck. The fact that a library exists and appears to do what you want to do now does not mean there’s no opportunity cost associated with using it instead of writing directly to an API especially in the case of something as core to a server as how it interacts with the network. The problem is exacerbated when the library you decided to use stops being maintained yet the underlying API continues to evolve.

Sometimes the API is complex because the problem is complex and you’re better off understanding what’s going on even if you end up using an abstraction most of the time. I really wish Java had launched with multiplexed i/o rather than pretending that multithreading is simple or less complex than multiplexing. Then we’d have a world where all the programmers that learned network server programming with Java would understand event-driven I/O and not a world where so many people don’t seem to understand how much more complex a multithreaded application really is.