Multi-Threaded Servers

Multi-Threaded TCP Server #

Background #

A TCP server listens on a socket by invoking the listen function. When a client connects to the server, the accept function returns another socket and the server can communicate with the client over this newly created socket. A well-behaved server will probably continue to somehow service the original socket. If not, other clients that try to connect to it will not be serviced promptly. A popular architecture (by no means the only one) uses a thread to listen on the primary socket and dispatches other threads to take care of communication with each individual client. Our tcpserver class uses this architecture.

The tcpserver Object #

Here is beginning of the class declaration for tcpserver:

class tcpserver : public sock, public thread
{
public:
  tcpserver (unsigned int max_conn=0, DWORD idle_time = INFINITE, const char *name = 0);
  ~tcpserver ();
//...

It is derived from sock, our C++ socket wrapper, and from thread, an encapsulation for Windows threads. The thread class will be discussed in more detail latter in this article. Because it is derived from sock, you can use all the functions available to control this socket’s behavior. In particular you will probably have to bind it to an interface and a port number.

Being derived from thread, this object, when started, creates another thread of execution that keeps waiting for new connections. The other constructor parameters, max_conn and idle_time, specify the maximum number of connections permitted (0 is for unlimited connections) and the maximum time the server thread will wait for a new connection (INFINITE means the server will wait forever).

Included with the code of this article is a small sample that implements an echo server. This server simply waits for lines sent by the client and echoes them back complete with newline. From this sample code, here is how the echo server is constructed:

  tcpserver srv;

  srv.bind (inaddr (INADDR_LOOPBACK, 12321));
//...
  srv.start ();
//...

We have to look a bit inside the tcpserv implementation (in file tcpserver.cpp) to see what happens when a client connects to the port where the main server thread is listening. As you might imagine, every thread object, and tcpserver is a thread, has a run() function that does the bulk of the work. Taking out the tedious bits, a fragment of the run() function for the tcpserver looks like this:

  if (is_readready(0))
  {
    // - check if there is space in connections table
    if (limit && count >= limit)
    {
      //too many connections
      s.close ();
      continue;
    }
    contab_lock.enter ();
    // - find an empty slot in connections table
    ...
    inaddr peer;
    contab[i] = new conndata;
    contab[i]->socket = accept (peer);
    ...      
    // - invoke make_thread to get a servicing thread
    contab[i]->thread = make_thread (contab[i]->socket);
    ...
    // - invoke initconn function
    initconn (contab[i]->socket, contab[i]->thread);
    contab_lock.leave ();

Translated into English, it says that the server maintains a table of active connections and, when a new client connects, it invokes a virtual function make_thread to create a new thread to service the connection. It is up to this thread to do whatever it feels appropriate. A simple echo server will just return the lines it received from the client while a HTTP server will implement the HTTP protocol.

This design implies that you will have to derive your own class from tcpserver, with its own make_thread function, to implement a specific server. For simple servers however, like our echo server, there is a shortcut: you can use the set_connfunc() function to pass a function pointer or a lambda expression and the server will create a thread that has this function as body. The signature of the set_connfunc method is:

  void set_connfunc (std::function<int (sock&)>f);

It receives a reference to the socket created for the client (as a result of accept() function) and is supposed to carry out the whole conversation with the client.

Here is the whole implementation of our echo server:

int main (int argc, char** argv)
{
  tcpserver srv;

  srv.bind (inaddr (INADDR_LOOPBACK, 12321));

  srv.set_connfunc (
    [](sock& conn)->int {
      sockstream strm (conn);
      std::string line;

      //echo each line
      while (getline (strm, line))
        strm << line << endl;

      return 0;
    }
  );

  srv.start ();
  
  while (_kbhit ())
    ;
  _getch ();
  
  srv.terminate ();
  return 0;
}

This is the whole shebang:

  • we create the server and bind the main socket to a port
  • the connection function is a lambda expression that receives the connection socket as parameter
  • it creates a socket stream on that socket and keeps reading lines (using getline function)
  • each line is sent back to the client
  • when the client closes connection the loop breaks and the thread terminates
  • after starting the server thread the main thread waits for a key press
  • when a key is pressed the server is unceremoniously shut down and whole show stops.

In less than 20 lines of code we have implemented a fully functional multi-threaded echo server. At Rosetta Code there is a page with implementation of this server in different programming languages. Compared with the other languages, ours doesn’t look that bad.

The thread class #

This class is part a collection of wrappers for basic Windows synchronization objects. I’m well aware that there are now many synchronization objects (including threads) as part of the standard C++ library. Back in the day when I wrote mine, there were no such niceties. Even today my wrappers still offer the advantage of closely mimicking the Windows API functions. All these classes are derived from an abstract base class syncbase that encapsulates a Windows handle, be it a thread handle, an event handle, semaphore, mutex, etc.

While most other classes are really thin wrappers over the corresponding Windows API object, thread class is a bit more complicated. Below are the most significant methods of the thread object:

public:
                thread (std::function<int ()> func, const char *name=0);
  virtual       ~thread   ();
  virtual void  start     ();
  ...
protected:
                thread (const char *name=0, bool inherit=false, DWORD stack_size=0, PSECURITY_DESCRIPTOR sd=NULL);

  /// Initialization function called before run
  virtual bool  init      ();

  /// Finalization function called after run
  virtual bool  term      ();;

  /// Thread's body
  virtual void  run       ();

The protected constructor allows you to create an object derived from thread that presumably overrides the run() function to implement thread’s behavior.

The public constructor accepts a function pointer or a lambda expression that becomes thread’s body. In many cases it is easier to use this public constructor instead of deriving another object. The draw-back is that you don’t have such a finer control like the one provided by methods like init() and term().

Threads are created in a state of “suspended animation”. To let them run you have to call the start() function (do not confuse the two functions: run() represents the body of the thread, start() begins execution of the thread).

We’ve seen that tcpserver is derived from thread and has its own implementation of the run() function. For an example of thread created from a lambda expression look at the implementation of tcpserver::make_thread() function:

thread* tcpserver::make_thread (sock& connection)
{
  if (connfunc)
  {
    auto f =
      [&]()->int {
      int ret = connfunc (connection);
      close_connection (connection);
      return ret;
    };
    return new thread (f);
  }
  return NULL;
}

We said that the signature of the connection function is:

  int f (sock& socket);

Meanwhile the thread constructor needs a function with the signature:

  int f ();

The lambda expression f takes care of invoking the connfunc with the appropriate connection parameter. It also has the proper signature to be passed as argument to thread constructor.