Dylan Socket Programming V1.0
=============================

I've been doing quite a bit of socket programming lately in Dylan, in
particular writing servers and the clients that connect to them. I've
been using the Functional Developer sockets module in the network
library.

This is my attempt at writing down some of the things I've discovered
so others can learn from my work and perhaps point out better ways of
doing things.

I'll start with server applications and in a later update write about
client applications. Server applications have a few gotchas and are a
bit more interesting as a result.

Creating a simple server is quite easy. The basic code looks something
like:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  while(#t)
    let remote-socket = accept(server-socket);
    do-something(remote-socket);
  end;

This will create a server socket listening on port 8000 and accepting
requests. The method 'do-something' is called with the remote socket
as a parameter. Note that it is quite safe to pass remote-socket to
another thread to do the work allowing the socket server to continue
accepting requests. It's as simple as:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  while(#t)
    let remote-socket = accept(server-socket);
    make(<thread>, function: curry(do-something, remote-socket));
  end;

Often you want to be able to use the ip address or host name of the
remote computer in some manner. For example, a web server would store
this information in a log file. The 'remote-host' method on the remote
socket returns an <internet-address> which provides this information:

  define method do-something(remote-socket)
    let remote-address = remote-socket.remote-host;

    format-out("Remote IP Address: %s\n", remote-address.host-address);
    format-out("Remote Host Name: %s\n",  remote-address.host-name);
  end;

The 'local-host' method on some sockets gives the address information
for the local computer. The address of the machine hosting the server
in other words. This cannot be called on a remote socket or you will
get a method not found error. Use it on a server socket in the
following manner:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  let local-address = server-socket.local-host;
  format-out("Local IP Address: %s\n", local-address.host-address);
  format-out("loacl Host Name: %s\n",  local-address.host-name);

Once the remote socket is returned from 'accept' you can treat it as a
<stream>. In this way writing to and reading from the socket is easy:

  format(stream, "Line 1\r\n");
  force-output(stream);
  let line = read-line(stream);

Note the use of 'force-output' to make sure the output is sent, and
the use of '\r\n' in the 'format' call. This sends the correct
new line sequence required by most 'internet' protocols (POP3, SMTP,
etc).

When using a remote socket it is possible for the remote end to close
the socket when you don't expect it. This can be detected by any
attempt to read from the stream. 

Reading from a socket closed by the remote end results in an
<end-of-stream-error> being raised. You can either handle this or use
the 'on-end-of-stream' keyword to the various read methods. Here are
some examples:

  // Using exception handling
  define method do-something(remote-socket)
    block()
      // If the remote socket closes on this call
      // then the exception clause is entered.
      let line = read-line(remote-socket); 
      format-out("Line: %s\n", line);

    exception(e :: <end-of-stream-error>)
      format-out("Remote host close the socket!\n");
    end block;
  end;

  // With 'on-end-of-stream' keyword
  define method do-something(remote-socket)
    // If the remote socket closes on this call
    // then the exception clause is entered.
    let line = read-line(remote-socket, on-end-of-stream: #f); 
	if(line)
      format-out("Line: %s\n", line);
    else
      format-out("Remote host close the socket!\n");
    end if;
  end;

If an attempt to write to a remote socket is made when the remote host
has closed the socket you will get a <connection-closed>
exception. This can be handled with an appropriate exception clause:

  define method do-something(remote-socket)
    block()
      // If the remote socket closes on this call
      // then the exception clause is entered.
      format(remote-socket, "Are you there?\r\n");
      force-output(remote-socket);

    exception(e :: <connection-closed>)
      format-out("Remote host close the socket!\n");
    end block;
  end;

Even if the remote host closes its end of the connection you must
still close your end of the connection. Using 'close' causes an
implicit 'force-output' to occur, sending an waiting data across the
socket. This will cause an error if the remote host has closed its
end. The 'close' method has a useful 'abort?' keyword argument that
when set to '#t' will not do the 'force-output'. 

So in any case where the remote socket has closed the connection you
need to close your end with 'abort?' set to '#t'. 

When closing the connection normally you would use 'close' without the
'abort?' keyword and allow the 'force-output' to occur.

The way I use to handle all these cases is code like the following:

  define method do-something(remote-socket)
    block()
      format(remote-socket, "Are you there?\r\n");
      force-output(remote-socket);
      
      let line = read-line(remote-socket); 
      format-out("Line: %s\n", line);
   
    exception(e :: <recoverable-socket-condition>)
      close(remote-socket, abort?: #t);

    exception(e :: <end-of-stream-error>)
      close(remote-socket, abort?: #t);

    cleanup
      close(remote-socket);

    end block;
  end method;

If a <recoverable-socket-condition> occurs, which covers a number of
conditions, including <connection-closed>, or an <end-of-stream-error>
occurs then the connection is closed without the 'force-output'. After
this call the socket is closed and any further attempt to close the
socket (as will happen in the 'cleanup' clause) is ignored due to a
'socket-open?' check in the 'close' method.

If no error occurs then the 'cleanup' clause correctly closes the
socket, 'force-output' occuring since it is not an abortive close.

This idiom cries out for a macro of course. Perhaps something like:

  define macro with-socket
  { with-socket(?:name = ?socket:expression)
      ?:body
    end }
  => { begin
         let ?name = ?socket;
         block()
           ?body
         
         cleanup
           close(?name);

         exception(e :: <recoverable-socket-condition>)
           close(?name, abort?: #t);

         exception(e :: <end-of-stream-error>)
           close(?name, abort?: #t);
         end
       end }
  end macro with-socket;

Then the previous code looks like:

  define method do-something(remote-socket)
    with-socket(socket = remote-socket)
      format(socket, "Are you there?\r\n");
      force-output(socket);
      
      let line = read-line(socket); 
      format-out("Line: %s\n", line);
    end;
  end method;

Functional developer uses blocking sockets. That is, 'read-line',
'format' and other calls block waiting for data to arrive or be
sent. The call to 'accept' also blocks which can cause a problem of
how to exit a server 'accept' loop. The problem occurs in code like:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  while(#t)
    let remote-socket = accept(server-socket);
    do-something(remote-socket);
  end;

How do you break out of the while loop? Placing a termination in the
loop is one approach:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  until(some-exit-criteria())
    let remote-socket = accept(server-socket);
    do-something(remote-socket);  
  end;

The problem occurs if the exit criteria says to exit but we are stuck
in the wait for the 'accept'. There is no timeout option for 'accept'
so the server is stuck there until a socket connection is
attempted. This is evident when a Ctrl+C keyboard interrupt is
tried. If the server is sitting inside 'accept' it doesn't get the
interrupt until a connection is made and 'accept' returns.

The approach I use to exit a server is to close the server
socket. Doing this on a server waiting inside 'accept' causes 'accept'
to raise a <blocking-call-interrupted> exception. Trapping this allows
exiting the server.

Unfortunately closing the server socket when not inside 'accept'
causes accept to throw a <type-error> if it is called later on the
closed socket. So this needs to be handled as well. Checking with
'socket-open?' before calling accept can help prevent this but in
multi-threaded code it is still possible. I use a 'safe-accept' method
to handle all this for me. It looks like:

  define method safe-accept(server-socket)
    block()
      when(socket-open?(server-socket))
        accept(server-socket);
      end;

    exception(e :: <socket-condition>)
      #f
    exception(e :: <type-error>)
      #f
    end block;
  end;

Note that I use <socket-condition> instead of just
<blocking-call-interrupted>. This is because Windows 98 and Windows NT
4.0 raise different exceptions at different points depending upon when
the 'close' on the server socket was called. To handle both OS's I use
<socket-condition> to bail out if there is any problem at all.

My server loop now looks like:

  let server-socket = make(<tcp-server-socket>, port: 8000);
  let remote-socket = #f;
  while(remote-socket := safe-accept(server-socket))
    do-something(server-socket, remote-socket);  
  end while;

When the server loop needs to be aborted/exited then the server socket
can be closed:

  define method do-something(server-socket, remote-socket)
    with-socket(socket = remote-socket)
      when(read-line(socket) = "quit")
        close(server-socket, abort?: #t);
      end;
    end;
  end;

I use the 'abort?' keyword passing '#t' when closing the server as
there is no data needing to be flushed.

This doesn't solve the problem of a keyboard interrupt not being
processed until the accept call returns. I'm open to any suggestions
on how to handle this case.

I hope this has been useful for anyone planning to use the sockets
module from Functional Developer. Comments and suggestions are
welcome.

Chris.