Sockets

Overview

Sockets allow bi-directional communication between processes (2 or more). The processes can be on the same computer or on different computers connected via a network using TCP/IP. This makes sockets different from the previous 3 methods since they only worked with processes on the same system. Communication between processes on the same system is called UNIX domain sockets.

There are several types of domains, and these are three of the most popular:

  1. UNIX (AF_UNIX) - allows communication between processes on the same system.
  2. IPv4 (AF_INET) - allows communication between processes running on different systems using Internet Protocol version 4.
  3. IPv6 (AF_INET6) - allows communication between processes running on different systems using Internet Protocol version 6.

There are at least two types of sockets: stream and datagram.

Propertystream
(SOCK_STREAM)
datagram
(SOCK_DGRAM)
Connection-oriented?YesNo
Message boundaries? NoYes
Reliable? YesNo

We will focus on UNIX domain sockets (i.e. communicating between processes on the same computer).

Let's go to the code! (The code is based on an example from The Linux Programming Interface, chapter 57, section 2.) One process, the server, sets up a socket and a second process, the client, connects to the socket and sends bytes to the socket for the server to receive. It's unidirectional: client → server.

The first process (the server):

  1. creates a socket
  2. binds the socket to an address (filename in the AF_UNIX case)
  3. listens for incoming connections
  4. accepts a connection
  5. reads bytes from the socket and displays them to stdout
The last two steps are in an infinite loop where the server just continues to accept connections and read bytes (until the server is terminated, Ctrl-C from the command line). Many (most?) servers sit in infinite loops, patiently waiting for something to do, e.g. accept connections, process data, return results, etc.

There are a couple of (arbitrary) defines that are shared between the two processes that are put in a header file (socktest.h) so that both processes have the same information:

#define SOCKET_PATH "mysocket"
#define BUF_SIZE 200
Here's the server code (server1.c):
#include <sys/un.h>    /* struct sockaddr_un                                  */
#include <sys/socket.h>/* socket, connect, bind, listen, AF_UNIX, SOCK_STREAM */
#include <stdio.h>     /* printf, perror                                      */
#include <unistd.h>    /* read, write, close, STDIN_FILENO                    */
#include <errno.h>     /* errno                                               */
#include "socktest.h"  /* shared defines                                      */

/* How many pending connections we'll allow */
#define BACKLOG 5

int main(void)
{
  int sfd;                       /* server file descriptor          */
  int cfd;                       /* client file descriptor          */
  ssize_t bytes_read;            /* count of bytes read from socket */
  char buffer[BUF_SIZE];         /* to transfer data in the socket  */
  struct sockaddr_un addr = {0}; /* zero out socket structure       */

    /* Create the server socket */
  sfd = socket(AF_UNIX, SOCK_STREAM, 0);
  if (sfd == -1)
  {
    perror("socket");
    return 1;    
  }

    /* Make sure pathname fits into the struct member */
  if (strlen(SOCKET_PATH) > sizeof(addr.sun_path) - 1)
  {
    printf("Server socket path too long: %s", SOCKET_PATH);
    return 2;
  }

    /* If an existing socket already exists, remove it */
  if (remove(SOCKET_PATH) == -1 && errno != ENOENT)
  {
    printf("remove-%s", SOCKET_PATH);
    return 3;
  }

    /* Set the domain type (same system) */
  addr.sun_family = AF_UNIX;

    /* Set the pathname (safely) */
  strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);

    /* Bind the socket to the pathname */
  if (bind(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un)) == -1)
  {
    perror("bind");
    return 4;
  }  

    /* Indicate that the socket is ready (listening) for a connection */
  if (listen(sfd, BACKLOG) == -1)
  {
    perror("listen");
    return 5;
  }

    /* Process incoming connections forever */
  for (;;)
  {
      /* sleep(5); */ /* Test BACKLOG values */

      /* 
       * Accept the next incoming connection. It returns the socket 
       * descriptor from the client that is connecting.             
       */
    cfd = accept(sfd, NULL, NULL);
    if (cfd == -1)
    {
      perror("accept");
      return 6;
    }
    
      /* Read from client socket until EOF (in BUF_SIZE chunks) */
    while ((bytes_read = read(cfd, buffer, BUF_SIZE)) > 0)
    {
        /* Write bytes to stdout */
      if (write(STDOUT_FILENO, buffer, bytes_read) != bytes_read)
      {
        printf("partial/failed write\n");
        return 7;
      }
    }
    
      /* Check if the read had an error */
    if (bytes_read == -1)
    {
      perror("read");
      return 8;
    }
    
      /* Close client file descriptor */
    if (close(cfd) == -1)
    {
      perror("close");
      return 9;
    }
  }

    /* Close server file descriptor */
  if (close(sfd) == -1)
  {
    perror("close");
    return 10;
  }

  return 0;
}
Note that about half of the code is for error handling. This is extremely important with IPC, as we've seen before. There are many ways/reasons things can go wrong. Having these checks makes it trivial to find out which function failed and why. Without these, you can expect to spend hours debugging your code!

The second process (the client):

  1. creates a socket, setting the filename of the socket
  2. connects to the socket
  3. reads bytes from stdin and sends them through the connection
  4. closes the connection (socket)
Here's the client code (client1.c):
#include <sys/un.h>     /* struct sockaddr_un                    */
#include <sys/socket.h> /* socket, connect, AF_UNIX, SOCK_STREAM */
#include <stdio.h>      /* printf, perror                        */
#include <unistd.h>     /* read, write, close, STDIN_FILENO      */
#include "socktest.h"   /* shared defines                        */

int main(void)
{
  int sfd;                       /* socket file descriptor         */
  ssize_t bytes_read;            /* count of bytes read from stdin */
  char buffer[BUF_SIZE];         /* to transfer data in the socket */
  struct sockaddr_un addr = {0}; /* zero out structure             */

    /* Create the socket */
  sfd = socket(AF_UNIX, SOCK_STREAM, 0);
  if (sfd == -1)
  {
    perror("socket");
    return 1;
  }

    /* Set domain type */
  addr.sun_family = AF_UNIX;

    /* Set pathname (safely) */
  strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);

    /* Connect socket to pathname */
  if (connect(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un)) == -1)
  {
    perror("connect");
    return 2;
  }
  
    /* Read from stdin until EOF (in BUF_SIZE chunks) */
  while ((bytes_read = read(STDIN_FILENO, buffer, BUF_SIZE)) > 0)
  {
      /* Write bytes to socket */
    if (write(sfd, buffer, bytes_read) != bytes_read)
    {
      printf("partial/failed write\n");
      return 3;
    }
  }

    /* Check if read had an error */
  if (bytes_read == -1)
  {
    perror("read");
    return 4;
  }

    /* Close the socket */
  close(sfd);

  return 0; 
}
Again, there is error handling for every call to help find any problems. Here is what the socket looks like in the directory:

total 36,864 -rwx------ 1 chico chico 8,770 Nov 18 13:29 client1 -rw-r--r-- 1 chico chico 1,541 Nov 18 13:32 client1.c srw------- 1 chico chico 0 Nov 19 12:48 mysocket -rwx------ 1 chico chico 9,037 Nov 19 12:48 server1 -rw-r--r-- 1 chico chico 2,994 Nov 19 12:48 server1.c -rw-r--r-- 1 chico chico 53 Nov 18 13:57 socktest.h

You'll notice that the first letter in the long listing is the letter 's', which stands for socket. You can also see the permissions granting only the user (chico) read/write access. This is how you control access to sockets in Linux. It is just like controlling access to regular files. And because it's a socket, not a file, the file size is always zero.

Here's a diagram of what's happening:

The Linux Programming Interface ©2010  


Linux Abstract Socket Namespace

An abstract namespace is a Linux-only feature that allows you to create a socket without creating a file in the file system. It still has a name, but it's not visible in the file system. This can be useful for a few reasons:

It's trivial to create an abstract binding. You simply prepend a NUL character ('\0') to the name of the socket. In the code above, there is only one very minor change that must be made (in both the client and server).

Change this line:

  /* Set pathname (safely) */
strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);
to this:
  /* Set pathname (safely) */
strncpy(addr.sun_path + 1, SOCKET_PATH, sizeof(addr.sun_path) - 2);
I've highlighted what has changed. Because the array structure member addr.sun_path has already been filled with zeros, simply copying into the array starting at the second byte (instead of the first byte) will leave us with a string that has a 0 as the first character. That's it!

Notes:

Things To Try: (Given the code above)

  1. Using a named (i.e. non-abstract binding) socket, what happens if you run two instances of the server? What does the client do?
  2. Using an abstract binding, what happens if you run two instances of the server? What does the client do?


A More Real World Example

OK, so that was a nice Hello, World! example on how to setup and use a socket.

We basically just had two unrelated processes connected via a socket. One of the processes (the client) wrote bytes to the socket and the other process (the server), read the bytes from the socket and printed them to the screen. Simple.

Let's do something that's a little more of a Real World use. We're going to create a logging server that can be used to, well, log all kinds of events that are taking place on the system. Most operating systems have many of these kinds of "servers" running for various reasons. (Just look inside of /var/log to see hundreds of log files).

I'm purposely going to keep it simple because we want to focus on the communication aspect of the system, not what's actually being logged. We are basically going to have a client that is playing a "game" that I showed in a previous semester. Currently, the client writes its output to the screen like a normal application would.

However, I'm going to modify things so that the client will send the various "messages" to the server (via a socket connection) and have the server display the messages. This will actually simplify the client, which can now focus on the game-play instead of worrying about formatting and displaying messages.

Here are a couple of sample outputs from "running the game" with 3 players on each team:

First runSecond run
Team 1:
I'm a Pyro named Pyro1-00 [85,8,11]
I'm a Soldier named Soldier1-01 [35,4,5]
I'm a Pyro named Pyro1-02 [84,9,11]
Team 2:
I'm a Scout named Scout2-00 [25,4,3]
I'm a Scout named Scout2-01 [29,1,7]
I'm a Scout named Scout2-02 [26,2,4]

Scout2-00 is attacking Soldier1-01 (health=32)
Pyro1-02 is attacking Scout2-02 (health=15)
Soldier1-01 is attacking Scout2-02 (health=10)
Soldier1-01 is attacking Scout2-00 (health=20)
Pyro1-02 is attacking Scout2-01 (health=20)
Scout2-01 is attacking Pyro1-00 (health=78)
Pyro1-00 is attacking Scout2-00 (health=9)
Scout2-01 is attacking Soldier1-01 (health=25)
Scout2-02 is attacking Soldier1-01 (health=23)
Pyro1-00 is attacking Scout2-02 (health=-1)
Scout2-02 was killed by Pyro1-00
Scout2-00 is attacking Pyro1-00 (health=74)
Pyro1-02 is attacking Scout2-01 (health=9)
Pyro1-00 is attacking Scout2-00 (health=-2)
Scout2-00 was killed by Pyro1-00
Scout2-01 is attacking Soldier1-01 (health=22)
Soldier1-01 is attacking Scout2-01 (health=5)
Scout2-01 is attacking Pyro1-00 (health=67)
Pyro1-00 is attacking Scout2-01 (health=-6)
Scout2-01 was killed by Pyro1-00
Team 1 wins!
Team 1: Pyro1-00[67]  Soldier1-01[22]  Pyro1-02[84]  
Team 1: (Still alive: 3)
Team 2: Scout2-00[-2]  Scout2-01[-6]  Scout2-02[-1]  
Team 1:
I'm a Scout named Scout1-00 [25,4,5]
I'm a Pyro named Pyro1-01 [63,8,10]
I'm a Scout named Scout1-02 [28,5,6]
Team 2: 
I'm a Scout named Scout2-00 [25,5,4]
I'm a Pyro named Pyro2-01 [63,8,10]
I'm a Pyro named Pyro2-02 [82,8,11]

Scout2-00 is attacking Scout1-00 (health=20)
Pyro2-02 is attacking Pyro1-01 (health=52)
Pyro2-02 is attacking Scout1-02 (health=20)
Scout2-00 is attacking Scout1-02 (health=16)
Pyro2-01 is attacking Scout1-00 (health=12)
Scout2-00 is attacking Scout1-00 (health=8)
Scout1-00 is attacking Scout2-00 (health=20)
Scout1-00 is attacking Pyro2-02 (health=78)
Scout1-02 is attacking Scout2-00 (health=14)
Pyro2-02 is attacking Scout1-02 (health=5)
Pyro1-01 is attacking Pyro2-01 (health=55)
Pyro2-02 is attacking Pyro1-01 (health=44)
Pyro1-01 is attacking Pyro2-02 (health=70)
Scout1-00 is attacking Scout2-00 (health=9)
Pyro2-01 is attacking Scout1-00 (health=0)
Scout1-00 was killed by Pyro2-01
Scout2-00 is attacking Pyro1-01 (health=40)
Pyro1-01 is attacking Pyro2-02 (health=60)
Pyro2-01 is attacking Pyro1-01 (health=32)
Scout2-00 is attacking Scout1-02 (health=1)
Pyro1-01 is attacking Pyro2-02 (health=52)
Scout2-00 is attacking Scout1-02 (health=-4)
Scout1-02 was killed by Scout2-00
Scout2-00 is attacking Pyro1-01 (health=27)
Pyro1-01 is attacking Pyro2-01 (health=45)
Pyro1-01 is attacking Pyro2-01 (health=35)
Pyro2-02 is attacking Pyro1-01 (health=16)
Pyro1-01 is attacking Pyro2-01 (health=27)
Pyro1-01 is attacking Pyro2-01 (health=19)
Pyro2-01 is attacking Pyro1-01 (health=8)
Pyro1-01 is attacking Pyro2-02 (health=42)
Pyro2-02 is attacking Pyro1-01 (health=0)
Pyro1-01 was killed by Pyro2-02
Team 2 wins!
Team 1: Scout1-00[0]  Pyro1-01[0]  Scout1-02[-4]  
Team 2: Scout2-00[9]  Pyro2-01[19]  Pyro2-02[42]  
Team 2: (Still alive: 3)

Each team has 3 players (configurable) with varying levels of health and power (also configurable). When the game runs, it just randomly chooses a player from each team, and then randomly chooses which team attacks the other. Eventually, all players on one team have been killed and the team with at least one player still standing is the winner. Admittedly, not very exciting, but it will serve our purpose.

The reason we're doing this: We want the logging server to print out the messages instead of the client (game). We can embellish the output with all kinds of things such as color, sound, or animations. We could even have the server send an email or text at the end of each game alerting some other entity of what's going on. We could also have the server write all of the messages to a file somewhere, maybe on the same system or some other system on the network/Internet. There is no limit to what we can do with the messages.

The key is that we don't want the "game" to have to deal with all of this unrelated, non-gaming, sophisticated code. Plus, we may want to reuse the logging server for other applications (or games) and we don't want to duplicate code. Again, this is not unlike other logging servers on most operating systems.

Here's how the (default) output printed from the server should look when we're finished:
INFO: Team 1:
INFO: I'm a Pyro named Pyro1-00 [77,8,12]
INFO: I'm a Soldier named Soldier1-01 [36,4,6]
INFO: I'm a Scout named Scout1-02 [29,3,4]
INFO: Team 2:
INFO: I'm a Soldier named Soldier2-00 [40,3,7]
INFO: I'm a Scout named Scout2-01 [24,2,4]
INFO: I'm a Scout named Scout2-02 [27,2,4]
SHOT: Pyro1-00 is attacking Soldier2-00 (health=28)
SHOT: Soldier2-00 is attacking Soldier1-01 (health=33)
SHOT: Pyro1-00 is attacking Scout2-01 (health=16)
SHOT: Scout1-02 is attacking Soldier2-00 (health=24)
SHOT: Pyro1-00 is attacking Scout2-01 (health=4)
SHOT: Scout1-02 is attacking Scout2-02 (health=23)
SHOT: Soldier1-01 is attacking Soldier2-00 (health=18)
SHOT: Soldier1-01 is attacking Scout2-02 (health=17)
SHOT: Scout2-01 is attacking Scout1-02 (health=25)
SHOT: Pyro1-00 is attacking Scout2-01 (health=-4)
KILL: Scout2-01 was killed by Pyro1-00
SHOT: Soldier2-00 is attacking Soldier1-01 (health=30)
SHOT: Soldier1-01 is attacking Soldier2-00 (health=12)
SHOT: Soldier2-00 is attacking Soldier1-01 (health=23)
SHOT: Soldier2-00 is attacking Scout1-02 (health=22)
SHOT: Scout1-02 is attacking Soldier2-00 (health=9)
SHOT: Soldier2-00 is attacking Soldier1-01 (health=16)
SHOT: Soldier1-01 is attacking Soldier2-00 (health=3)
SHOT: Pyro1-00 is attacking Scout2-02 (health=5)
SHOT: Pyro1-00 is attacking Scout2-02 (health=-7)
KILL: Scout2-02 was killed by Pyro1-00
SHOT: Soldier1-01 is attacking Soldier2-00 (health=-1)
KILL: Soldier2-00 was killed by Soldier1-01
INFO: Team 1 wins!
INFO: Team 1: Pyro1-00[77]  Soldier1-01[16]  Scout1-02[22]  
INFO: Team 1: (Still alive: 3)
INFO: Team 2: Soldier2-00[-1]  Scout2-01[-4]  Scout2-02[-7]  
You'll notice we have 3 "levels" of logging:
  1. INFO - This is for informational messages.
  2. SHOT - This is when a player attacks (shoots) another player.
  3. KILL - This is when a player is killed (no health left).
These are just arbitrary levels that I made up for this demonstration. You could have as many or as few different kinds as you want. This is the very simple protocol that I made up. This means that, as well as sending the text to the server, we need to also indicate which type of message it is (INFO, SHOT, or KILL).

My solution is very simple: Just prepend a character to the message which specifies the type of message. There will be something like this in the code for both the client and server:

enum MSG_TYPE {mtUNKNOWN, mtINFO, mtSHOT, mtKILL};
So, when the client (game) sends messages to the server, this is what the strings would actually look like:
1This is an informational message.
2This is a message when an attack is made.
3This is a message when a player is killed.
Then, the server simply reads the first character to determine which string to print (INFO, SHOT, KILL) and prints all characters after the number, essentially skipping over the first character. As I stated, very simple. This "type" character will also allow the server to print the message in a certain color, or play a specific sound, or send an email, or something else that is relevant for the message type.

Here's a code snippet of how the client (game) currently displays a message using C++ I/O streams:

cout << "Soldier1-00 is attacking Soldier2-01 (health=44)" << endl;
and this is how it is done using a logging server via a socket:
send(mtSHOT, "Soldier1-00 is attacking Soldier2-01 (health=44)\n");
The send function is just a wrapper function I wrote around socket code in the client:
/* Socket file descriptor (server), file scope */
static int sfd; 

void send(int level, const std::string& string)
{
  ssize_t len;                 /* string length and count of bytes sent */
  char buffer[BUF_SIZE] = {0}; /* to format/transfer data in the socket */

    /* No error checking done on string length! */
    /* Format like this: "1This is a message"   */
  sprintf(buffer, "%i%s", level, string.c_str());
  len = strlen(buffer);

    /* Write the bytes to the socket, making sure it worked.       */
    /* The socket (sfd) was setup in main when the program started */
  if (write(sfd, buffer, len) != len)
  {
    printf("write failed\n");
    return; /* Should this be fatal? Call exit instead? */
  }
}
This display function in the logging server does the actual displaying:
/* IMPORTANT: The "strings" are not guaranteed to be NUL-terminated */
/* That's why the length must be passed to the function             */
void display(const char *string, int len)
{
    /* Determine the type of the message */
  const char *type;
  switch (string[0])
  {
    case '1':
      type = "INFO: ";
      break;
    case '2':
      type = "SHOT: ";
      break;
    case '3':
      type = "KILL: ";
      break;
    default:
      type = "";
      break;
  }

    /* Write the type first (e.g. INFO: ) */
  write(STDOUT_FILENO, type, strlen(type));

    /* Write the message (skip over the first byte) */
  write(STDOUT_FILENO, string + 1, len - 1);
}
OK, so that seems simple and straight-forward. Let's do a sample run and see what the server outputs. To keep it very simple, I only have one player on each team.
INFO: Team 1:
INFO: I'm a Scout named Scout1-00 [30,3,3]
1Team 2:
INFO: I'm a Pyro named Pyro2-00 [54,7,10]
2Scout1-00 is attacking Pyro2-00 (health=51)
2Pyro2-00 is attacking Scout1-00 (health=23)
SHOT: Pyro2-00 is attacking Scout1-00 (health=16)
2Scout1-00 is attacking Pyro2-00 (health=48)
2Scout1-00 is attacking Pyro2-00 (health=45)
2Pyro2-00 is attacking Scout1-00 (health=9)
2Scout1-00 is attacking Pyro2-00 (health=42)
2Scout1-00 is attacking Pyro2-00 (health=39)
2Scout1-00 is attacking Pyro2-00 (health=36)
2Pyro2-00 is attacking Scout1-00 (health=-1)
3Scout1-00 was killed by Pyro2-00
INFO: Team 2 wins!
1Team 1: Scout1-00[-1]  
1Team 2: Pyro2-00[36]  
1Team 2: (Still alive: 1)
Something is wrong. Some lines look fine, but others still have the integer (type) being displayed. What's going on?

Yes, the code appears to be "broken". The socket code is actually correct. I'm showing you this on purpose to demonstrate/explain what is happening and why. This will help you better understand the whole concept of a "stream of bytes", which is what the SOCK_STREAM means when creating the socket. Then, we'll see the trivial "fix" for it.

In fact, the "problem" is actually mentioned in this table (reprinted from above):
Propertystream
(SOCK_STREAM)
datagram
(SOCK_DGRAM)
Connection-oriented?YesNo
Message boundaries? NoYes
Reliable? YesNo
The "problem" is that we are using SOCK_STREAM which does NOT preserve message boundaries. In a nutshell, there is no guarantee that if the client writes 3 messages that the server will read 3 messages.

So, the client may write these 3 messages (showing the newline at the end of each):

2Scout1-00 is attacking Pyro2-00 (health=45)<NL>
2Pyro2-00 is attacking Scout1-00 (health=9)<NL>
2Scout1-00 is attacking Pyro2-00 (health=42)<NL>
But, if the server doesn't call read before the client calls write again, then they are all buffered like this:
2Scout1-00 is attacking Pyro2-00 (health=45)<NL>2Pyro2-00 is attacking Scout1-00 (health=9)<NL>2Scout1-00 is attacking Pyro2-00 (health=42)<NL>
until the server does a read. When the server eventually does do a read, instead of getting 3 separate messages, it just gets one big message. As another example, if a client wrote 4 messages, each 25 bytes, it's quite possible that when the server does a read, it gets one message that's 100 bytes. This is what is meant when we say that "message boudaries are not preserved", and SOCK_STREAM sockets DO NOT have any concept of a boundary. Afterall, it's just an unstructured "stream of bytes".

So, what's the solution? There are several, but we're going to take the easy way out (and learn more about sockets in the process). You'll notice that the table says that SOCK_DGRAM sockets do preserve message boundaries. Let's see how this kind of socket will fix things. You'll also notice that SOCK_STREAM is connection-oriented, and SOCK_DGRAM is not (connectionless). Here are diagrams that show the differences:

Connection-oriented (SOCK_STREAM)Connectionless (SOCK_DGRAM)
The Linux Programming Interface ©2010  

You'll notice some glaring differences in the SOCK_DGRAM sockets compared with the SOCK_STREAM type. With SOCK_DGRAM sockets:

OK, so how do we set things up to use SOCK_DGRAM instead of SOCK_STREAM? That's fairly simple, too. In both the server and client, change this (highlighted):
  /* Create the server socket */
sfd = socket(AF_UNIX, SOCK_STREAM, 0);
to this:
  /* Create the server socket */
sfd = socket(AF_UNIX, SOCK_DGRAM, 0);
In the server, remove the listen and accept lines and in the client, remove the connect line. Also, in the client, change the write call into a sendto call and in the server change the read call into a recvfrom call.

This was the original client write

  /* Write bytes to socket */
if (write(sfd, buffer, len) != len)
{
  printf("write failed\n");
  return;
}
and this is the new client sendto
  /* Send bytes to socket */
if (sendto(sfd, buffer, len, 0, (struct sockaddr *) &addr, sizeof(struct sockaddr_un)) != len)
{
  perror("sendto");
  exit(10);
}
This was the original server read
  /* Read from client socket until EOF (in BUF_SIZE chunks) */
while ((bytes_read = read(cfd, buffer, BUF_SIZE)) > 0)
{
  display(buffer, bytes_read);
}
and this is the new server recvfrom
  /* Process incoming connections forever */
for (;;)
{
    ssize_t bytes_read = recvfrom(sfd, buffer, BUF_SIZE, 0, NULL, NULL);
    if (bytes_read == -1)
    {
      perror("recvfrom");
      return 5;
    }
  
  display(buffer, bytes_read);
}
The prototypes:
size_t recvfrom(int sockfd, void *buf, size_t len, int flags,
                struct sockaddr *src_addr, socklen_t *addrlen);

ssize_t sendto(int sockfd, const void *buf, size_t len, int flags,
               const struct sockaddr *dest_addr, socklen_t addrlen);
The first 3 parameters are identical to read and write. The flags parameter will modify the behavior of the calls (which we don't need). The src_addr in recvfrom has information about the sending process and the dest_addr in sendto has information about the receiving process.

Since there is no "connection", the dest_addr parameter is necessary with sendto. Otherwise, where is the data going to be "sent"? We didn't need it with write because we were already connected to the socket. With recvfrom, the parameter tells us where the data came from, in case we want to reply to the sending process via the socket. However, since we are only doing communication in one direction (client → server), we set these parameters to NULL in recvfrom, indicating that we don't care about that information.

Now, we can run the code to see the new output (2 players on each team):

INFO: Team 1:
INFO: I'm a Scout named Scout1-00 [22,1,4]
INFO: I'm a Soldier named Soldier1-01 [30,4,8]
INFO: Team 2:
INFO: I'm a Scout named Scout2-00 [24,4,7]
INFO: I'm a Scout named Scout2-01 [20,1,5]
SHOT: Scout1-00 is attacking Scout2-01 (health=19)
SHOT: Scout2-01 is attacking Scout1-00 (health=17)
SHOT: Scout2-01 is attacking Soldier1-01 (health=25)
SHOT: Scout1-00 is attacking Scout2-00 (health=23)
SHOT: Scout2-00 is attacking Scout1-00 (health=10)
SHOT: Soldier1-01 is attacking Scout2-00 (health=15)
SHOT: Scout2-00 is attacking Soldier1-01 (health=18)
SHOT: Soldier1-01 is attacking Scout2-01 (health=15)
SHOT: Scout2-01 is attacking Scout1-00 (health=9)
SHOT: Scout1-00 is attacking Scout2-00 (health=11)
SHOT: Scout2-01 is attacking Soldier1-01 (health=17)
SHOT: Scout2-00 is attacking Soldier1-01 (health=10)
SHOT: Scout1-00 is attacking Scout2-00 (health=10)
SHOT: Soldier1-01 is attacking Scout2-00 (health=6)
SHOT: Scout2-01 is attacking Scout1-00 (health=4)
SHOT: Scout2-01 is attacking Scout1-00 (health=-1)
KILL: Scout1-00 was killed by Scout2-01
SHOT: Soldier1-01 is attacking Scout2-00 (health=-2)
KILL: Scout2-00 was killed by Soldier1-01
SHOT: Scout2-01 is attacking Soldier1-01 (health=9)
SHOT: Scout2-01 is attacking Soldier1-01 (health=4)
SHOT: Soldier1-01 is attacking Scout2-01 (health=11)
SHOT: Soldier1-01 is attacking Scout2-01 (health=3)
SHOT: Scout2-01 is attacking Soldier1-01 (health=3)
SHOT: Soldier1-01 is attacking Scout2-01 (health=-5)
KILL: Scout2-01 was killed by Soldier1-01
INFO: Team 1 wins!
INFO: Team 1: Scout1-00[-1]  Soldier1-01[3]  
INFO: Team 1: (Still alive: 1)
INFO: Team 2: Scout2-00[-2]  Scout2-01[-5]  
So, after those few, small changes, everything is working correctly. All message boundaries are preserved, so if the client calls sendto 5 times, the server will call recvfrom 5 times with each call retrieving one complete message. That somewhat easily solves the problem we had when we were using SOCK_STREAM.

However, you should still be worried that this might not work in all cases. Referring back to this table:

Propertystream
(SOCK_STREAM)
datagram
(SOCK_DGRAM)
Connection-oriented?YesNo
Message boundaries? NoYes
Reliable? YesNo
Remember what we said about reliability? It means that the messages are guaranteed to be delivered in the exact order they were sent. Also, no message will get lost and no message will be duplicated (received twice or more). This seems like we made a bad trade. With SOCK_DGRAM:
  1. We get messages that preserve the boundaries. Good!
  2. We can't guarantee that messages will get delivered in order or at all. Bad!
If this is true, then this "solution" is not going to work. However, the solution does work and will always work.

The reason this works is that when you are sending datagrams (SOCK_DGRAM) to another process on the same computer, then you are guaranteed all of the reliability that SOCK_STREAM guarantees. If, however, you send datagrams across a network to another computer, you will get no such guarantees. (Of course, there are ways to fix this, as well.)

Recall the socket domains from before. All of our code had this line:
  /* Set the domain type (same system) */
addr.sun_family = AF_UNIX;
which meant that we were communicating between processes on the same system. So we have a reliable socket that preserves message boundaries. Problem solved!


Connected Datagram Sockets

Even though datagram (SOCK_DGRAM) sockets are connectionless, it is still possible to call connect on them. It allows the kernel to record the destination address on the other end. This allows us to send datagrams through the socket using write and retrieve them using read. These system calls are simpler and may lead to better performance. However, from the book: The Linux Programming Interface (56.6.2):

On some TCP/IP implementations, connecting a datagram socket to a peer yields a performance improvement ([Stevens et al., 2004]). On Linux, connecting a datagram socket makes little difference to performance.

To demonstrate, we only have to make a few minor changes to our client/server code.

In the client, here are the changes. Notice the #define and #ifdefs:

#define USE_READ_WRITE

static int sfd;                 /* socket file descriptor (server)  */
static struct sockaddr_un addr; /* address (filename) of the socket */

void setup()
{
    /* Create the socket */
  sfd = socket(AF_UNIX, SOCK_DGRAM, 0);
  if (sfd == -1)
  {
    perror("socket");
    exit(1);
  }

    /* Clear out the structure */
  memset(&addr, 0, sizeof(struct sockaddr_un));

    /* Set domain type */
  addr.sun_family = AF_UNIX;

    /* Set pathname (safely) */
  strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);

  #ifdef USE_READ_WRITE
      /* A "connected datagram socket" which allows us to use read/write */
    if (connect(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un)) == -1)
    {
      perror("connect");
      exit(2);
    }
  #endif
}
The client's modified send function:
void send(int level, const std::string& string)
{
  ssize_t len;                 /* count of bytes read from stdin */
  char buffer[BUF_SIZE] = {0}; /* to transfer data in the socket */

    /* No error checking done on string length! */
  sprintf(buffer, "%i%s", level, string.c_str());
  len = strlen(buffer);

  #ifdef USE_READ_WRITE
  if (write(sfd, buffer, len) != len)
  {
    perror("write");
    exit(10);
  }
  #else
  if (sendto(sfd, buffer, len, 0, (struct sockaddr *) &addr, sizeof(struct sockaddr_un)) != len)
  {
    perror("sendto");
    exit(10);
  }
  #endif
}
The only modification on the server:
  /* Process incoming connections forever */
for (;;)
{
  #ifdef USE_READ_WRITE
    ssize_t bytes_read = read(sfd, buffer, BUF_SIZE);
    if (bytes_read == -1)
    {
      perror("read");
      return 5;
    }
  #else
    ssize_t bytes_read = recvfrom(sfd, buffer, BUF_SIZE, 0, NULL, NULL);
    if (bytes_read == -1)
    {
      perror("recvfrom");
      return 5;
    }
  #endif
  
  display(buffer, bytes_read);
}


Additional Features

Although this has nothing to do with sockets or operating systems, I still want to show it. The log server can do pretty much anything it wants with the messages. Displaying them to the screen was just a simple way to demonstrate that. I also said we could play a sound, send an email/text, or write to a file.

Here's a simple demonstration showing that we can colorize the output. Since we have 3 different "levels" or "types" of messages, we can print in 3 different colors: white for "normal" messages, yellow for "attacks" and red for "kills". This is what the output might look like if we added this ability to the logging server:

INFO: Team 1: INFO: I'm a Soldier named Soldier1-00 [49,5,5] INFO: I'm a Soldier named Soldier1-01 [46,6,8] INFO: Team 2: INFO: I'm a Soldier named Soldier2-00 [47,7,6] INFO: I'm a Pyro named Pyro2-01 [68,8,12] SHOT: Pyro2-01 is attacking Soldier1-01 (health=38) SHOT: Soldier2-00 is attacking Soldier1-01 (health=32) SHOT: Soldier1-01 is attacking Soldier2-00 (health=39) SHOT: Pyro2-01 is attacking Soldier1-00 (health=41) SHOT: Pyro2-01 is attacking Soldier1-01 (health=20) SHOT: Soldier1-01 is attacking Soldier2-00 (health=33)        SHOT: Pyro2-01 is attacking Soldier1-00 (health=33) SHOT: Soldier1-00 is attacking Pyro2-01 (health=63) SHOT: Pyro2-01 is attacking Soldier1-00 (health=21) SHOT: Soldier1-00 is attacking Soldier2-00 (health=28) SHOT: Soldier1-00 is attacking Soldier2-00 (health=23) SHOT: Pyro2-01 is attacking Soldier1-01 (health=8) SHOT: Soldier1-01 is attacking Soldier2-00 (health=15) SHOT: Soldier2-00 is attacking Soldier1-01 (health=2) SHOT: Soldier2-00 is attacking Soldier1-01 (health=-5) KILL: Soldier1-01 was killed by Soldier2-00 SHOT: Soldier1-00 is attacking Soldier2-00 (health=10) SHOT: Soldier2-00 is attacking Soldier1-00 (health=15) SHOT: Soldier1-00 is attacking Pyro2-01 (health=58) SHOT: Soldier1-00 is attacking Pyro2-01 (health=53) SHOT: Soldier1-00 is attacking Soldier2-00 (health=5) SHOT: Pyro2-01 is attacking Soldier1-00 (health=3) SHOT: Pyro2-01 is attacking Soldier1-00 (health=-5) KILL: Soldier1-00 was killed by Pyro2-01 INFO: Team 2 wins! INFO: Team 1: Soldier1-00[-5] Soldier1-01[-5] INFO: Team 2: Soldier2-00[5] Pyro2-01[53] INFO: Team 2: (Still alive: 2)

Much easier to see the different types of messages. Of course, this is a contrived example. The point, again, is that the "game" (or whatever application) doesn't have to include any of this logic, keeping the game simpler.

If you've never worked with colorized output in a terminal, you can find a quick tutorial here: Mead's Guide to Colorized Console Output on Linux/Unix (MacOS) using ANSI escape sequences.

The logging server (syslog) on Linux (and many other operating systems) has levels based on SEVERITY (how severe the issue is). On Linux, there are 8 levels ranging from 0 (emergency, the system is unusable) to 7 (debug messages).

Super-critical messages, in addition to being written to a file, will likely be sent directly to a system administrator (phone, terminal, etc.), while non-critical messages will just be logged to a file for later inspection.


Sending a File

Suppose you wanted to send an entire file (or a portion) over the socket? You could just use read/write (or send/recv), but that may be inefficient. There is another way to do that using sendfile. And, like its name, that's what it does. It sends a file to the receiver. Here's the prototype:

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);
where: This is a diagram showing the differences between read/write and sendfile:
The Linux Programming Interface ©2010  
Here's a sample client that demonstrates the use. It's a modified version of the first client that I showed above. The server is the same one that we started with above and is unchanged.

Check out the manpage for sendfile for all of the details.


Monitoring Sockets with netstat and the /proc filesystem

Running the command:

netstat --all --unix
shows all of the UNIX domain sockets that are currently in-use:
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node     Path
unix  2      [ ACC ]     STREAM     LISTENING     28814      /tmp/tmux-1000/default
unix  2      [ ACC ]     STREAM     LISTENING     169463556  /var/run/cups/cups.sock
unix  2      [ ]         DGRAM                    170410361  log-socket
unix  2      [ ACC ]     STREAM     LISTENING     60271780   @/tmp/dbus-yQwN9GkA9L
unix  2      [ ACC ]     STREAM     LISTENING     12788      /tmp/mongodb-27017.sock
unix  2      [ ACC ]     STREAM     LISTENING     60269057   /tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     60269056   @/tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     11813      /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     9933       /tmp/.vbox-chico-ipc/ipcd
unix  2      [ ACC ]     STREAM     LISTENING     11048      /var/run/avahi-daemon/socket
unix  2      [ ACC ]     STREAM     LISTENING     170405237  log-socket
unix  2      [ ACC ]     STREAM     LISTENING     60317280   @/tmp/dbus-Fvv9eOMEyp
unix  2      [ ]         DGRAM                    15742      /var/lib/samba/private/msg.sock/1497
unix  3      [ ]         SEQPACKET  CONNECTED     142440140  @00184
unix  3      [ ]         STREAM     CONNECTED     65068050   @00081
unix  2      [ ACC ]     STREAM     LISTENING     65062501   socket
unix  3      [ ]         SEQPACKET  CONNECTED     132875789  @00170
unix  3      [ ]         STREAM     CONNECTED     143180645 
unix  3      [ ]         STREAM     CONNECTED     142447833 
unix  3      [ ]         STREAM     CONNECTED     142243609  @/tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     132883155 
unix  3      [ ]         STREAM     CONNECTED     132875226 
unix  3      [ ]         STREAM     CONNECTED     60270628 

[1800+ entries removed ...]
I ran netstat after running both versions of the logserver (SOCK_STREAM and SOCK_DGRAM), which you can see in the output. The @ in front of the path represents the abstract sockets, which, if you recall, is just a path that has a leading NUL byte.

You'll also notice that I had over 1800 lines in the output. That shows you just how much the operating system and processes rely on UNIX domain sockets. They are ubiquitous and they are everywhere.

You can also see if a process is using any sockets by looking in the /proc pseudo-filesystem for the processes PID. You can get the PID from netstat, as well:

netstat --all --unix --program 
Output: (showing just the relevant line)
Proto RefCnt Flags       Type       State         I-Node     PID/Program name    Path  
unix  2      [ ACC ]     STREAM     LISTENING     170405237  6001/logserver      log-socket
We can see the name of the executable and its PID. We can see all of this processes open descriptors (files, sockets, etc.) by running this command:
ls -l /proc/6001/fd
and the output:

lrwx------ 1 chico chico 64 Nov 23 08:02 0 -> /dev/pts/41 lrwx------ 1 chico chico 64 Nov 23 08:02 1 -> /dev/pts/41 lrwx------ 1 chico chico 64 Nov 22 09:48 2 -> /dev/pts/41 lrwx------ 1 chico chico 64 Nov 23 08:02 3 -> socket:[170405237]    

You can clearly see that the socket is present and the inodes match. I ran the command on Firefox's PID and got a list of 172 open descriptors! Among the descriptors were 63 sockets and 26 pipes. The other 80 or so were open files, mostly SQLite database files. Try it on your own and see what you get.


Prelude to Networking

We've been focusing on UNIX domain sockets, which, as you may recall, was for connecting processes on the same computer, as opposed to connecting to another computer across a network (including the Internet). Sockets can also be used for the latter: connecting processes on different systems.

As a quick look into this, we'll modify the original client1.c and server1.c code to allow connections over the network. There are only a few minor changes in both the client (about 5 lines) and server (about 5 lines) that must be made in order for this to work.

This is what's in socktest2.h that is shared between the client and server:

#define BUF_SIZE 100
#define PORT_NUMBER 12345
These are the primary changes in the server:
Original server1 (AF_UNIX)New server2 (AF_INET)
  /* address for connecting */  
struct sockaddr_un addr = {0};

  /* Create the server socket */
sfd = socket(AF_UNIX, SOCK_STREAM, 0);

  /* Set the domain type (same system) */
addr.sun_family = AF_UNIX;

  /* Set the pathname (safely) */
strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);
  /* address for connecting */
struct sockaddr_in addr = {0};

  /* Create the server socket */
sfd = socket(AF_INET, SOCK_STREAM, 0);

  /* Set the domain type (Internet) */
addr.sin_family = AF_INET;


  /* Set the address and port */
addr.sin_addr.s_addr = INADDR_ANY;  /* Any address */
addr.sin_port = htons(PORT_NUMBER); /* Port number */
These are the primary changes to the client:
Original client1 (AF_UNIX)New client2 (AF_INET)
  /* address for connecting */
struct sockaddr_un addr = {0};

  /* Create the socket */
sfd = socket(AF_UNIX, SOCK_STREAM, 0);

  /* Set domain type */
addr.sun_family = AF_UNIX;

 /* Set pathname (safely) */
strncpy(addr.sun_path, SOCKET_PATH, sizeof(addr.sun_path) - 1);
  /* address for connecting */
struct sockaddr_in addr = {0};

  /* Create the socket */
sfd = socket(AF_INET, SOCK_STREAM, 0);

  /* Set domain type (Internet) */
addr.sin_family = AF_INET;

  /* Set the address and port */
addr.sin_addr.s_addr = inet_addr(argv[1]); /* IP address  */
addr.sin_port = htons(PORT_NUMBER);        /* Port number */
When the server runs, it will bind its socket to its IP address and port 12345. (The header file has the port number so the client and server are on the same page.)

When the client runs, it must provide the IP address of the server on the command line. We could have hard-coded that IP address, but that would make the program work with only one computer on the network. Not very interesting.

So, if our server is running on a system with an IP address of 192.168.5.115, then the client would connect like this:

./client2 192.168.5.115
Now, anything that is typed from the command line will be sent to the sever. Remember, the client reads from stdin and writes to the socket. We could also do something like this:
./client2 192.168.5.115 < somefile
Where somefile is a file that will be redirected to the client's stdin.

Here are the complete listings for client2.c and server2.c so you can see all of the relevant details.