Archive

Posts Tagged ‘select’

Non-blocking TCP client in C, where we can send and receive data at every moment

January 10, 2014 No comments

If you are creating a program which interacts with a server, this may interest you. First, we’re going to make a simple TCP client, we connect to a server and it will allow us to send whatever we write on our keyboard. We can’t receive anything from the server right now, it’s just a first step to reach our goal:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <errno.h>
#include <unistd.h>

#define MENS_MAX_LEN 500

void usage()
{
  fprintf (stderr, "Wrong arguments. Must give two:\n");
  fprintf (stderr, "tcpclient SERVER PORT\n\n");
  fprintf (stderr, "For example:\n");
  fprintf (stderr, "  tcpclient totaki.com 80\n");
  exit(1);
}

void panic(char *msg)
{
  fprintf (stderr, "Fatal error: %s (errno %d, %s)\n", msg, errno, strerror(errno));
  exit(2);
}

int main(int argc, char *argv[])
{
  char *server;
  int port;
  int socketfd;
  int finish = 0;
  struct sockaddr_in serverAddress, clientAddress;
  struct hostent *h;
  char mens [MENS_MAX_LEN];
  size_t recvsize;

  if (argc<3)
    usage();

  server = argv[1];
  port = atoi(argv[2]);

  /* Create a TCP socket */
  socketfd = socket(AF_INET, SOCK_STREAM, 0);
  if (socketfd==-1)
    panic("Failed to create socket");

  /* Sets client address */
  clientAddress.sin_family = AF_INET;
  clientAddress.sin_addr.s_addr = htonl(INADDR_ANY);
  clientAddress.sin_port = htons(0);

  /* Bind the address to the socket */
  if (bind(socketfd, (struct sockaddr *)&clientAddress, sizeof(clientAddress))==-1)
    panic("Binding address");

  /* Get the hostname address */
  h = gethostbyname(server);

  /* Sets server address */
  serverAddress.sin_family = h->h_addrtype;
  memcpy((char*) &serverAddress.sin_addr.s_addr, h->h_addr_list[0], h->h_length);
  serverAddress.sin_port = htons(port);

  if (connect(socketfd, (struct sockaddr * )&serverAddress, sizeof(serverAddress))==-1)
    panic("Cannot connect");

  do
    {
      fgets(mens, MENS_MAX_LEN, stdin);

      if (send(socketfd, mens, strlen(mens)+1, 0)==-1)
    panic ("Cannot sent message");

    } while (!finish);      /* Never finish */

  return EXIT_SUCCESS;
}

We’ll analyze the main() function directly, because usage() just display text and panic() makes the program exit on an error.
The first thing we do is to determine the server and port we will connect to, they both will be the first and second arguments to our program. Then we create the socket and build the client address. The port is 0 to let the system pick any port, we don’t mind which port is picked. Then bind() will assign the address to the socket.

Then, create the server address. We will use getbyhostname() before because we may resolve the server address first (if it comes with a hostname, or even if it comes as a IP address we will have to translate this address to a suitable format), so let’s connect() and begin sending messages.

In this case we’re using fgets() to ask the user for information through the standard input (usually keyboard) and then send() it, we must set the size to strlen(mens)+1 to include the string terminator ‘\0′ at the end of the string being sent.

Note: fgets() will send the line feed at the end when we press return. We can use a trim() function or filter it with:

1
2
if (mens[strlen(mens)-1]=='\n')
   mens[strlen(mens)-1] = '\0';

And to receive a message from the server, we must:

1
2
3
4
5
size_t recvsize;
recvsize = recv(socketfd, mens, MENS_MAX_LEN, 0);
if (recvsize==-1)
  panic("Cannot receive information");
mens[recvsize] = '\0';

Reception is similar to sending, but this time we don’t know the size of the message, so we provide a maximum size (or the size of our buffer), if the size of the data we are receiving is greater than our buffer, we can continue the message calling again and again recv().
As this message may not have a terminator, to display it properly on screen with printf() we must put the terminator manually at the end of the string.

It’s easy. The biggest problem, comes when we want the application to send and receive information whenever there is data on each side (sending or reception), making our application full duplex. With this code, right now, we just can give turns, make the user send information sometimes, and sometimes the application will receive data.
We will make a more complex example using select(), this function will detect whenever there is data waiting on some descriptors, and we can later choose whether to read the socket or the keyboard. But the most interesting is the timeout, we can give select() a wait timeout, when the function will stop waiting and the program can continue, it won’t block indefinitely waiting for data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <errno.h>
#include <unistd.h>

#define MENS_MAX_LEN 500

void usage()
{
  fprintf (stderr, "Wrong arguments. Must give two:\n");
  fprintf (stderr, "tcpclient SERVER PORT\n\n");
  fprintf (stderr, "For example:\n");
  fprintf (stderr, "  tcpclient totaki.com 80\n");
  exit(1);
}

void panic(char *msg)
{
  fprintf (stderr, "Fatal error: %s (errno %d, %s)\n", msg, errno, strerror(errno));
  exit(2);
}

int main(int argc, char *argv[])
{
  char *server;
  int port;
  int socketfd;
  int finish = 0;
  struct sockaddr_in serverAddress, clientAddress;
  struct hostent *h;
  char mens [MENS_MAX_LEN];
  fd_set readmask;
  struct timeval timeout;
  size_t recvsize;

  if (argc<3)
    usage();

  server = argv[1];
  port = atoi(argv[2]);

  /* Create a TCP socket */
  socketfd = socket(AF_INET, SOCK_STREAM, 0);
  if (socketfd==-1)
    panic("Failed to create socket");

  /* Sets client address */
  clientAddress.sin_family = AF_INET;
  clientAddress.sin_addr.s_addr = htonl(INADDR_ANY);
  clientAddress.sin_port = htons(0);

  /* Bind the address to the socket */
  if (bind(socketfd, (struct sockaddr *)&clientAddress, sizeof(clientAddress))==-1)
    panic("Binding address");

  /* Get the hostname address */
  h = gethostbyname(server);

  /* Sets server address */
  serverAddress.sin_family = h->h_addrtype;
  memcpy((char*) &serverAddress.sin_addr.s_addr, h->h_addr_list[0], h->h_length);
  serverAddress.sin_port = htons(port);

  if (connect(socketfd, (struct sockaddr * )&serverAddress, sizeof(serverAddress))==-1)
    panic("Cannot connect");

  do
    {
      /* We must set all this information on each select we do */
      FD_ZERO(&readmask);   /* empty readmask */
      /* Then we put all the descriptors we want to wait for in a */
      /* mask = readmask */
      FD_SET(socketfd, &readmask);
      FD_SET(STDIN_FILENO, &readmask); /* STDIN_FILENO = 0 (standard input) */
      /* Timeout, we will stop waiting for information */
      timeout.tv_sec=0;
      timeout.tv_usec=100000;

      /* The first parameter is the biggest descriptor+1. The first one
       was 0, so every other descriptor will be bigger.*/

      /* readfds = &readmask */
      /* writefds = we are not waiting for writefds */
      /* exceptfds = we are not waiting for exception fds */
      if (select(socketfd+1, &readmask, NULL, NULL, &timeout)==-1)
    panic("Error on SELECT");

      /* If something was received */
      if (FD_ISSET(socketfd, &readmask))
    {
      recvsize = recv(socketfd, mens, MENS_MAX_LEN, 0);
      if (recvsize==-1)
        panic("Cannot receive information");
      mens[recvsize] = '\0';
      printf (">> %s\n", mens);
    }

      /* If something was written by the user */
      if (FD_ISSET(STDIN_FILENO, &readmask))
    {
      fgets(mens, MENS_MAX_LEN, stdin);

      if (send(socketfd, mens, strlen(mens)+1, 0)==-1)
        panic ("Cannot sent message");
    }
    } while (!finish);

  return EXIT_SUCCESS;
}

Photo: Julien Gong Min (Flickr) CC-by

MySQL loops and cursors with examples

December 16, 2013 No comments

First of all, you may not abuse of these techniques, and you must use them only when necessary. Most of the times you can find a faster solution.

Let’s see an easy loop, similar to a for loop, we will make a SELECT x with x from 1 to 9:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
DELIMITER $$
CREATE PROCEDURE simple_loop ( )
BEGIN
  DECLARE counter BIGINT DEFAULT 0;
 
  my_loop: LOOP
    SET counter=counter+1;

    IF counter=10 THEN
      LEAVE my_loop;
    END IF;

    SELECT counter;

  END LOOP my_loop;
END$$
DELIMITER ;

if we do:

1
CALL simple_loop();

We’ll see something like

+———+
| counter |
+———+
| 1 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 2 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 3 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 4 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 5 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 6 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 7 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 8 |
+———+
1 row in set (0.01 sec)

+———+
| counter |
+———+
| 9 |
+———+
1 row in set (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

The code we iterate is between LOOP…END LOOP. What we see right before (my_loop) is a label, just to give a name to that loop, and reference it. In this example, we simply increment counter variable, and with a condition we can exit the loop when this variable reaches the value 10. We won’t see the 10 because we leave the loop before printing this number.

Let’s do something a bit more complicated, we will record scores in a game, this game will be a skill test to do in the shortest time as possible, hopscotch jumping and with obstacles. There are two types of penalty, touch the ground with both feet and hitting an obstacle. At the end of the tests the results will be written on a table, and a score will be assigned, this score will be also stored on that table to avoid calculating it each time.

1
2
3
4
5
6
7
8
9
CREATE TABLE Runners (
    Runner_id BIGINT NOT NULL AUTO_INCREMENT,
    Name VARCHAR(120) NOT NULL,
    Time BIGINT NOT NULL,
    Penalty1 BIGINT NOT NULL,
    Penalty2 BIGINT NOT NULL,
    Points BIGINT,
    PRIMARY KEY (Runner_id)
) ENGINE=InnoDB DEFAULT CHARSET=UTF8;

Now, enter some test information:

1
2
3
4
5
6
7
INSERT INTO Runners VALUES (NULL, 'Michael', 123, 5, 2, NULL);
INSERT INTO Runners VALUES (NULL, 'Sarah', 83, 3, 3, NULL);
INSERT INTO Runners VALUES (NULL, 'John', 323, 1, 1, NULL);
INSERT INTO Runners VALUES (NULL, 'Ramon', 100, 8, 4, NULL);
INSERT INTO Runners VALUES (NULL, 'Andrew', 143, 4, 3, NULL);
INSERT INTO Runners VALUES (NULL, 'Antoine', 199, 3, 2, NULL);
INSERT INTO Runners VALUES (NULL, 'David', 101, 2, 1, NULL);

The first thing to do is a stored procedure with the basic loop, and a SELECT in each iteration, just to see that we are doing ok. (I will explain the code later):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
DROP PROCEDURE IF EXISTS cursorTest;
DELIMITER $$
CREATE PROCEDURE cursorTest (
) BEGIN
-- Variables where we will store what the SELECT returns
  DECLARE v_name VARCHAR(120);
  DECLARE v_time BIGINT;
  DECLARE v_penalty1 BIGINT;
  DECLARE v_penalty2 BIGINT;
-- Variable to control the end of the loop
  DECLARE fin INTEGER DEFAULT 0;

-- El SELECT to query
  DECLARE runners_cursor CURSOR FOR
    SELECT Name, Time, Penalty1, Penalty2 FROM Runners;

-- Exit condition
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET fin=1;

  OPEN runners_cursor;
  get_runners: LOOP
    FETCH runners_cursor INTO v_name, v_time, v_penalty1, v_penalty2;
    IF fin = 1 THEN
       LEAVE get_runners;
    END IF;

  SELECT v_name, v_time, v_penalty1, v_penalty2;

  END LOOP get_runners;

  CLOSE runners_cursor;
END$$
DELIMITER ;

Must have in mind something. With the CURSOR we will go through the result of a SELECT statement, and we’ll have to store the values returned for each row in variables (that’s why I declared v_name, v_time, v_penalty1 and v_penalty2). In the end, each iteration will do a SELECT Name, Time, Penalty1, Penalty2 INTO v_name, v_time, v_penalty1, v_penalty2 WHERE …, so we’ll have these variables filled with the data obtained for each row in each iteration. That’s DECLARE xxx CURSOR FOR SELECT …

We must put a finish condition, usually we will finish the loop when no more results are found, that’s why we use DECLARE CONTINUE HANDLER FOR NOT FOUND SET fin=1, In this case, we will set fin to 1 when no more rows are found.

Inside the loop, we will test the value of this variable and LEAVE the loop if fin is set to 1, it’s automatically done when we reach the condition given before.

One step more, let’s create a function to assign the scores to each one of the runners with a formula. For example, if Time is the time taken in seconds, 500-Time will be the initial score and we will take away 5*penalty+3*penalty2. So:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
DROP FUNCTION IF EXISTS calculate_runner_points;
DELIMITER $$
CREATE FUNCTION calculate_runner_points (
  In_time BIGINT,
  In_penalty1 BIGINT,
  In_penalty2 BIGINT
) RETURNS BIGINT
BEGIN
  DECLARE points BIGINT;
 
  SET points = 500 - In_time - In_penalty1*5 - In_penalty2*3;
 
  RETURN points;
END$$
DELIMITER ;

Now, this is the code to calculate the final score of all players:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
DROP PROCEDURE IF EXISTS calculate_all_points;
DELIMITER $$
CREATE PROCEDURE calculate_all_points (
) BEGIN
-- Variables where we will store what the SELECT returns
  DECLARE v_name VARCHAR(120);
  DECLARE v_time BIGINT;
  DECLARE v_penalty1 BIGINT;
  DECLARE v_penalty2 BIGINT;
  DECLARE v_runner_id BIGINT;
-- Variable to control the end of the loop
  DECLARE fin INTEGER DEFAULT 0;

-- SELECT to query
  DECLARE runners_cursor CURSOR FOR
    SELECT Runner_id, Name, Time, Penalty1, Penalty2 FROM Runners;

-- Exit condition
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET fin=1;

  OPEN runners_cursor;
  get_runners: LOOP
    FETCH runners_cursor INTO v_runner_id, v_name, v_time, v_penalty1,
v_penalty2;
    IF fin = 1 THEN
       LEAVE get_runners;
    END IF;

  UPDATE Runners SET Points=calculate_runner_points(v_time, v_penalty1,
v_penalty2) WHERE Runner_id=v_runner_id;

  END LOOP get_runners;

  CLOSE runners_cursor;
END$$
DELIMITER ;

Of course, as I said in the beginning of this post we must see if there is no other method to do the same. I know using MySQL loops is amazing, as I always say with regex, but there may be faster methods, like:

1
UPDATE Runners SET Points=calculate_runner_points(Time, Penalty1, Penalty2);

But, we can do more things with the loop, for example, if the time is greater than 250, we can swap penalties, editing the loop, with a IF statement, we could also do it in the procedure, but that’s an option.

A small example (or not so small) come to mi mind. Imagine we have a user system. Each user has its information stored in three tables: one for login, password and access info; another one for profile data and the last one for permissions. In this cas, all tables except permission table has one row per user. But as the permission table stores the access level for a user and another object (maybe a web page), and one single user can have permissions over several pages, there may be some rows for one user.

We will also have a table to store messages between users.

We will also have pages, these pages will be objects in our system, and users can see , edit, create derivates and delete them (if they are allowed to). A hierarchy may exist with pages, so we can have child pages, but when we create a new page:

  • If a user is allowed to edit a parent page, will be allowed to edit the new child page
  • If a user could create derivatives in a parent, will be also allowed in the child
  • If a user was allowd to edit and create derivatives in the parent page, will also be allowed to remove the child page.
  • We will have to send a message to the user telling him or her, about the new page and what is allowed to do with it.
  • We also have these procedures and functions:
    • can_create_derivatives(user, page) – It will return TRUE if the user can create derivatives
    • can_edit(user, page) – It will do the same, but with edit permission
    • new_permission(user, page, permission) – It will allow the user to do what permission says with this page
    • message(from, to, message) – Send a message to a user.

Functions can_create_derivatives() and can_edit() seem to be easy to understand, but what they do internally is far more complicated. It’s done by a colleague and I don’t want to fight with it. The same with new_permission() (it can insert rows or updates existent ones) or message(), it can send notifications and create a job to send real e-mail, so our procedure may be something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
DROP PROCEDURE IF EXISTS create_page;
DELIMITER $$
CREATE PROCEDURE create_page (
  IN in_name VARCHAR(120),
  IN in_parent BIGINT
) BEGIN
  DECLARE v_user_id BIGINT;
  DECLARE v_create_derivatives TINYINT;
  DECLARE v_object_id BIGINT;
  DECLARE v_msg TEXT;

-- Variable to control the end of the LOOP
  DECLARE fin INTEGER DEFAULT 0;

-- The SELECT
  DECLARE users_cursor CURSOR FOR
    SELECT User_id FROM Users;

-- Exit condition
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET fin=1;

  INSERT INTO Pages (Name, Parent) VALUES (in_name, in_parent);
  SELECT LAST_INSERT_ID() INTO v_object_id;

  OPEN users_cursor;
  get_users: LOOP
    FETCH users_cursor INTO v_user_id;

    IF fin = 1 THEN
       LEAVE get_users;
    END IF;

    SET v_msg = CONCAT('New permissions on page ',in_name,': ');  

    IF can_create_derivatives(v_user_id, in_parent) THEN
      CALL new_permission(v_user_id, v_object_id, 'derivatives');
      SET v_msg = CONCAT(v_msg, 'Create derivatives ');
      SET v_create_derivatives=1;
    ELSE
      SET v_create_derivatives=0;
    END IF;

    IF can_edit(v_user_id, in_parent) THEN
      CALL new_permission(v_user_id, v_object_id, 'edit');
      SET v_msg = CONCAT(v_msg, 'Edit ');
      IF v_create_derivatives=1 THEN
         CALL new_permission(v_user_id, v_object_id, 'remove');
         SET v_msg = CONCAT(v_msg, 'Remove ');
      END IF;
    END IF;

    CALL message(1, v_user_id, v_msg);
  END LOOP get_users;

  CLOSE users_cursor;
END$$
DELIMITER ;

I’m sure I can give you better examples in the future. I accept suggestions in comments, so I can recreate them in my computer and solve them in future posts.

Photo: Kellie Bollaret (Flickr CC) Licensed CC-by

Top