Archive

Archive for the ‘Linux’ Category

Starting with concurrency and a “Hello world” using pthreads

December 31, 2013 No comments


Nowadays is very common to see dual-core or quad-core CPUs, but there are still systems with one single core.
In this post, I want to about the very basis of concurrency, a brief introduction to this world, so some descriptions will be so so simple, but real world is a little bit more complex.

In multi-core or multi-CPU systems, the operating system (OS) distributes tasks within our CPUs, and we can run several tasks at the same time, but we have been enjoying multitasking for decades, long before multi-core appeared, so, how is it possible? Let’s think about single-core systems, they can only perform one task, but the operating system distributes the CPU control along all tasks very fast so it makes us think all of them are running at the same time. So single-core CPUs usually have one execution thread, dual-core system, will have two execution threads, so they can perform two tasks at the same time. But don’t worry about how it is distributed, because the operating system does it for you.

We’ve seen the OS is smart enough to take advantage of our CPUs or CPU cores, so does it have any sense to make our application use several cores, or make our application run two things at the same time?
Of course it does, let’s see some examples:

  • Imagine a GUI program performing a heavy task. While this task is running, we may want to give the user some little control to cancel the process, for example, or to carry on using our software. We can run the heavy task in one thread and the GUI in another one.
  • A server which attends several clients simultaneously.
  • A program which does intense calculations, if we have dual-core, it can be twice faster

In the last case, we must think about something a little bit. Distributing tasks in time takes some time for the OS, so we must make sure the time we save (separating the process in several threads) is bigger than the time the OS wastes managing the new tasks and sometimes we must put all the results of all individual threads together (and it takes time too).
We may think it will be valid only for multi-core processors but the time taken by our tasks may vary depending on what we are doing, for example, calculations take all CPU Time, but if we access external devices (keyboard, screen, hard disk, USB, etc), even access a file, it makes our process wait for the data and while we are waiting, the CPU is wasting time, so we can take this time to perform calculations for some other process, and it is valid even on single-core CPUs. Sometimes running a tasks in two threads can squeeze our CPU better than one thread using CPU while the other thread is just waiting for data an vice-versa.

Ok, before playing a little bit with it we must know there are some ways to do that. We can create several processes, it’s like running some programs at the same time, and the other way is creating threads. The difference is that processes must store in RAM memory, data and stack among other things, and some threads can share code and data, so only stack and a little information about execution status belongs to each thread exclusively. We can say process initiates also a single execution thread.

Let’s code a bit, we will use pthreads (POSIX threads):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

void *newtask(void *null)
{
   printf("Hello world! I'm a new thread\n");
   sleep(1);
   printf("Goodbye world! I'm a thread about to die\n");
   pthread_exit(NULL);
}

int main (int argc, char *argv[])
{
   pthread_t thread;
   int rc;

   printf ("Main process just started.\n");
   rc = pthread_create(&thread, NULL, newtask, NULL);
   if (rc)
     {
       printf("ERROR in pthread_create(): %d\n", rc);
       exit(-1);
     }

   printf ("Main process about to finish.\n");
   /* Last thing that main() should do */
   pthread_exit(NULL);
}

This example can be compiled including pthread library, this way:

$ gcc -o onethread onethread.c -lpthread

We are just executing a program that creates another thread that performs a task, this task is writing a couple of messages on screen and wait a bit between them. We can see a result like this:

Main process just started.
Main process about to finish.
Hello world! I’m a new thread
[ wait for a second ]
Goodbye world! I’m a thread about to die

The first two messages belongs to the main thread, one when it stats and the other one when it ends, but before ending we are invoking the other thread that writes the other two messages. As we can see, each thread is executed independently, but to see it a bit more clearer, let’s see the next code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

void *newtask(void *null)
{
   printf("Hello world! I'm a new thread\n");
   int i;

   for (i=0; i<10;++i)
     {
       printf (" THREAD: %d\n", i);
       usleep(60);
     }
   printf("Goodbye world! I'm a thread about to die\n");
   pthread_exit(NULL);
}

int main (int argc, char *argv[])
{
   pthread_t thread;
   int rc;
   int i;

   printf ("Main process just started.\n");
   rc = pthread_create(&thread, NULL, newtask, NULL);
   if (rc)
     {
       printf("ERROR in pthread_create(): %d\n", rc);
       exit(-1);
     }

   for (i=0; i<10;++i)
     {
       usleep(100);
       printf (" MAIN: %d\n", i);
     }
   printf ("Main process about to finish.\n");
   /* Last thing that main() should do */
   pthread_exit(NULL);
}

What we’ve done is that the main thread (MAIN) and secondary thread (THREAD) will write on screen several messages inside a for loop. They will be counting from 0 to 9, but waiting a little bit between each writing (instead of doing heavier time eater tasks we simulate them with sleeps).

We can see a execution result like this:

Main process just started.
Hello world! I’m a new thread
THREAD: 0
THREAD: 1
MAIN: 0
THREAD: 2
MAIN: 1
THREAD: 3
THREAD: 4
MAIN: 2
THREAD: 5
MAIN: 3
THREAD: 6
MAIN: 4
THREAD: 7
THREAD: 8
MAIN: 5
THREAD: 9
MAIN: 6
Goodbye world! I’m a thread about to die
MAIN: 7
MAIN: 8
MAIN: 9
Main process about to finish.

Messages are written alternatively both MAIN and THREAD, like interlaced, now we can see they are both running concurrently, at the same time. We can start making multi-thread software in C.

Photo: Toastyken (Flickr) CC-by

Call a variable whose name is contained by another one [BASH]

November 4, 2013 No comments

numbers

I’m sure this has happened to you. Imagine we have three variables (RED = “ff0000″ ; GREEN = “00ff00″ ; BLUE = “0000ff”) and a function which giving the name of the color, it gives us the code. Ok, we can do this many ways, but this is just an example. What I want is to tell get_color() function a name of a variable, and it will give me the value:

1
2
3
4
5
6
7
8
9
10
RED="ff0000"
GREEN="00ff00"
BLUE="0000ff"

function get_color()
{
   echo ${!1}
}

get_color BLUE

The key, as you may see is the ! sign before the name of the variable in braces. It can be a bit more clearer:

1
2
3
4
5
6
7
SEVILLE="Torre del Oro"
LONDON="Big Ben"
NEWYORK="Statue of Liberty"

CITY=SEVILLE

echo I really like to go to $CITY to see ${!CITY}

If I change the value of CITY, it will pick another monument from the list before.

Backup MySQL users

October 28, 2013 No comments

When we are dumping a MySQL database, is normal to ignore “mysql”, “test” and “information_schema” databases. We could have undesirable problems when restoring, for example if we change MySQL version.
But sometimes, when backing up the database we want to save also user table information. If we have several applications using the same database, we should have a user for each application and we don’t want to lose them.

So we could use the query SHOW GRANTS FOR [email protected] in MySQL. We could automate it to give us all user permissions with a little bash script:

1
2
USERINFO="-uroot -ppasswd"
mysql $USERINFO -BNe "select concat('SHOW GRANTS FOR \'',user,'\'@\'',host,'\'; ') from mysql.user where user != 'root'" | mysql $USERINFO -BN | sed 's/$/;/g' > users_backup.sql

In the USERINFO variable, we store login data for MySQL, we can use root user for this.

With the first call, we generate the queries we are going to make to the database to get user privileges generation SQL. We look who are the users, their hosts and we generate for each row the following string:

1
SHOW GRANTS FOR ‘user’@'host’;

In the second call, we make all queries, and it should return for each one of them something like this:

1
2
GRANT USAGE ON *.* TOuser’@'localhost’ IDENTIFIED BY PASSWORD ‘*1C3116A0C895332DE9EE63FFF08863CC9397ED8E’
GRANT ALL PRIVILEGES ON `database`.* TO ‘user’@'
localhost’

If the user has privileges on more tables or databases, they will appear here too.

The third call is to finish each line with ; (semicolon), so MySQL won’t give us problems when restoring them.

To restore the users just do:

1
mysql -uroot -ppaswwd < users_backup.sql

Get the HOME directory in C

October 23, 2013 No comments

In our programs, it’s usual to know where is the user home directory, to read/store configuration files, to search for something, or to know if the program is installed globally or locally.

There is a short function to get it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <pwd.h>

char *getHomeDir()
{
  static char *home = NULL;
 
  if (!home)
    {
      home = getenv("HOME") ;
    }
  if (!home)
    {
      struct passwd *pw = getpwuid(getuid());
      if (pw)
          home = pw->pw_dir ;
    }
  return home;
}

int main(int argc, char *argv[])
{
  printf ("HOME: %s\n", getHomeDir());

  return EXIT_SUCCESS;
}

If we look at getHomeDir() function, the directory we’re looking for is stored in a static variable, as we are not freeing this variable, we will get the value of this variable when we ask again (the user isn’t changing his home). We will ask the system just for the first time.

In the other hand, we have two ways of getting the PATH, the first is with the HOME environment variable, most of the times it will be defined and we can get it, but if it’s not there, we can get this information from /etc/passwd. There is much more information that what we want and we can have a look at the manual of getpwuid() to know what we can get (the user password is not included, muahaha, it isn’t stored there anymore, and if it is in an old or embedded system, you will probably see a hash).

Foto: Neetesh Gupta (Flickr) CC-by.

stermp – conio.h compatible terminal features in Linux (for small projects)

October 15, 2013 No comments

When making CLI programs, sometimes we want to add some colors, or get the position of the cursor, terminal dimension or something like that. If you have been programming for years, you probably would remember conio.h, it was a C library used in the 80′s ? or 90′s to make user friendly terminal environments. And it is still being taught in some courses and even universities (omg). And I decided to create this some years ago, because conio.h was only compatible with MSDOS and Windows.

There are some Linux libraries to do so, or cross-platform, what is even better. But they are not as easy to use as conio.h was. And this was my intention, provide easy of use colors and position for terminal programs.

To use stermp, you just have to add stermp.h / stermp.c files to your project, and compile it, it will add a few (~10Kb) to your project. But it offers you these capabilities:

  • Terminal positioning (go to specific position, and get current terminal position.
  • Gets terminal dimensions (in MSDOS/Win) this is limited to some fixed resolutions, but in *nixes you can have lot of terminal sizes.
  • Text coloring
  • Background coloring
  • Text effects (underline, blink!! (ohh nooo), please don’t abuse)
  • Get keys without waiting for ENTER key
  • Detect whether if a key is pressed or not

To download it, just get stermp.tar (4.7Kb), or go to my github.

Categories: C, Linux Tags:
Top