Posted by: Catalin | November 29, 2008

Linux kernel proper locking – generalities

While having some daunting tasks within printing industry I’ve decided to have some relax (in some working hours break, obviously) grasping, and remembering about proper-locking techniques in Linux kernel as follows:

Facts and issues:

  1. It provides code path (critical sections) syncronization in context of concurrency, and re-entrancy. Any code that does not have re-entrancy, or concurrency protection will encounter the so called race conditions
  2. True concurrency is encountered in symmetric multiprocessing systems (SMP hardware with SMP kernel in this case), but pseudo concurency that may lead to race conditions are give by interrupt handlers, and preemptibile kernels also. In later case different code is not actually executed in the same time (having one CPU), but accesing shared data may cause race conditions.
  3. SMP and UP (uniprocessor) linux kernels are kept distinct. Some locks exists in one type, but not in the other. Though, developer should not care because at compile time some checkings, and translations are performed. Proper locking allways should be performed.

Methods:


1. Atomic operators (uninterruptible operations)

  • atomic_set(), atomic_inc(), atomic_dec(), and others.

2. Spinlocks

  • They are defined in include/linux/spinlock.h
  • when code tries to acquire an unavailable spinlock, the process will continue trying (spinning) until the lock becames available. Obviously it’s eating CPU and the process is not allowed to sleep.
  • API:
    • spinlock_irqsave(&spinlock_t, unsigned long), spinlock_irqrestore(.same.), SMP safe, and interrupt handlers safe.
    • spin_lock(spinlock_t), spin_unlock(spinlock_t) used basically for user-context kernel code unique data (in syscalls), no interaction with interrupts.
    • spin_lock_bh(), spin_unlock_bh(), standard spinlock that disables software interrupts
    • some other, check the code
  • when using spinlocks don’t call from inside a critical section code that may sleep :-) . You’r kernel will definetly explode :-) . (ie copy_from{to}_user(), kmalloc(GPF_KERNEL) etc)
  • don’t protect code that takes much time to execute. In fact, protect only shared data if it’s possible.

3. Semaphores:

  • It’s defined in include/asm/semaphore.h
  • When execution encounters a code section locked by a semaphore, the task will sleep, not spin :-) , therefore semaphores are used when spinlocks are not possible.
  • As semaphores are sleeping locks (not spinning), task context switch happens and much time is wasted, so semaphores are not used with code that takes short time to execute.
  • struct semaphore that describes a semaphore contains an usage count and a list with waiting tasks. Usage count is initialized at semaphore creation and represents the number of tasks that may concurently acquire the semaphore.
  • A semaphore with usage count equal with 1 is called a mutex
  • API:
    • sem_init(struct semaphore &, int usage_count),
    • down(semaphore&), puts task in uninterruptible sleep if usage count is less or equal with 0. After aquiring the semaphore,it decreases the usage count. Marks the begining of critical section.
    • up(semaphore&), increases the usage count, marks the end of critical section.
    • down_interruptibile(sempahore&), returns > 0 if semaphore was not acquired, else is like down(), but is interruptibile

4. Reader/Writer locks:

  • Are are variants of spinlocks, or semaphores.
  • They allows multiple concurently readers, or one single writer with no readers
  • Reader/Writer spinlock:
    • rwlock_t mr_rwlock = RW_LOCK_UNLOCKED;
      read_lock(&mr_rwlock);
      /* critical section (read only) ... */
      read_unlock(&mr_rwlock);
      write_lock(&mr_rwlock);
      /* critical section (read and write) ... */
      write_unlock(&mr_rwlock);
  • Reader/Writer semaphore
    • struct rw_semaphore mr_rwsem;
      init_rwsem(&mr_rwsem);
      down_read(&mr_rwsem);
      /* critical region (read only) ... */
      up_read(&mr_rwsem);
      down_write(&mr_rwsem);
      /* critical region (read and write) ... */
      up_write(&mr_rwsem);
Posted by: Catalin | November 8, 2008

Cosmopolitanism and Corporate ideology – part 1

I will explain Cosmopolitanism first, which is one part of the Cosmopolitanism/Communitarianism dichotomy. This dichotomy stays at the fondation of our International system and world order.

In the same time the corporation and its ideology has at base cosmopolitanist ideas. This is my personal opinion and it is presented here.

So, let us embark to a trip back in time, 2500 years ago in ancient Greece.

At that time, the world was that of the independent city-states. They were not aware in their daily life about countries, or empires. In their understanding, the meaning and essence of life was the city (you should read here – polis). All religious, social, and political activity of people had at its center the city. Whatever was outside the polis, were considered either beasts, or gods. Try to imagine!

So, the meaning, and essence of life was the polis. The city was their Universe, and the value of life was stemming out from the city. The polis was the source of value.

But what happend when the macedoneans got in the scene and conquered the independent cities ? That happend at 338BC, after the battle of Chaeronea. Their view about world entered into a deep crisis. They couldn’t explain anymore the reason of world and life for a while, now they were making part into an empire.

The system of reference that made their life morally meaningful, being a free citizen of polis dissapeared, as cities lost their independence. While moral and political crisis was deepening a new religous atitude emerged in purpose of making their life meaningful. It was the Stoicism.

The differences between men of different cities were not taken into consideration anymore, as it was in the old World Order. Human nature was a part cosmic nature governed by the divine law of nature. “There is one divine universe, one rational human nature, and therefore one appropriate attitude to all men”. The Stoic was the citizen of the cosmos, not of the polis,and the attitude that was been cultivated since was that of a citizen of the one universal city. That citizen is a cosmopolitan.

There is one thing that deserves to be mentioned. Cosmopolitanism doesn’t aims to a world governance, or domination. In fact that was the problem, the world governance – the empire, which generated the Cosmopolitan thought. as a reaction to the old world order collapse.

Being back to our times. I will present in a future post, how in fact the Cosmopolitanism really shaped our international system.

Stepping in into corporation’s ideology, we all know that a corporation is an international system, that is based on bussines activities. Usually it is a transnational, and transcontinental organization which competes more and more with the main actors of the international arena – the states.

Being an international system actor, the corporation is shaped by the written and unwritten laws of the international system, including by its cosmopolitan/communitarian essence of world order.

Bibliography:

- cosmopolitanism – “International Relations Theory, New Normative Approaches” by Chris Brown,

Posted by: Catalin | November 5, 2008

Recruiting and Knowledge management

I’ve found a very interesting fact about HR recruiting in conjunction with Knowledge management that deserves to be shared. First you should read about KM strategies to have a base of ideas.

So, regarding HR, in some firms the recruiting policies are performed in relation with the KM strategy implemented there.

Having this said, it must be stressed out two kinds of recruiting policies depending of the type of KM.

If the KM is centralized, candidates are selected based on “hard” skills. It’s mandatory for a candidate to have a skill portfolio defined in terms of knowledge goals. This is a hard-skills-oriented recruiting policy.

As it’s logically, if the performed KM is de-centralized, the recruiting policy is oriented to “soft” skills candidates, as the knowledge system is implemented through networking of individuals. To conlude, this is a soft-skills-oriented recruiting policy.

Posted by: Catalin | November 4, 2008

Knowledge management strategies

The article is business oriented, but the essence of ideas can be disseminated in conjunction with other kind of activities, such as from academic environments, or from any organizations where a state of knowledge is present.

So, a Knowledge Management (KM) strategy is a pragmatic practice, developed from overall business strategy, having its goals and characterized by its methods and techniques. Since 1995, it has been a distinct discipline and it’s being thought at good Universities.

Furthermore large companies, and especially corporations have resources dedicated to KM, which is a part of HR, IT&C, and Business Strategy department.

Basic KM strategy has 3 sub-processes:

  • Knowledge generation
  • Knowledge maintenance
  • Knowledge distribution,

and are performed in a central or de-central way, in relation with business type. The obvious purpose is to codify, store, disseminate, and allow reuse of knowledge.

Codifying tasks are usually a done using a person-to-document approach, the resulting document being extracted from the person who developed it, it’s made independent of it, and finally re-used for different purposes.

Ancillary, an IT management system is established, for instance a distributed database of documents with a fancy, easy to use, intuitive GUI. In this case the system is a top-down system, with a centralized KM approach because the knowledge is disseminated from one central point (the database) to individual requestors.

Another strategic approach is to make the knowledge management de-centrally. The KM sub processes are performed de-centralized. In this case, the knowledge is closely tied to persons who acquired it. In this context the dissemination of knowledge is done using a person-to-person approach while networks of individuals are made. In fact the entire KM system is implemented by networks of individuals, based on their initiative, not on a clear managerial task. It’s obvious that it does not cost and it’s the often encountered KM system in firms.

Posted by: Catalin | October 22, 2008

How to create a static library using C

Before you start doing your library you should be clear about three concepts:

  • type of linkage (internal, external, and no linkage)
  • type of objects (internal, and external)
  • scope (per function, per program, …)

, and good explanations you find at this location => Linkage <= .

Ok, let’s assume that you already have a libheader.h, and a libdeclared.c files, where in the first one you have all your definitions, and in the second your program declarations.

Compiling and  assembling your library is simple and performed with gcc, obviously! The proper command to do it is:

gcc -g -c -Wall libdeclared.c # it will produce a libdeclared.o file

Now you have an object file and must create an archive with your lib:

ar rcs libdeclared.a libdeclared.o

Good, now you have a static library and you wish to see what objects does your lib has:

nm -s libdeclared.a

So, using your library is quite simple and looks like this:

gcc -o <your_binary_file> <your_file.c> -L<directory_of_your_lib> <yourlib.a>

Enjoy now!

Posted by: Catalin | October 5, 2008

Consultant what?

Some of you have heard about consultants, but don’t have a broad understanding about their activity. So, I’m giving you a scientific view:

A consultant is a highly professional individual, helping one, ore more organizations to develop ultra-competent teams engaged in complex activities. He can be an internal, or external actor, collaborating directly with superior management, human resources departament and the teams.

The role of a such person is to have a comprehensive perspective about the activity of the organization, or team, and discuss its improvements.

Selecting consultants is usualy performed by the higher management and human resources department.

Selection process consists in a set of evaluation steps like:

  1. Evaluate if he has a result oriented personality. In other words, he shouldn’t cope only with group relations and processes, He should tackle the real problems that organization, or team has, and improve activity.
  2. Test if he can quick obtain the higher management credibility and if they are satisfied with him. Will the management take its advice as a good one? Could he influence them using its expertise?
  3. Evaluate if he can have a proactive influence and collaborate with team(s) leader(s) and basic members and improve the main activity.
  4. Evaluate if he is able to halt the current activity flow if it’s needed. A consultant links its name of the client’s success and based on its experience he shouldn’t let continue an activity which goes towards failure, or a mediocre result

–bibliography -  “The GOWER Management Manual”

Posted by: Catalin | October 3, 2008

Double linked lists

Kernel has a good implementation of double linked lists and objectives in this post are to emphasize how can you do the following list operations:

  • Defining/Declaring and Initializing
  • Passing through
  • Addition and deletion of elements

The linked list API is located in include/linux/list.h and the basic MACROs and functions that you need to know about are:

  • INIT_LIST_HEAD()
  • list_add(), list_entry(), and list_del()
  • list_for_each() and list_for_each_safe()

The first thing you should do when you want to use a linked list, is to declare and declare your list node data type. In this case the node is different than any linked list node that you may know outside the kernel.

Therefore a double linked list node would seem like:

struct person {
	char name[64];
	// ..

	struct person *next;
        struct person *prev;
};

but in the kernel way would be as:

struct person {
	char *name;
	// ..

	struct list_head engineers;
        struct list_head managers;
        // we can have multiple lists!
};

How do we define a linked list?

Simple:

struct person employees;

Pushing forward towards our objectives, examples of code are:

Initializing:

INIT_LIST_HEAD(&employees.engineers);

Element addition:

struct person *node = NULL;
node=(struct person *)vmalloc(sizeof(person));
if (!node){
   printk(KERN_INFO "mymodule:vmalloc: could not alocate memory");
   return -ENOMEM;
}
memset(node, 0, sizeof(person));
strcpy(node->name, person_name);

list_add(&(node->engineers),&(employees.engineers));

Passing through list and deleting an element:

    struct list_head *pos = NULL;
    struct list_head *q = NULL;
    struct person *node = NULL;
    list_for_each_safe(pos, q, &employees.engineers) {
            node=list_entry(pos, struct person, engineers);
            if (!strcmp(node->name, person_name)) {
                 list_del(pos);
                 vfree(node);
            }
    }

I hope I haven’t forgotten, or misplaced anything during code translation.

Posted by: Catalin | October 2, 2008

Basic proc file system module

This is an example of proc file system basic usage regarding:

  • adding and removing a directory, or file under proc
  • handling I/O  of proc files

Having the proc_demo.ko module inserted into kernel, it will create /proc/{demo, demo/file} under you are able to perform simple I/O like:

  • echo “what ever you want” > /proc/demo/file
  • cat /proc/demo/file

Its purpose is to demonstrate the minimal usage of kernel proc API.

You should read Basic proc file system kernel API page first.

proc_demo.c

#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include<linux/vmalloc.h>
#include <linux/string.h>
#include<linux/proc_fs.h>
#include <asm/uaccess.h>


MODULE_LICENSE("GPL");

static char *file_content=NULL;
static struct   proc_dir_entry *proc_dir=NULL;
static struct   proc_dir_entry *proc_file=NULL;

ssize_t file_write(struct file *filp, const char __user *buff,
		   unsigned long len, void *data);

int file_read(char *page, char **start, off_t off,
		int count, int *eof, void *data);

ssize_t file_write( struct file *filp, const char __user *buff,unsigned long len, void *data)
{
    if (len>PAGE_SIZE) {
	printk(KERN_INFO "proc_demo module: file_write function len > PAGE_SIZE\n");
	return -ENOSPC;
    }

    if (copy_from_user(&file_content[0], buff, len))
	return -EFAULT;

    file_content[len]='\x0';
    printk(KERN_INFO "proc_demo module: you've written: %s\n",file_content);
    return len;
}

int  file_read( char *page, char **start, off_t off,int count, int *eof, void *data)
{
    return sprintf(page,"The user entered:\n%s\n",file_content),
	    strlen(page);
}

static int proc_demo_init(void) {

	int  ret=0;
	// We're alloc'ing memory for the file content buffer
	file_content=(char *)vmalloc(PAGE_SIZE);
	if (!file_content){
	    printk(KERN_INFO "proc_demo module: vmalloc: could not alocate memory\n");
	    goto demo_out;
	}

	memset(file_content, 0, PAGE_SIZE);

        // Creating the proc 'demo' directory
	proc_dir=proc_mkdir("demo", NULL);
        if (!proc_dir) {
	    printk(KERN_INFO "proc_demo module: demo directory creation failed\n");
            ret=-EAGAIN;
            goto proc_mkdir_failure;
	}

	/* creating 'file' and setting the read&&write function
	   handlers */
	proc_file=create_proc_entry("file",0600, proc_dir);
        if (proc_file) {
	    proc_file->read_proc  = file_read;
	    proc_file->write_proc = file_write;
	    proc_file->owner      = THIS_MODULE;
	}
	else {
    	    printk(KERN_INFO "proc_demo module: create_proc_entry: file\n");
	    ret= -EAGAIN;
	    goto proc_file_failure;
	}

	printk(KERN_INFO "proc_demo module: succesfully loaded\n");
        return ret;
proc_file_failure:
	remove_proc_entry("demo", NULL);
proc_mkdir_failure:
	vfree(file_content);
demo_out:
	printk(KERN_INFO "proc_demo module: not loaded loaded\n");
	return ret;
}

static void proc_demo_exit(void) {

        remove_proc_entry("file",proc_dir);
	remove_proc_entry("demo", NULL);

	if (file_content)
	    vfree(file_content);

	printk(KERN_INFO "proc_demo module: unloaded.\n");
}

module_init(proc_demo_init);
module_exit(proc_demo_exit);

Regarding a Makefile you can find a good example here.

Posted by: Catalin | October 2, 2008

Basic proc file system kernel API

Proc fs is a virtual file system mounted in /proc directory which has at least two functions:

  • Exporting kernel state to user-space. It’s like a kernel’s point of view about the system.
  • Communicating particular configuration to the kernel on the fly.  It’s an interface to kernel components through which a system operator can interactive modify system behavior.

Shell commands that are common used with proc are: cat, echo and less. Examples of its usage could be:

  • See system devices:
    cat /proc/devices
  • Enable IPv4 forwarding:
    echo 1 >/proc/sys/net/ipv4/ip_forward

Now having enough about what we should know about operating proc from user space it is the moment to see how things are really done into kernel space.

Two main files from kernel source tree that you should inspect to see some proc API functions are:

  • fs/proc/generic.c
  • include/linux/proc_fs.h

Doing some Linux Cross Reference code search about proc_mkdir , create_proc_entry, and remove_proc_entry you will find that the kernel API is simple and their function signatures are like this:

struct proc_dir_entry *proc_mkdir
                      (const char *name,
                       struct proc_dir_entry *parent)

struct proc_dir_entry *create_proc_entry
                      (const char *name, mode_t mode,
                        struct proc_dir_entry *parent)

void remove_proc_entry(const char *name,struct proc_dir_entry *parent)

At this point is intuitive how we can create, or destroy directories and files under proc file system, but how do we process their I/O? The answer is like this: “Using a proc read and a proc write function setup using the resulting struct proc_dir_entry* after we link in the desired file.”

By a proc read function you should understand a function used by kernel to process data when a /proc file is read from user space; usually performed in the reader user process context; and by a proc write function a function that is used by the kernel to process data when someone writes in a proc file.

proc read and write functions have the prototype as follows:

    int func_read(char *buffer, char **start, off_t offset,
             *int count, int *peof, void *dat)
ssize_t func_write(struct file *filp, const char __user *buff,
                   unsigned long len, void *data)

Furthermore, you can find a good example at Basic proc file system module page.

Posted by: Catalin | September 30, 2008

Cross-referencing the linux kernel sources

In process of  exploring whatever corner of Linux kernel source tree it is cute to consult the sources using some automated search system. Cross reference Linux Project provides a good solution of what you need.  It’s a versatile cross-referencing tool for relatively large code repositories.

So, LXR (formerly “the Linux Cross Referencer“) and some of its forks may be your good friend.  One instance can be found here http://lxr.linux.no

Google’ing after Linux Cross Reference Source phrase you will find some good sites where the kernel sources are indexed.

Good examples could be:

Ups, that last link is devilish!

Basic things that you could do with a cross reference repository:

  • search for a variable, macro, function name etc and find the places in kernel where it’s defined and used (Identifier search)
  • search for a particular file name (file search)

and many more, depending of your current repository.

Older Posts »

Categories