Sunday, November 8, 2015

light weight process(LWP)

        Light Weight Process is created by Linux system to support better performance in multitasking programming. Two LWP shares common memory space and their context significantly less than the normal processes.
       
        When a child process created it is almost identical to its parent process and inherit all properties from parent process. it also receives a copy of parent address space. Parent and child share same page for code text but they do have their separate copies of stack and heap segment so the changes done by chide is invisible by parent process and vice versa.
       
        In multitasking programming where dependency exists between parent and child process this invisibility may leads to severe problems some time corrupting memory to crashing the entire execution.
       LWPs may share same resource and address space so changes done by one process is immediately noticed by all the process running together. Synchronization among the LWPs is quite possible so corruption of memory can be prevented. POSIX-compliant  pthread library widely used to create LWP and make them synchronized.

pthread related process creation and synchronization is discussed later in this blog.

How to create a Process in Linux System?

To perform a certain task we required an application so it will act on given command and lifetime set by program. Whatever it needs in application but eventually we need a process to perform a task.
There are several ways in Linux where we can create a process through a program otherwise opening a shell creates a process and running command over there creates another process.

But running command and using pre-installed application is already designed to their job. what will do if we have to create a separate process for our own need. Linux provides several system calls to create process, duplicate processes.


clone() fork() and vfork()

clone() is a wrapper function defined in the C library, which set up the
new light weight process(LWP), clone() system call hidden to the programmer. The
sys_clone() service routine that implements the clone() system call
does not have the fn and arg parameters.

    fork() system call creates a new process which is duplicate of its
parent process. The child process creates a new entry in process table with
many of the same attributes as the current process. it is almost identical to
the original process, execute the same code but with its own data space,
environment and file descriptors.
    the fork() system call is implemented by Linux as a clone() system call
whose flag parameter specifies both a SIGCHILD signal and all the clone flag
cleared, and whose child_stack parameter is the current parent stack pointer.
Therefore, the parent and child temporarily share the same User Mode stack. But
by using Copy On Write mechanism, they usually get separate copies of the User
Mode Stack as soon as one tries to change the stack.

    vfork() system call creates a new process which shares memory address space
of its parent. there may a chance of parents and data overlapping during
concureent execution, so need to make an arrangement to block either of one
during concurrent execution.
    vfork() is also implemented as a clone() system call whose flags parameter
specifies both a SIGCHLD signal and flags CLONE_VM and CLONE_VFORK, and whose
child_stack parameter is equal to the current stack pointer.

Application of all system calls

Saturday, November 7, 2015

Process Identifier (PID)

Whenever a process starts one unique positive integer number is assigned to that called Process Identifier(PID). It is ranged between 2 to 32,768. Number 0 and 1 typically reserved for idle process and special init Process. When a new process is started a next unused in sequence number is assigned to that process.
     All current process status and their PID can be seen by using command

# ps ax  
  PID TTY   STAT  TIME COMMAND  
   1 ?    Ss   0:02 /sbin/init  
   2 ?    S   0:00 [kthreadd]  
   3 ?    S   0:18 [ksoftirqd/0]  
   5 ?    S<   0:00 [kworker/0:0H]  
   7 ?    S   0:29 [rcu_sched]  
   .................  
   ................  
   ................  
  2292 ?    Sl   0:08 /usr/lib/upower/upowerd  
  2367 ?    SNl  0:01 /usr/lib/rtkit/rtkit-daemon  
  2410 ?    Sl   0:00 /usr/lib/colord/colord  
  3987 ?    SLl  0:00 /usr/bin/gnome-keyring-daemon --daemonize --login  
  3989 ?    Ss   0:02 init --user  
  4077 ?    Ss   0:19 dbus-daemon --fork --session --address=unix:abstract=/tmp/dbus-7SdHSkpTgx  
  4088 ?    Ss   0:00 upstart-event-bridge  
  4095 ?    Ss   0:00 /usr/lib/x86_64-linux-gnu/hud/window-stack-bridge  
  4106 ?    Sl   0:17 /usr/lib/x86_64-linux-gnu/bamf/bamfdaemon  
  4112 ?    Sl   0:00 /usr/lib/at-spi2-core/at-spi-bus-launcher  

ps command has many option (-ef, -A..)which gives more resolution. To see more option go to manual page of ps command, type
 #man ps  







What is socket in linux?

Socket is a way of communication between two correlated and uncorrelated process a kind of IPC(Inter Process Communication). It provides an interface  between all the network entities and makes data communication possible. Socket can also be used for data transfer between the processes within a system. Socket is client/server based communication that can be created withing system or across the network.
    Socket generally requires only IP address and Port number of destination to send the packet. IP provides connection between two system based on their logical addresses and port number provides mapping between data packer received and its corresponding application.
      Socket has its transport layer (UDP, TCP and Raw IPs) dependency. based on their transport layer requirement socket is categories into two type Stream Socket and Datagram Socket.

All aspects of socket can be easily understood by seeing the socket system call and the prototype is

 int socket(int domain, int type, int protocol);  

And its prototype defined in following header files

 #include <sys/types.h>  
 #include <sys/socket.h>  //Basically this

Socket system calls accept three arguments and return socket descriptor that can be used in further socket communication.

arg1- domian
Domain parameter specify the address family which will used for communication.

AF_UNIX
Used when socket is created to transfer information within a system also called system socket
AF_INET
Used when socket is created to communicate across the network also called Network socket.
AF_IPX
Novell IPX protocol
AF_NS
Xerox Network Systems protocols
AF_APPLETALK
Appletalk


arg2 - type
Type specify the type of communication is being used and they are..

SOCK_STREAM It is connection oriented, stream based and reliable communication. It provides two way of communication, Acknowledge is sent for each successful packet and re-transmission in case of any drop. TCP communication falls under this category, TCP also provides fragmentation and reassembly for long messages.

SOCK_DGRAM  it is datagram service and connection less transmission. In this type data is directly sent to destination without making any prior connection. So less reliable, no reordering of packets and no acknowledgement. chances of packet drop is more but despite all it is very fast and robust in compare to connection based data transmission. UDP protocol falls under this category.

arg3- Protocol
It is normally Zero. but can be chosen based on Socket domain and type.



What is System calls ?

System calls are set of function which interacts with system from user space. User space where most of the application runs but to make use of system applications like creation/reading/writing a file, creating a task, communication between the process, sending data across the network it required a system call.

Ex: File handling related system calls
open(),read(),write(),close()

Networking related system calls
socket(),bind(), sento(), recvfro(), send(), recv()

Task and process related system calls
fork(),vfork(),exec()


Difference between System call and API(Application Programming Interface)

API is set of functions that completely run in user space and they need system call to interact with system(Kernel).

Inter Process Communication(IPC)


All About Process

In computer to perform any task we requires a process which leads task to an end. Process is nothing but running instance of a program. Each Process is started by some other process called Parent Process and newly created process will become Child Process. Conventionally Linux operating system treats all process in the same way, resource hold by the parent process are duplicated in the child process. 

Whenever we run any application on linux system it creates an entry in process table with a process ID(PID). Based on PID only system tracks the process status. Once process perform its job it immediately gets removed from process table and allow other process to acquire the resource. 

Process table is nothing but data structure which describes of all running process and their status (Sleeping, Running, Orphan etc.). will discuss all the process states later in this blog.

When we execute our simple "Hello world.c" a.out, it creates one process called main process for that application which is currently in running state as long as it not coming back to terminal. 
     If we see  a.out is not the first process created in Linux system, There is a hierarchic of processes from Process 1 or init process to current running process . init process in Linux operating system who take cares of all running process or hung process or orphan process and they starts with starting of operating system. we will see one by one how process gets created, maintained, terminated and all other aspects of it.

How to Create a Process in Linux system



    

Static and Dynamic library and linking

Programmer can not write similar function bodies again and again which are used very frequent in programming. To reduce this overhead, library is created. Library contains all those functions  called library function, which is used often in programming i.e pritf(), scanf(), read(), write() and many more. All these function bodies are stored in object files in library. Libraries are compiled separately and  linked to your program on type of linking basis. two types of linking used to add objects to program and they are compile time linking (Static linking) and run time linking(Dynamic linking), explained below.

Based on linking with code library divided into two types 1) Static Library or Linking. 2) Dynamic Library or Linking.

Static Linking (Static library)

It is simplest form of library where all objects files are collectively kept in ready-to-use form. Before using this library, programmers need to include header which contains function declaration and use -l option(explained later) so that compiler and linker can link all function type to library function and create a single executable file. all the linking happens at last stage of compilation called linking stage. Programmers can create their own static library and method is very simple :).


Creating a Static library

Here we are creating two library function to demonstrate  static linking. These two files contains function for adding and subtraction.
step-1 : Create add.c
 #include <stdio.h>  
 int my_library_add(int arg1, int arg2)  
 {  
      printf("Add function\n");       
      return (arg1 + arg2);  
 }  

File 2 sub.c
 #include <stdio.h>  
 int my_library_sub(int arg1, int arg2)  
 {  
      printf("Sub function\n");  
      return (arg1-arg2);  
 }  

It is not always mandatory to create multiple files for all separates function. It is up to programmers based on requirement and type of application they can crate as many as files or all can contain in a single file.

Step-2 Compile both add.c and sub.c
 $ gcc -c add.c sub.c   
 $ ls  
 add.c add.o sub.c sub.o  

Step-3 create one .h file locally where u can define your library functions ex: my_lib.h
 #include <stdio.h>  
 int my_library_add(int, int);  
 int my_library_sub(int, int); 

Step-4 Create archives of name libmylib.a using command 'ar' including both add.o and sub.o object files. In terms of linker to identify any kind of library, library name should must start with "lib". Other wise programmer may face "/usr/bin/ld: cannot find -lmylib" kind of error.


$ar crv libmylib.a add.o sub.o   
 a - add.o  
 a - sub.o  
 $ ls  
 $ ls  
 add.c add.o libmylib.a main.c my_lib.h sub.c sub.o  

Step-5 Create main.c file where all library function created above can be used
main.c
 #include <stdio.h>  
 #include <stdlib.h>  
 #include "my_lib.h"  
 int main(int argc, char *argv[])  
 {  
      int arg1, arg2, sum, sub;  
      if ( argc != 3)  
      {  
           printf("Invalid args<Enter arg1:integer arg2:integer>");  
           exit(0);  
      }  
      arg1 = atoi(argv[1]);  
      arg2 = atoi(argv[2]);  
      sum = my_library_add(arg1,arg2);  
      printf("Sum : %d\n",sum);  
      sub = my_library_sub(arg1,arg2);  
      printf("Sub : %d\n",sub);  
      exit(0);  
 }

Add local header file into main to avoid "implicit declaration of function" kind of warning. And declaration of prototypes is good programming practice.

Step-6 Now compile main file and link static library to program.
  gcc -Wall -I. -o run main.c -lmylib  

-l option tells compiler to link lib<libraryname> to executable file so that all library functions used in program can be linked with their bodies.

Step -7 Run the executable file "run"
 $ ./run 5 8  
 Add function  
 Sum : 13  
 Sub function  
 Sub : -3  



Dynamic Linking (Shared/Dynamic library)

Unlike static linking, dynamic linking happens at run time so that once program started running all library function will load and unload as soon as usage of function is over. Dynamic libraries are stores same place as static library, but filename for shared libraries are different. mostly denoted as libmylib.so.

Creating a shared library 

Like static it is also collection of object files with the difference of, it will not combine with program at compilation time.

Will make use of same programs and files which were used to create static library.

Step-1 :Compile add.c and sub.c using -fPIC or fpic option it is needed to create shared library.
$ gcc -fPIC -g -c -Wall add.c   
$ gcc -fPIC -g -c -Wall sub.c  
$ ls  
 add.c add.o main.c sub.c sub.o  

Step-2 : Create shared library (.so) file by using command 

 $ gcc -shared -o libmyshared.so add.o sub.o  
 $ ls  
 add.c add.o libmyshared.so main.c sub.c sub.o  

libmyshared.so is created as shared library.

Step-3 : Now use this shared library in program
# gcc -Wall -o run main.c libmyshared.so  

It is not yet done before running executable file we need to copy this .so file to /usr/lib/

# cp *.so /usr/lib/  
or can be installed using
 # ldd run  
      linux-vdso.so.1 => (0x00007fff9c3c2000)  
      libmyshared.so => (0x00007fff9c3c3000)  
      libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f81fa14e000)  
      /lib64/ld-linux-x86-64.so.2 (0x00007f81fa539000)  

Step -4 Run the executable file "run"

 $ ./run 5 8  
 Add function  
 Sum : 13  
 Sub function  
 Sub : -3  


There are many advantages of using Shared library over Static library...

  • When we use static library it links before creation of executable file(linking stage) hence as a result large size of .exe file. whereas shared library gets loaded on run time so does not affect the size of .exe file.
  • when multiple applications are running and all are using same library, all ended up with redundant copy of same function in memory. As many as times the function is called it will create a separate image of that function in memory and this is not at all good for memory constrain applications. Dynamic library is shared among all the application so all are using same image of function multiple times, no extra image is created for library function.  
  • In static linking all library functions are loaded before running the code and unloaded after execution is done.  Dynamically linked library (shared libraries) are loaded on ad-hoc basis on run time, only loaded into memory when it is needed and unloaded immediately after use.
  • Static library has to be linked explicitly using -l<libname> option but shared library will link automatically at run time.
  • Shared library can be updated separately for the application rely on it.  

References:
1. Beginning Linux Programming, 4th Edition
Neil Matthew, Richard Stones
2.Program-Library-HOWTO