ls -l: What happens behind the scenes?

A step by step explanation of what happens when you type ls -l and hit Enter in a shell

On UNIX systems, the user is interacting with the shell every time they use the terminal .It works as an intermediary between the operating system and the user, and provides the interface necessary for the user to input human readable commands and converts them into something the kernel can understand. The kernel is the core of a computer’s operating system, and has complete control over everything from file management, process management, memory management and more. Therefore, the shell can be described as a command-line interpreter for executing other programs.

When the user inputs a command, let’s say “ls -l”, the output appears almost instantly on screen. It seems to be something that the system does in a heartbeat, yet there is a lot that happens backstage.

Below, you will find a step by step breakdown of what happens when a command is entered, and the process the system carries out to interpret, execute and output the result.

Every shell carries out the same process in order to interpret and execute the command being input by the user.

Prompt

The first interaction that occurs is the printing of a prompt. The prompt is a combination of specific characters that when displayed, the system is ready to receive a command. Usually, the shell prompt ends with the “$” character. Also, in the Shell the prompt is usually followed by the blinking cursor, signifying that it is waiting for the user to input characters.

Most Unix commands receive input from what is called “standard input”, which is usually from the keyboard, and send output to the “standard output” typically the screen.

The user can input whatever combination of characters they wish, and press the Enter (Return) key once their input is completed. In the meantime the program will not do anything until the Enter key is pressed and the information is passed to the shell.

Get line

When a command is entered, the keyboard drivers will interpret this and send it to the system where the process of command recognition will begin. On success of obtaining the user input, the next step is to correspond it to an executable program. On failure, the error message will be output — but either way something, somewhere will happen. The string passed by the user will be saved as a buffer and the system will carry out actions to begin the interpretation process.

In C, a function that carries out such a task is getline(); — it receives &buffer (the address of the first character position where the input string will be stored), &bufsize ( is the address of the variable that holds the size of the input buffer, another pointer) and where to take the input from (in this case stdin because the standard input is read). Getline returns 1 if it finds an input and 0 if it encounters the end of the file (EOF). If there is some error in getting a record, such as a file that cannot be opened, then getline returns -1.

Tokenize

The first step is to tokenize (or parse) the string into independent substrings, usually separated by a delimiter, in this case a blank space. The tokenization process will separate the program name from the array of parameters that were passed. In our example, the first token will be ls and the second will be -l. There can be as many tokens as arguments passed.

In C, the strtok() function is used to split a string into a series of tokens based on a particular delimiter. A token is a substring extracted from the original string. The general prototype for the strtok() function is:
char *strtok(char *str, const char *delim)
Where str is the string which is to be split and delim is the character on the basis of which the split will be done. The function performs one split and returns a pointer to the token split up. A null pointer is returned if the string cannot be split.

Once the user input is tokenized, the system can now begin the process of interpreting which command was prompted to the shell and finding the program it calls, if any. For this, the first step is to check if the command is an alias.

Is it an alias…

An alias in this context means the same as it does in the common usage of this word.

Aliases are a (usually short) name that the shell translates into another (usually longer) name or command. Aliases allow you to define new commands by substituting a string for the first token of a simple command. They are typically placed in the ~/.bashrc (bash) or ~/.tcshrc (tcsh) startup files so that they are available to interactive subshells.

If an alias is found, it will be replaced by the original command it stands for and continue the process to find the executable program that it calls. If not, the tokens will continue as they are onto the next step, which is to check if the command is a built-in function.

..or a built-in function?

Built-ins are specific functions that are executed directly by the shell without the need of invoking another program or beginning a separate process.

In other words we can say that the built-in functions commands will always be available in RAM so that accessing them is bit faster when compared to external commands which are stored on the hard disk. Sometimes there is high chance of a system crash and you can not access anything on a machine. But if we still have shell access we can execute all built-in commands to do basic troubleshooting and recovery system. Built-in commands may vary from shell to shell.

To check if a command is external or internal command(built-in) we have to execute type command:

Again, if the program finds the command to be a built-in function, it will be executed directly without the need of invoking another program, else it will continue searching in one last place

Then it has to be in the PATH!

The next step is to search for the executable file within the environment variable PATH ($PATH). This environment variable stores a set of directories where executable files are located. Every route is separated by “:”.

The shell will then concatenate the program name to the end of every directory route. Then, it will use the system call stat to check if the route passed is valid. Stat is used to check the status of a file, such as when the file was last accessed, modified, permissions, and more. On successful completion it will return 0, and on failure it will return -1. Therefore, if the return is 0, it means that the file exists and so the system found the corresponding executable program to the passed command.

Ready, Set, EXECUTE!

So, the user input was obtained, interpreted, and the program it corresponds to was located. Now, it is time to execute it.

BUT WAIT!

In order to execute a program, the shell has to duplicate itself and execute the program on a parallel process.

There is one important piece to all of this: processes. Until now, the program has been in the main thread of our program. However, we cannot execute the command in the main thread itself, because of the following reasons:

• An erroneous command will cause the entire shell to stop working. We want to avoid this.
• Independent commands should have their own process blocks in order for the process to be traceable and to maintain the program working in a loop returning to the prompt after executing the command instead of finishing the program.

Fork

To be able to duplicate create a parallel process, the shell uses the system call fork.

fork(); will create a copy of the current process. The copy is known as the child and the main thread will remain in the parent process. Each process in a system has a unique process id (pid) associated to it.

The program should be executed in the child process and the parent will be called once the child process is finished. This is done with the wait(); syscall on the parent process. The wait syscall will halt the parent process until the child processes execute their corresponding commands, and the parent, when all the child processes are done, will return the prompt to the screen so that the user can input the next command.

Can you execute the program already!?

The program route is passed to the execve() syscall that executes the program and shows the output on the screen, whether it succeeds or fails.

When the program is finished executing, the parent process will be called and the prompt will appear again on screen awaiting the user’s next command.

The execve will run the program with whatever arguments were passed with the command.

So… is that it?

Well yes, this is certainly it! This is the whole process the shell carries out when a command is prompt by the user. This happens in milliseconds, and the user sees the output almost immediately, yet what happens backstage is a lot, isn’t it?

So, in conclusion, when the user types in ls -l on the terminal, the process explained above will be carried out and the user will see the files in the current directory listed in long format.

ls -l

By itself, ls lists the files in the current working directory:

With the -l argument, the files are displayed in long format, meaning that the output shows a lot more information about every file.

The total output consists on 10 fields per file, where:

• The first field could be ‘-’ for File, d for Directory, l for Link
• The second, third and fourth fields are permissions. r = read, w = write and x = execute. There are three fields regarding permissions. The first field shows the permissions the owner has over the file, the second field shows the permissions the group has over the file and the third field shows the permissions everybody else has over the file.
• The fifth field specifies the number of links or directories inside this directory.
• The sixth field is the user that owns the file, or directory.
• The seventh field is te group that the file belongs to, and any user in that group will have the permissions given in the third field over that file.
• The eighth field is the size in bytes.
• The ninth field is the date of the last modification to the file.
• And the tenth field is the name of the file

So, all the process explained in the previous paragraphs is what occurs when a user inputs ls -l on the terminal. The reception of the command, interpretation, recognition, execution of the program, breakdown of the output and coming back to displaying the prompt awaiting the next command.

Blog written by Soledad Frechou and Tadeo Grach for Holberton School Cohort 13