🔁 File Descriptors and Redirections
In Unix-like operating systems, the kernel manages all Input/Output (I/O) operations using a system of unique identifiers known as File Descriptors (FDs). An FD is a non-negative integer that serves as a reference to an open file, network socket, pipe, or any I/O resource maintained by the system.
You can visualize a File Descriptor as a session token given by the operating system (the kernel) for any active data stream. Whenever a process needs to read data or write output, it uses the FD as a precise pointer to the resource. Understanding FDs is fundamental because all shell redirections are based on manipulating these numbered streams.
The Standard File Descriptors
By default, every Linux process opens three standard data streams upon execution:
| Descriptor ID | Stream Name | Function |
| 0 | STDIN (Standard Input) | The data stream the program reads from (usually the keyboard). |
| 1 | STDOUT (Standard Output) | The stream where the program writes normal output (usually the terminal screen). |
| 2 | STDERR (Standard Error) | The stream where the program writes error and diagnostic messages (usually the terminal screen). |
Example of STDIN and STDOUT:
When running the cat command without arguments, the shell waits for your input (STDIN). Once you finish and press [ENTER], cat processes that input and immediately writes it back out as STDOUT to your terminal.
Shell
user@host:~$ cat
THIS IS MY STDIN (Input via keyboard)
THIS IS MY STDOUT (Output echoed back)
Example of STDOUT and STDERR:
Commands can produce both normal output (STDOUT) and error messages (STDERR) simultaneously. When using the find command to search a privileged area like /etc/ for the shadow file, permission errors will occur alongside the normal results:
Shell
user@host:~$ find /etc/ -name shadow
# (Results might be)
find: ‘/etc/cron.daily’: Permission denied (STDERR - FD 2)
find: ‘/etc/default’: Permission denied (STDERR - FD 2)
/etc/shadow (STDOUT - FD 1)
I/O Redirection Basics
Redirection is the process of manipulating the flow of data streams, changing the default source or destination of STDIN, STDOUT, or STDERR. This is achieved using special shell characters (<, >, >>).
Redirecting and Suppressing Errors (STDERR)
A common use case is suppressing error messages to keep the output clean. We redirect FD 2 (STDERR) to the null device (/dev/null), which acts as a virtual black hole, discarding all data sent to it.
Shell
user@host:~$ find /etc/ -name shadow 2>/dev/null
/etc/shadow # Only STDOUT is visible now
Redirecting Standard Output (STDOUT) to a File
The greater-than sign (>) redirects STDOUT (FD 1) to a file. If no FD number is specified, the shell defaults to FD 1.
Shell
user@host:~$ find /etc/ -name shadow 2>/dev/null > results.txt
In this single command: STDERR (2) goes to /dev/null, and STDOUT (1) goes to results.txt.
Crucially: Using a single > will overwrite the file if it already exists without prompting.
Appending Output to a File
To redirect STDOUT and append the new content to the end of an existing file without overwriting it, use the double greater-than sign (>>).
Shell
user@host:~$ find /etc/ -name passwd >> results.txt 2>/dev/null
Redirecting Streams to Separate Files
To maintain clarity, you can redirect STDOUT and STDERR to different files explicitly by using their FD numbers:
Shell
user@host:~$ find /etc/ -name *.conf 2> errors.log 1> config_files.list
Redirecting Standard Input (STDIN)
The less-than sign (<) redirects a file’s content to act as STDIN (FD 0) for a command.
Shell
user@host:~$ cat < config_files.list
# cat reads the contents of config_files.list as its input stream
Using Pipes for Command Chaining
The pipe symbol (|) is a form of redirection that sends the STDOUT (FD 1) of the command on the left directly into the STDIN (FD 0) of the command on the right. This allows commands to be chained together sequentially, making complex data processing highly efficient.
Example: Filtering and Counting
Here we use grep to filter a large list of configuration files (STDOUT from find) and then pipe that filtered list to wc (word count) to get a final tally.
- Find and Filter: Find all
.conffiles, suppress errors, and pipe the results. - Filter by Pattern:
grep systemdfilters the incoming list, retaining only lines containing the pattern “systemd”. - Count Results:
wc -lreceives the filtered list and counts the number of lines.
Shell
user@host:~$ find /etc/ -name *.conf 2>/dev/null | grep systemd | wc -l
(The final output is a single number: the count of systemd-related config files in /etc/)
By mastering file descriptors, redirection, and pipes, you gain precise control over data flow, allowing you to structure commands to extract, process, and manage information with immense efficiency and precision.