Inter-Process Communication (IPC)

The Separate Offices Problem

Alice works in Room 101. Bob works in Room 205. Their offices are locked — neither can walk into the other's space uninvited, touch the other's files, or read papers off the other's desk. Yet they must collaborate on a project.

What do they do? They use shared folders on a network drive, send emails, or pass notes through a receptionist. They communicate through established channels — not by directly reaching into each other's space.

Operating system processes work exactly the same way. Every process has its own isolated address space. One process cannot directly read or write another process's memory. Yet modern applications constantly need processes to share data, coordinate actions, and pass results. Inter-Process Communication (IPC) is the set of mechanisms the OS provides for this.

Why Processes Cannot Share Memory Directly

When the OS creates a process, it assigns it a private virtual address space. This isolation is intentional — it prevents buggy or malicious processes from corrupting others. A crash in one process does not take down the system.

But isolation creates a problem: cooperation requires communication. A web server spawns a worker process to handle a request. That worker must return results. A shell runs ls | grep python — the output of ls must flow into grep. These scenarios require explicit communication channels.

IPC provides those channels.

IPC Mechanisms

1. Pipes (Anonymous Pipes)

A pipe is a unidirectional data channel between two related processes (parent and child, or siblings from the same fork). Data flows in one direction: one process writes, the other reads.

ls | grep python

This shell command creates a pipe. ls writes its output into the pipe's write end. grep python reads from the pipe's read end. The kernel buffers the data in between.

Limitation: Anonymous pipes only work between related processes (those sharing a common ancestor that created the pipe).

2. Named Pipes (FIFOs)

Named pipes are pipes with a name in the filesystem. Any two processes — even unrelated ones — can open the FIFO by name and communicate through it.

mkfifo /tmp/mypipe           # create named pipe
cat /tmp/mypipe &            # reader (runs in background)
echo "Hello IPC" > /tmp/mypipe  # writer sends data
# Output: Hello IPC

3. Message Queues

Message queues allow processes to send structured messages to a kernel-maintained queue. The receiver picks up messages in order (FIFO by default). Messages can have types, allowing selective retrieval.

Advantages: sender and receiver are decoupled — the sender does not need to wait for the receiver to be ready.

4. Shared Memory

Shared memory is the fastest IPC mechanism. The OS maps the same physical memory region into the address spaces of two or more processes. They communicate by reading and writing that memory directly — no kernel involvement per operation.

Critical caveat: Shared memory requires explicit synchronization (mutexes or semaphores) to prevent race conditions. Without synchronization, two processes writing simultaneously corrupt the shared data.

5. Signals

Signals are software interrupts sent to a process. They carry no data beyond the signal type itself. Common signals:

Signals are limited to notification — not data transfer. A process registers a signal handler function to respond to incoming signals.

6. Sockets

Sockets support communication between processes on the same machine (Unix domain sockets) or across a network (TCP/UDP sockets). They are bidirectional and the most flexible IPC mechanism.

Web servers, databases, and microservices communicate via sockets. A web browser connects to a web server using a TCP socket.

IPC Comparison Table

The Pipe in Action: ls | grep python

When you type ls | grep python in a shell:

1. Shell creates a pipe (two file descriptors: read-end and write-end)
2. Shell forks a child for ls, redirecting its stdout to the pipe's write-end
3. Shell forks a child for grep, redirecting its stdin to the pipe's read-end
4. ls writes directory entries into the pipe
5. grep python reads from the pipe, filtering lines containing "python"
6. When ls exits, it closes the write-end; grep sees EOF and exits

The pipe is anonymous, lives entirely in kernel memory, and is cleaned up when both ends close.

Python Example: multiprocessing.Pipe()

from multiprocessing import Process, Pipe

def producer(conn):
    messages = ["task_1", "task_2", "task_3", "DONE"]
    for msg in messages:
        conn.send(msg)
        print(f"Producer sent: {msg}")
    conn.close()

def consumer(conn):
    while True:
        msg = conn.recv()
        if msg == "DONE":
            print("Consumer: all tasks received, exiting")
            break
        print(f"Consumer processing: {msg}")
    conn.close()

if __name__ == "__main__":
    parent_conn, child_conn = Pipe()

    p1 = Process(target=producer, args=(parent_conn,))
    p2 = Process(target=consumer, args=(child_conn,))

    p1.start()
    p2.start()
    p1.join()
    p2.join()

# Output:
# Producer sent: task_1
# Consumer processing: task_1
# Producer sent: task_2
# Consumer processing: task_2
# Producer sent: task_3
# Consumer processing: task_3
# Producer sent: DONE
# Consumer: all tasks received, exiting

Pipe() returns two connection objects. Each process gets one end. send() and recv() are the IPC operations. The OS handles buffering, blocking, and wakeup automatically.

Shared Memory: Speed with Responsibility

from multiprocessing import Process, Value, Array
import ctypes

def increment(shared_val):
    for _ in range(1000):
        shared_val.value += 1   # WARNING: race condition without lock!

# Correct version with lock:
def safe_increment(shared_val):
    for _ in range(1000):
        with shared_val.get_lock():
            shared_val.value += 1

counter = Value(ctypes.c_int, 0)
p1 = Process(target=safe_increment, args=(counter,))
p2 = Process(target=safe_increment, args=(counter,))
p1.start(); p2.start()
p1.join(); p2.join()
print(counter.value)  # Output: 2000 (correct with lock)

Without the lock, the output would be unpredictably less than 2000 — a race condition in shared memory.

Summary

IPC is the nervous system connecting isolated processes. The right mechanism depends on the scenario:

• Pipes: simple parent-child streaming (shell pipelines)
• Message Queues: decoupled tasks, event systems
• Shared Memory: maximum performance, needs synchronization
• Signals: lightweight notifications
• Sockets: distributed systems, cross-machine communication

Understanding IPC is foundational to building any multi-process application — from shell scripts to microservices to operating system kernels.