Prompt and context
Two local processes coordinate over a Unix domain socket: a privileged process opens a file or creates a listening socket and hands it to a less-privileged worker. Explain how to pass the descriptor, what the receiver gets, and how to handle truncation and authorization risks.
This tests Linux/Unix IPC, the distinction between a file-descriptor table and an open file description, and the ancillary-data protocol of sendmsg/recvmsg. Linux unix(7) defines SCM_RIGHTS for sending or receiving a set of open file descriptors between processes.
What the interviewer is testing
- Distinguishing a process-local fd integer from the kernel's open file description.
- Knowing that
SCM_RIGHTSuses ancillary data rather than placing an integer in the ordinary payload. - Correctly sizing
cmsghdr,CMSG_SPACE, andCMSG_LEN, and checkingMSG_CTRUNC. - Covering socket-path permissions, sender identity, resource limits, close timing, and failure cleanup.
Questions to clarify first
- Do the processes share a user, and where is the privilege downgrade or trust boundary?
- Is the descriptor a regular file, connected socket, listening socket, epoll fd, or device fd?
- Is the channel
SOCK_STREAMorSOCK_DGRAM, and does the protocol need message boundaries and acknowledgements? - Can the receiver verify sender credentials, resource type, read-only properties, and the expected fd count?
A 30-second answer
I would create a Unix-domain socket pair, carry descriptors in a SOL_SOCKET/SCM_RIGHTS control message from sendmsg, and put a protocol version and request ID in the real payload. The receiver would call recvmsg with a sufficiently large CMSG_SPACE buffer, check level, type, length, and MSG_CTRUNC, then use the received fd as a new integer in its own process. What crosses the boundary is a reference to an open file description, so the receiver normally gets a different fd number. The protocol would verify peer credentials, cap counts, set close-on-exec, and close unaccepted or unused descriptors on every error path.
Step-by-step deep dive
Distinguish fd numbers from open file descriptions
An fd is an integer index in a process's fd table. An open file description is a kernel object holding open state such as file offset and status flags. SCM_RIGHTS copies a reference to the latter; the receiver normally gets a different fd integer, semantically similar to duplicating an fd into another process's fd table.
Use ancillary data instead of ordinary bytes
sendmsg and recvmsg carry a chain of cmsghdr records through msghdr.msg_control. Set cmsg_level to SOL_SOCKET, cmsg_type to SCM_RIGHTS, and put an integer fd array in the data area. The ordinary payload can carry version, purpose, and an acknowledgement ID, but cannot replace the control message.
Size the control buffer correctly
For the actual count, the sender uses CMSG_LEN(n * sizeof(int)) for cmsg_len; the receiver allocates aligned space of at least CMSG_SPACE(n * sizeof(int)). Parse with CMSG_FIRSTHDR and CMSG_NXTHDR, rejecting short lengths and unexpected types.
Handle truncation and stream boundaries
If the receive control buffer is too small, ancillary data can be truncated or discarded and MSG_CTRUNC is set; the receiver must not use a partial descriptor list. Linux requires at least one real byte with ancillary data on a SOCK_STREAM, and ancillary data forms a receive barrier, so bind the control message to a request ID instead of relying on byte-stream position.
Establish identity and authorization boundaries
Directory and socket permissions are the first boundary for a filesystem socket. The server should also use SO_PEERCRED or SCM_CREDENTIALS and confirm tenant, purpose, and resource type at the application layer. Receiving an fd does not grant extra authority by itself; the sender must only transfer an authorized reference.
Manage lifetime and resource limits
The sender may close its own fd after sending, but the kernel keeps an in-flight reference until the receiver accepts it. Linux limits the operation with RLIMIT_NOFILE and SCM_MAX_FD; the current man page records SCM_MAX_FD as usually 253, while older versions used 255. Limit descriptors per message and worker, and make rejections observable.
struct msghdr msg = {0};
struct iovec iov = {.iov_base = "F", .iov_len = 1};
union { char buf[CMSG_SPACE(sizeof(int))]; struct cmsghdr align; } control;
msg.msg_iov = &iov; msg.msg_iovlen = 1;
msg.msg_control = control.buf; msg.msg_controllen = sizeof(control.buf);
struct cmsghdr *c = CMSG_FIRSTHDR(&msg);
c->cmsg_level = SOL_SOCKET; c->cmsg_type = SCM_RIGHTS;
c->cmsg_len = CMSG_LEN(sizeof(int));
memcpy(CMSG_DATA(c), &fd, sizeof(fd));
sendmsg(sock, &msg, MSG_NOSIGNAL);Example of a strong answer
I would make this a local IPC protocol with acknowledgements. The sender uses sendmsg on a Unix-domain socket with a SOL_SOCKET/SCM_RIGHTS control message; the real payload carries a protocol version, purpose, and request ID. The receiver allocates control space with CMSG_SPACE, checks type, length, and MSG_CTRUNC, verifies peer credentials, descriptor count, and resource type, and only then hands it to the worker. The important semantic point is that the transfer references an open file description, so the receiver's integer is usually different and file offset or open state may be shared. I would set close-on-exec, cap in-flight descriptors, handle RLIMIT_NOFILE and SCM_MAX_FD, and close and audit every failure path.
Common mistakes
- Writing the fd integer into JSON or a byte payload and assuming the other process can use it directly.
- Ignoring
CMSG_SPACEalignment and allocating onlysizeof(int)for control data. - Skipping
MSG_CTRUNCand using a descriptor list that was truncated. - Checking only the Unix socket path without verifying peer credentials and purpose.
- Forgetting that the receiver gets a new fd number or failing to specify shared offset and status semantics.
- Ignoring close-on-exec, fd limits, send failures, and closing unused descriptors.
Follow-ups and responses
Does the receiver get the same file descriptor?
Usually not the same integer. The kernel copies a reference to the same open file description into the receiver's fd table, so file offset and some open state may be shared. If independent offsets are required, reopen or copy the data instead of assuming equal fd numbers.
Why send a real byte?
Linux Unix stream sockets require at least one real byte in the same sendmsg when ancillary data is sent; it also lets the protocol associate the control message with a request. Linux datagrams can omit it, but portable code should still include one real byte.
What if the control buffer is too small?
Ancillary data may be truncated or discarded and MSG_CTRUNC is set. Close invalid or excess descriptors, return a protocol error, and record the event; never treat a partial list as complete authorization.
How do you stop a privileged process from handing out the wrong resource?
Use socket permissions and peer credentials to restrict the connection, then bind request ID, tenant, purpose, and resource type at the application layer. The sender selects only from an allowlist; the receiver checks read-only properties, path or socket state, and audits every authorization and close.