# Terminals in Your Pocket (2): An Architecture of Three Parts

- URL: https://wuming.si/blog/4084/
- Author: Wuming
- Published: 2026-10-07
- Language: en
- Categories: Technology, English

> Previous: [Why Reach Your Computer From a Phone](https://wuming.si/blog/4083) | Next: [Teaching Coding Agents to Knock](https://wuming.si/blog/4085) | 中文版：[三个部件的架构](https://wuming.si/blog/4079)

The first post set the principles: the computer is the centre, connections are direct, code never leaves. This one is about the shape those principles produce.

The whole system has three kinds of part, each with a deliberately narrow job.

| Part | Runs where | Owns | Can see |
|---|---|---|---|
| Resident process | On every computer | tmux sessions, files, agent state, local rules | Everything |
| Control plane | A small cloud service | Accounts, device certificates, push, billing | Whose devices are online, nothing more |
| Relays | A few public machines | Forwarding bytes when a direct path fails | Ciphertext |

Phones and browsers are clients, not parts. They talk to the computer, not to a server.

## The resident process

This is the only place where real work happens, and it does little:

- on first start, generate a key pair, register the device with the control plane, and get back a device certificate;
- announce itself: direct addresses, plus which relays it holds a reservation on;
- accept connections from phones, verify the other side's certificate, then serve what is asked for: a stream onto a tmux window, a file listing, the state of agents;
- send a heartbeat to the control plane now and then, and collect any admin commands waiting for it (end a session, say).

What it refuses to do matters more. It doesn't manage sessions itself; tmux already does that well. It doesn't parse terminal output; the terminal emulator on the phone renders it. It doesn't cache files; every read hits the disk.

It is written in Go, one binary, no dependencies. The same networking core is compiled as a library into the phone app, so both ends run the same protocol implementation. That alone saved weeks of interoperability debugging later.

## The control plane: authenticate, never decrypt

The control plane is a thin HTTP service. What it knows fits on one page: which accounts exist; which devices each account has and their public keys; when each computer last sent a heartbeat and which relays it sits on; which devices have been revoked.

It issues device certificates: short-lived statements that "this public key belongs to this device of this account, and it is a computer (or a phone)". Certificates expire within days and devices renew them automatically. Revoking a device means adding its serial number to a revocation list that computers and relays fetch on a schedule.

It never holds a session key and never sees a byte of terminal content. If the whole control plane were stolen, the attacker would have a list of devices. That boundary is intentional: the control plane is the most exposed part, so it holds the least.

## Relays: the fallback when direct fails

Whether two devices behind NATs can connect directly depends on luck and on the kind of NAT. The approach here is to try, and fall back if it fails.

When a computer starts, it reserves a slot on the one or two relays nearest to it, effectively a forwarding address. When a phone wants the computer, it first opens a relayed connection, then both sides exchange their addresses over it and punch towards each other at the same time. If the punch succeeds, traffic switches to the direct path and the relay drops out. If not, the relayed connection stays.

A relay is a standard circuit forwarder. It does one thing: deliver the bytes from one end to the other unchanged. The encryption handshake happens between the two devices; keys never pass through the relay. All it can observe is which two nodes are talking and how many bytes they exchange.

Relays have one more duty: they only forward between devices of the same account. They sync an allow list from the control plane, and nodes not on it can't connect at all. That is the first gate. The second is on the computer itself, which the next post and the fourth one cover.

## How a connection comes together

Putting the three parts together, "open a window on the phone" looks like this:

1. The phone asks the control plane which computers it has and which relays each sits on.
2. The phone connects to the computer through a relay; the two complete an encrypted handshake with their device keys and verify each other's certificate.
3. The computer checks whether this phone has already been approved locally (post four).
4. Both sides attempt a direct connection and switch to it on success.
5. The phone opens a stream on the connection and asks for a tmux window. The computer attaches it, and from then on terminal bytes flow both ways on that stream.

Every keystroke from the phone and every line from the computer travels on that end-to-end encrypted stream. At most one machine in the middle sees ciphertext; most of the time there is none.

## A few trade-offs

**Why not an off-the-shelf VPN mesh.** Those solve "make two machines reachable" and know nothing about what runs on top. This system needs to understand terminals, files and agents to do notifications, permissions and sharing. Owning the network layer freed the layers above it.

**Why not WebSockets to a server that forwards.** It is the simplest build, but the server sees everything, and every keystroke detours through a data centre. The latency difference is something you feel when typing on a phone.

**Why relays don't do accounts.** Relays are the part most likely to be deployed in odd places, including inside a customer's own network. A relay that only knows an allow list and only forwards bytes is easy to deploy and easy to audit.

Next: the most interesting part of the system. How to know an agent is waiting, and how to deliver an answer to a terminal without ever typing into the wrong place.

## The series

1. [Why Reach Your Computer From a Phone](https://wuming.si/blog/4083)
2. An Architecture of Three Parts (this post)
3. [Teaching Coding Agents to Knock](https://wuming.si/blog/4085)
4. [The Computer Decides Who Gets In](https://wuming.si/blog/4086)
5. [Troubleshooting Notes](https://wuming.si/blog/4087)
