Where the Sound Comes From and Where It Goes: USB Handsets, Browser Calls and an ESP32 (4)
Previous: VoWiFi from Scratch | Next: On a Mac | 中文版:USB 话机、浏览器通话与 ESP32
The previous post got Wi-Fi calling’s signaling and voice stream working end to end. But “working” only meant this: a stream of 8 kHz samples was flowing around in the Raspberry Pi’s memory. Nobody could actually hear anything yet.
This post is entirely about one question: which wire does the sound actually travel on?
- How to get audio on the cellular calls the module places itself;
- How to hook up a real USB telephone handset, one that can go off-hook, dial from its keypad, and ring;
- How to use the browser directly as a phone;
- Call recording;
- How an ESP32 dev board with pitifully little memory can act as a Wi-Fi extension.
One audio bus, one format
Every piece of audio in the system uses a single format: 8 kHz, mono, 16-bit, one frame every 20 ms, exactly 320 bytes.
That wasn’t picked at random. It happens to be the native format of the module’s USB sound card, and it happens to be the sample rate of the G.711 telephony codec. Once everything is unified on this format, any endpoint can plug in with no codec, no resampling and no negotiation. You just move the bytes.
Wi-Fi call voice ──┐ ┌─ USB handset
module cellular ───┼──> audio bus <──┼─ browser
│ 8k/mono/16-bit ├─ phone app
│ ├─ ESP32 extension
└───────────────────└─ recording
There’s also one rule that runs through the whole system: only one goroutine may hold the AT command port, and every feature queues for it. That makes the “two places send AT commands at the same time and the replies get crossed” class of bug structurally impossible. Part one covered how painful those crossed replies are to track down, so the design simply rules them out.
Getting audio on the module’s own calls
As part one said, ATD handles signaling only. To get sound, you have to build a route inside the module:
| Step | What it does |
|---|---|
| 1 | Enable the module’s ADB and audio features (some firmware versions ship with them on) |
| 2 | Push the voice driver into the module and load it. Kernel 3.18.44 |
| 3 | Run audio calibration |
| 4 | Start the bridge program that connects the DSP’s audio to the USB sound card |
| 5 | Read and write that sound card from the host |
| 6 | Start moving audio only after confirming the call is actually connected |
Step 3 is the easiest to miss, and missing it shows up as pure silence: everything looks fine, there’s just no sound. The calibration program has to run as a daemon; you send it a set command and then wait for the calibration-complete marker to show up in the log.
The trap in step 6: this firmware usually doesn’t send the generic “connected” notification, so you have to poll AT+CLCC to find out.
The whole procedure is wrapped in one script, and VoHive runs it before every outgoing and incoming call:
sudo /opt/vohive-call/qdc507-route status
A healthy state has to show three things at once: the sound card inside the module exists, the routing session is alive, and both data channels are in the running state.
The driver itself is a GPL derivative, and redistributing it carries an obligation to provide source, so the project doesn’t bundle it. It only loads it from a designated directory.
Audio on the first call, silence on every call after
Data: the first call had 7412 non-silent frames out of 7410 (sound almost the whole way through). Every call after that had only 0 to 2, and the network hung up after about 20 seconds.
Cause: the route the bridge program builds is per-call state. Once a call ends, it’s gone.
Fix: restart the bridge program before every call. The script is self-healing: if the module has rebooted (ramdisk wiped, sound card gone), it reloads the driver; if the calibration marker is missing, it recalibrates; and then it restarts the bridge no matter what. A full recovery takes about 13 seconds; the normal path takes 1.4 seconds. After a module reboot, a measured 997 out of 1000 frames were non-silent.
Two adb traps, several hours each
adb shell doesn’t pass along the remote exit code. adb shell test -f xxx “succeeds” even for a file that doesn’t exist. So the “check whether the driver is there, push it if not” step was being silently skipped.
The workaround is to echo a marker on the remote side and check whether it appears in the output. Don’t trust the exit code.
Processes backgrounded inside a one-shot adb shell get reaped when that shell exits. Both the calibration daemon and the bridge program were being killed, and once they were dead, every later write into the calibration pipe would block forever. The symptom was the whole procedure hanging, with no error at all.
The fix is that they have to run under an adb connection that the host side keeps open.
Having a sound card doesn’t mean having sound
A module reboot clears its kernel modules, but the Raspberry Pi may still have the previously enumerated sound card attached. The device node is there, ls can see it, and reading from it yields nothing but zeros.
That is exactly why the status check runs before every call and rebuilds on demand. The device exists, the process is running, the status is true: none of these three proves that data is actually flowing.
The bridge was built, but nothing was driving it
The audio bridge was registered as an endpoint on the bus, but nothing was “pumping” it, so not a single frame ever moved.
The fix was to make the audio bridge itself the bus’s frame pump, clocked by the capture stream. It also counts frames and non-silent frames, so that a “silent route” is visible in the logs instead of waiting for someone to complain that they can’t hear anything.
Hooking up a real telephone
The device is a USB handset with a dial keypad and a hook switch (Koncept KU1110; it’s also sold as AICO TalkPro U-100 and wizarDial WZU-02, same thing).
It isn’t a standard USB audio device. Keys, hook switch and audio all use a proprietary protocol, which had to be reverse engineered by capturing traffic with libusb.
Both directions are now verified on real hardware: on an incoming call it rings, and you pick up and talk; off-hook you hear a dial tone, keys produce key tones, and the number gets dialed out. Speed dial is supported too: configure short codes of one to six digits, and both the handset and the web dial pad expand them before dialing. (# can’t be used as a speed-dial code, because on the handset it’s the “done dialing” key.)
Because it uses libusb, this part has to be compiled natively on the Raspberry Pi; it can’t be cross-compiled:
sudo apt install libusb-1.0-0-dev
CGO_ENABLED=1 go build -tags handset ...
Choppy audio, and recordings full of zero samples
Cause: the transport layer used a plain mutex, and the status poll grabbed it every 200 ms. Playback and capture were simply starved.
Fix: all operations share a single read-write lock. One change fixed both symptoms, the choppiness and the zero samples, because they were the same disease all along.
Playback latency that kept growing
Cause: pacing was driven by a timer, and each call also pre-filled a little, so the backlog snowballed. On a long call, the other party’s voice would end up several seconds behind.
Fix: pace by queue depth instead: top up only when the queue runs low, wait when it’s full. Flush the queue when switching sounds (for example from the dial tone to the call).
Calls hanging up for no apparent reason
Cause: the hook switch (the switch the receiver rests on) is mechanical, and it bounces. A single “on-hook” reading doesn’t mean the receiver was actually put down.
Fix: debounce it. A state only counts after several consecutive identical readings.
When initialization fails, suspect the handset first
Two ordering rules turned out to be critical: a certain init command may only be sent once per channel, and a playback has to be submitted between any two sends; and the status poll must not overlap with initialization, or the start command times out.
But the more important lesson is this: the handset’s firmware hangs easily, and once it’s hung, it fails in exactly the same way no matter which driver you use. Several rounds of debugging were spent hunting for an imaginary bug in code that was completely correct.
The current approach: when initialization fails, first run a minimal reference program to verify the hardware itself. Recovery works by toggling the USB authorization state, with no unplugging required.
Don’t use usbreset, and don’t cut power to a Raspberry Pi 4 USB port. As the previous post explained, its USB ports are ganged: cutting one cuts them all, and takes the module down with it.
Incoming call forwarded to the handset, no sound after picking up
Data: the cellular side’s bus received 699 non-silent frames out of 750, while the handset’s earpiece was dead silent.
Cause: this is a transfer scenario that bridges two buses. The handset was registered as an endpoint on the bus, but nothing was pumping that bus, exactly the same shape as the audio bridge trap above.
Fix: the handset line is driven directly by its own 20 ms clock. Also, the handset is deliberately not a bus member; otherwise its microphone would be mixed into its own earpiece, and you’d hear yourself while talking.
A ringing call hung up as “busy”
Cause: the caller ID notification reports the ringing state repeatedly. The second report was treated as a new incoming call, the session layer replied “busy”, and in doing so hung up the call that was actually ringing.
Fix: ignore repeated reports of the ringing state.
Calling straight from the browser
The calls page in the admin console can dial and answer directly. The browser exchanges raw PCM with the audio bus over a WebSocket:
/api/devices/{id}/call/voice/{session}/ws binary frames, 8 kHz mono 16-bit
No sound in the browser has two causes, both permission-related. Browser audio capture requires a secure context, and if the WebSocket is rejected over a certificate problem there’s no voice, so the self-signed certificate has to be added to the system trust store rather than clicking “proceed anyway” on the warning page (part two covered this). And when microphone permission is denied, it likewise fails silently.
Calls dialed from the web page had signaling but no sound: the dial handler used an empty bus and only set up the signaling session. The fix is to attach the local audio device to that call’s bus at dial time, until the call ends.
Cellular calls answered on the web page had no sound: media was being set up before the call was actually connected. The order was changed to answer first, then set up media.
Looking at these three together is interesting: they’re all cases of “signaling succeeded, so we assumed the whole thing succeeded.” Signaling and media are two independent paths, something part one already said, but it keeps coming back in new forms.
Recording
Recording is a tap on the audio bus, not a recording taken from the WebSocket. That design decision has one nice payoff: calls taken on the USB handset, when no client is streaming at all, still get recorded.
A few details:
- Recorded in stereo: the far end on the left channel, the near end on the right. A mono mixdown can’t be separated afterwards, and the whole point of a recording is usually to tell who said what.
- Alignment: the two directions arrive from different callers, and normally alternate one frame each every 20 ms. If one side stops for more than 1 second, the other side gets padded with silence, instead of waiting idly and then dumping a burst of misaligned frames.
- Disk writes happen on a separate goroutine, so slow storage can’t drag down the audio clock.
- Off by default. Recording involves both parties to a call, and regulations differ from place to place. Retention is governed by two limits, days and total size; setting either to 0 means unlimited.
- Calls with no audio leave no record. Downloads support resuming, so you can drag the seek bar without downloading the whole file.
How it was verified: on a 42.34-second call, the left channel (far end) had a level of 0.142 and the right channel was silent, because nothing was connected on that side at the time. The numbers line up, which shows the channels aren’t swapped.
An ESP32 as an extension
This is a little board called the FoloToy AI Passport: ESP32-C3, 8 MB of flash, no PSRAM, a 240x320 screen and three buttons. It has no cellular capability whatsoever; the goal is to have it borrow the gateway’s line and act as a Wi-Fi phone.
TLS doesn’t fit
During the handshake, mbedTLS peaks at 35 to 45 KB of internal memory, while this board’s largest contiguous free block measured only about 13 KB. An encrypted handshake and a call in progress just don’t fit on the same chip.
I tried shrinking the TLS buffer settings, to no avail: the server is written in Go, and Go’s TLS server doesn’t negotiate the extension that would reduce fragment size.
Fix: the gateway opens two “cheap doors”, both closed by default:
server:
lan_plain_http: true # allow plaintext only for localhost and private ranges
plain_http_cidrs: [] # extra ranges to allow; each non-private one logs a warning at startup
voice_udp_port: 7576 # shared voice UDP port
voice_udp_public_port: 0 # set when behind a port forward with a different external port
One ephemeral port per session can’t be port-forwarded
When the device is on another network it can’t reach the gateway, except through a port forward. But nobody can forward a port that the server picks at random only after the call has started.
The solution is for all sessions to share one fixed port and tell themselves apart with an access key: the device first sends its key to that port, and the server remembers its address and replies with an acknowledgment. Audio from anything not on the known-address list is dropped, and an unrecognized key gets no reply at all. Scanners get silence; a real client costs at most one extra retry.
Using UDP instead of TCP is deliberate: a late voice frame isn’t worth retransmitting, and retransmitting one frame delays every frame behind it.
An admin token has no business sitting on a dev board
Over a plaintext link, anyone who captures a session token gets thirty days of access to the entire API, including eSIM switching and device management.
Fix: add an endpoint that issues restricted tokens, which can only make calls, send and receive SMS, and subscribe to events, and cannot issue new tokens.
To be clear: this only narrows the scope of access; it does not provide confidentiality. Plaintext is plaintext. The real fix is to put the device on a VPN, for example running WireGuard on the ESP32. Its ChaCha20 is far cheaper than a TLS handshake, and the memory is enough.
No echo cancellation
This chip can’t do echo cancellation (the relevant solutions pretty much all require a beefier model plus PSRAM), and the board has no headphone jack, so it’s speaker only.
The firmware’s approach: mute the microphone while the far end is talking (voice-activated switching with a 300 ms hold), plus a push-to-talk button as a deterministic fallback. That reduces echo, but it doesn’t eliminate it. Some limits are simply hardware limits, and admitting that beats pretending to have solved it.
What this Raspberry Pi can do now
| Endpoint | How it connects | Biggest trap |
|---|---|---|
| Module cellular calls | USB sound card | Calibration, rebuilding the route for every call, the two adb traps |
| USB handset | libusb proprietary protocol | Lock starvation, queue backlog, hook-switch bounce, firmware hangs |
| Browser | WebSocket PCM | Certificate trust, empty bus |
| Recording | Bus tap | Two-channel alignment |
| ESP32 | Plaintext HTTP + UDP | Memory, port forwarding, token scope |
That completes the Raspberry Pi side. The next post switches platforms: the same dongle plugged into a Mac, with no serial port and no ALSA, starting over from scratch.
The Full Series
- What Is It, Really
- An SMS and eSIM Gateway on a Raspberry Pi
- VoWiFi from Scratch
- USB Handsets, Browser Calls and an ESP32 (this post)
- On a Mac
- The iPhone Client
- An Android Tablet Drives It Directly
This series is a personal record of learning and technical research. The module’s voice driver is a GPL derivative, is not distributed with the project, and you need to obtain it yourself. If you use the recording feature, please comply with your local laws on call recording.
Comments