# Protocol Analysis and Event Output with Zeek (Part 4)

- URL: https://wuming.si/blog/4027/
- Author: Wuming
- Published: 2025-02-11
- Language: en
- Categories: Technology, English

> Previous: [Suricata DPI and Rule Management](https://wuming.si/blog/4026) · Next: [An Event Pipeline with Flume and Esper](https://wuming.si/blog/4028)

Suricata, covered in the last post, answers one question: does this traffic match a known signature? A lot of detection doesn't work that way. "Which domains did this host resolve today?", "Who signed the certificate on this TLS session?", "What kind of file was just downloaded over HTTP?" To answer those you need a different kind of engine, one that turns raw packets into **structured protocol logs**. In our platform that engine is Zeek (formerly Bro). It sits next to Suricata behind the capture layer and reads the same mirrored traffic. The logs it produces (connections, HTTP, DNS, TLS, files and more) feed the event pipeline, intel matching, anomaly detection and the machine learning modules further down.

This post covers how we run Zeek: the cluster layout, how detection scripts are written, and the output plugins we added so logs stream out in real time instead of sitting in files.

## How Zeek differs from a signature IDS

| Aspect | Signature IDS (e.g. Suricata) | Zeek |
|---|---|---|
| Detection model | Rule match fires an alert | Event-driven; policy written as scripts |
| Main output | Alerts, plus some metadata | Rich structured logs, plus notices |
| State | Per-flow | Full connection/session state, can persist across connections |
| How you extend it | Write rules | Write scripts and plugins |

In short, Suricata is good at catching bad actors and Zeek is good at bookkeeping. The platform needs a complete network ledger with clear semantics before the other detectors have anything to work with.

## Cluster layout

Zeek ships with a cluster mode, started and stopped by its control tool (BroControl / ZeekControl). There are three node roles:

- **worker** processes the packets: protocol detection, stream reassembly, application parsing. Workers are what generate events.
- **proxy** syncs shared state between workers so the manager carries less load.
- **manager** collects logs and notices from the workers, runs global scripts and hands everything to the log writers.

```
  mirrored traffic
        │
        ▼
 capture layer (PF_RING, hash by flow)
        │
 ┌──────┼──────────┬───────────┐
 ▼      ▼          ▼           ▼
worker-1  worker-2  ...  worker-N   ← parse protocols, raise events
        └──────────┼───────────┘
                   ▼
                 proxy               ← shared state
                   ▼
                manager              ← global scripts, notices, log writers
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
      Redis     MongoDB    local files
   (real time) (history)   (fallback)
```

To split traffic across workers, the control tool supports load-balancing plugins. The usual choice is a PF_RING cluster. It hashes the 5-tuple (source/destination IP and port, plus protocol) so every packet of a connection lands on the same worker, and that one process can reassemble the whole stream. An illustrative config:

```ini
[manager]
type=manager
host=localhost

[proxy-1]
type=proxy
host=localhost

[worker-1]
type=worker
host=localhost
interface=<capture-if>
lb_method=pf_ring
lb_procs=8
```

In standalone mode the manager and workers share one machine. When traffic grows, the workers can move to several machines that all report to a single manager.

## Writing detection as scripts

Zeek is built around events. Each time a parser works out a piece of protocol meaning, it raises an event (`http_request`, `dns_request`, `connection_state_remove`, `file_new`, and so on), and scripts subscribe to those events. Here's a small example of my own. It flags connections that stay open a long time but carry almost no data, which is a common pattern for heartbeat-style remote-control channels:

```zeek
module LowAndSlow;

export {
    redef enum Notice::Type += { Long_Quiet_Connection };
    const min_duration = 1 hr &redef;
    const max_bytes    = 1024 &redef;
}

event connection_state_remove(c: connection)
    {
    if ( ! c?$duration ) return;
    local total = c$orig$size + c$resp$size;
    if ( c$duration > min_duration && total < max_bytes )
        NOTICE([$note=Long_Quiet_Connection,
                $msg=fmt("quiet connection, %d bytes", total),
                $conn=c]);
    }
```

You can write many scripts like this: suspicious User-Agents, SQL keywords in URIs, very long random-looking subdomains, repeated 404s on admin paths. Each one is a weak signal and noisy by itself. They enter the pipeline as notices, and that's where they can be correlated with other evidence.

## Streaming logs out: custom writer plugins

Out of the box, Zeek writes logs to local text files and rotates and compresses them every hour. That's fine for forensics after the fact. It isn't enough for a real-time detection platform, where downstream modules need every DNS and HTTP record within seconds.

Zeek's logging framework has layers: **log stream → filter → writer**. We added two writer plugins at the bottom layer:

- **Redis writer.** It serializes each log record to JSON and publishes it to a Redis pub/sub channel, with one channel per log type (connections, HTTP, DNS, TLS, files, mail, notices, and so on).
- **MongoDB writer.** It writes one collection per log type, for cases where you need to query history by time.

The plugins are written in C++ and register a new writer type through Zeek's plugin API. On the script side, all you do is attach a filter to each log stream at startup. The idea, simplified:

```zeek
event zeek_init() &priority=-5
    {
    for ( id in Log::active_streams )
        {
        if ( id in RedisOut::excluded ) next;
        if ( |RedisOut::only| > 0 && id !in RedisOut::only ) next;
        Log::add_filter(id, [$name="to-redis",
                             $writer=Log::WRITER_REDIS,
                             $interv=0 sec]);   # no buffering, send each record
        }
    }
```

Two sets decide what gets sent. `only` is an allowlist: when it isn't empty, only those streams go out. `excluded` is a denylist for noisy streams such as packet-filter and internal reporter logs. The plugin also exposes a few BIFs (built-in functions, implemented in C++ and callable from scripts) so scripts can `set`, `get` or `publish` against Redis directly. That's useful for caching lookups or pushing custom alerts.

A DNS record on the wire looks roughly like this. We keep Zeek's original field names:

```json
{
  "ts": 1700000000.123456,
  "uid": "C<random>",
  "id.orig_h": "<client>", "id.orig_p": 53211,
  "id.resp_h": "<resolver>", "id.resp_p": 53,
  "proto": "udp",
  "query": "example.com",
  "qtype_name": "A",
  "rcode_name": "NOERROR",
  "answers": ["<addr>"]
}
```

`uid` matters a lot. The conn, http and files records for one connection all carry the same uid, and that's how downstream consumers stitch them back into a single session.

## Two extras: files and certificates

**File extraction plus YARA.** Zeek's file analysis framework can rebuild files carried over HTTP, SMTP, FTP and other protocols. We attach the extract analyzer to executables, Office documents, scripts and similar types. Once a file is on disk we scan it with YARA, a rule language that describes malware by strings and conditions. Our rules are grouped by purpose: exploit documents, samples for specific CVEs, and web shells in various languages. A generic web-shell rule looks like this:

```yara
rule php_eval_webshell_demo {
    strings:
        $e = "eval("
        $p = "$_POST"
        $b = "base64_decode"
    condition:
        $e and $p and $b
}
```

A match produces a notice. The file itself also goes on to the sandbox and multi-engine AV scanning (Parts 10 and 11).

**Weak certificate keys.** A community plugin checks whether the RSA public key in an X.509 certificate is affected by ROCA (CVE-2017-15361, a flaw that made RSA keys from certain hardware factorable). It's a typical use for a Zeek plugin: a detection that has to run one computation inside a certificate event.

## Performance tuning

- **Worker count.** A common rule of thumb is CPU cores minus two, which leaves room for the manager and the OS.
- **Memory.** Plan for 2-4 GB per worker. The connection and DNS state tables live in memory, and any table you define in a script needs an expiry (`&create_expire` and friends), or it will keep growing.
- **Packet loss.** Check drop rates with `capstats`/`netstats` first. Then increase kernel receive buffers and PF_RING slot counts, or add workers.
- **CPU pinning.** Pin workers to dedicated cores to avoid scheduler jitter, and keep them on the same NUMA node as the NIC queues where you can.

## Pitfalls and trade-offs

1. **Pub/sub doesn't guarantee delivery.** Redis pub/sub is fire-and-forget. If a subscriber is disconnected or falls behind, messages are dropped. For high-volume logs where losing a few records is acceptable, that's fine. Important alerts should go through a buffered queue instead (a List, a Stream, or Kafka).
2. **Don't send everything.** Streaming every log type will flatten Redis and every consumer behind it. Use the allowlist to send only the types someone actually consumes, and leave the rest in local files.
3. **Dotted field names.** Names like `id.orig_h` are fine in JSON, but Elasticsearch treats them as object paths. We rename them to flat fields such as `src_ip`/`dst_ip` in the downstream pipeline.
4. **Duplicate work with Suricata.** Both engines parse HTTP, DNS and TLS, so some CPU goes to waste. Our split: Suricata owns signatures and the main file-extraction path, Zeek owns the complete protocol ledger and scripted detection, and each one turns on only the protocol metadata it actually needs to emit.
5. **Put a cap on script state.** Cross-connection counters (say, failed logins per IP per hour) are easy to write. Without expiry and size limits, though, a single scan storm can eat all the memory.

## Wrap-up

In this platform Zeek is the network ledger. It turns packets into meaningful events and logs, and our custom writer plugins push them downstream in real time. Zeek can detect things on its own, but most of its value is the clean data it gives every detector after it.

Next we follow that data downstream: an event pipeline built on Flume and Esper that collects events from several engines, aggregates and correlates them, and routes the results out.

## The Series

1. [The Big Picture](https://wuming.si/blog/4024)
2. [10G Packet Capture](https://wuming.si/blog/4025)
3. [Suricata DPI and Rule Management](https://wuming.si/blog/4026)
4. [Protocol Analysis with Zeek](https://wuming.si/blog/4027)
5. [An Event Pipeline with Flume and Esper](https://wuming.si/blog/4028)
6. [Threat Intel Matching and the Knowledge Base](https://wuming.si/blog/4029)
7. [Rule-Based Anomaly Detection](https://wuming.si/blog/4030)
8. [Machine Learning Detection](https://wuming.si/blog/4031)
9. [Alert Correlation, Noise Reduction and Aggregation](https://wuming.si/blog/4032)
10. [Dynamic Sandbox](https://wuming.si/blog/4033)
11. [Multi-Engine AV Scanning](https://wuming.si/blog/4034)
12. [The Management Console](https://wuming.si/blog/4035)
13. [A Lightweight Sensor](https://wuming.si/blog/4036)
14. [Deployment, Versioning, Upgrade and Rollback](https://wuming.si/blog/4037)
15. [Ops Monitoring and Self-Healing](https://wuming.si/blog/4038)

---

*This series is a personal study of system architecture. Details are abstracted and anonymized; it does not describe any specific commercial product or any organization's internal configuration or data.*
