# Overview

Source: https://docs.gryt.chat/docs/sfu

The Go SFU that carries voice, video and screen share

The SFU is the media plane. Every participant sends their audio, camera and
screen share to it once, and it forwards each of those streams on to everybody
else in the room. Nothing is transcoded on the way through — the packets that
arrive are the packets that leave.

It is a Go service built on [Pion WebRTC v4](https://github.com/pion/webrtc),
and it is deliberately small: one of it runs per machine, shared by every Gryt
server on that machine.

## How it handles media

Forwarding is the whole job. A room of five people means each client sends one
copy up and receives four down, so the SFU spends almost no CPU compared to
something that mixes streams together, and upload bandwidth is what runs out
first.

Streams that carry the Dependency Descriptor RTP header extension get parsed for
their temporal layer, which means the forwarder is able to drop the higher
layers for a receiver that cannot keep up. It does not do that on its own.
Every receiver gets every layer unless a client asks for fewer with `set_layer`,
so the frame rate you configured is the frame rate you get, instead of sagging
whenever a bandwidth estimate dips.

Receiver feedback is relayed selectively. A receiver reporting a lost picture
(PLI) or asking for a full intra refresh (FIR) results in the SFU asking the
original sender for a keyframe. Bandwidth estimates (REMB) go no further than
the SFU and play no part in choosing layers. The sender's own congestion
controller already has better information from transport-cc, which measures the
sender's link to the SFU instead of somebody else's link away from it. The
reasoning is written down in `internal/signaling/coordinator.go` if you want the
longer version.

The SFU never sees message content. Text, identity and permissions belong to the
Gryt server. Room IDs, peer IDs and RTP are all it knows about.

## Codecs

Registered at startup, in `registerCodecs`. The client picks with
`setCodecPreferences`; the SFU forwards whatever gets negotiated.

| Kind | Codecs |
|------|--------|
| Audio | Opus (48 kHz stereo, in-band FEC), G.722, PCMU, PCMA |
| Video | H.264 (five profile/packetization combinations), VP9 (two profiles), VP8, AV1 |

H.264 leads the video list because every current GPU encodes it in hardware —
NVENC, Quick Sync, AMF. AV1 is there for hardware new enough to want it.

## Configuration

Every variable the SFU reads. There are no others.

| Variable | Default | What it does |
|----------|---------|--------------|
| `SFU_PORT` | `5005` | HTTP and WebSocket port. `PORT` is read as a fallback |
| `ICE_UDP_MUX_PORT` | `3478` | The one UDP port all media flows over |
| `STUN_SERVERS` | `stun:stun.l.google.com:19302` | Comma-separated, used both for ICE and for the startup readiness probe |
| `DISABLE_STUN` | `false` | Stop discovering server-reflexive candidates |
| `ICE_ADVERTISE_IP` | — | Comma-separated IPs to advertise as host candidates instead of the ones found on the interfaces |
| `MAX_PEERS` | `200` | How many peers may be connected at once, across all rooms |
| `DEBUG` | `true` | Room, connection and signaling logging. On unless explicitly set to `false` |
| `VERBOSE_LOG` | `false` | Per-packet RTP forwarding detail |

### One UDP port, not a range

All media shares `ICE_UDP_MUX_PORT`. One port is far easier to get through a
firewall than a range, and restrictive networks that drop UDP on high ports
often let a port they recognise through.

The default is 3478, the IANA STUN port. It needs no privileged bind, and it is
the UDP port a locked-down network has most likely already opened, since
Microsoft Teams requires outbound 3478–3481.

UDP 443 looks like the better choice and generally is not. Enterprise firewall
vendors recommend blocking it, because QUIC on UDP 443 cannot be TLS-inspected
and blocking it forces browsers back to TCP where it can be. On the networks you
would pick 443 to get through, it is the one most likely to be shut on purpose.

Ports are per-protocol either way. UDP 443 and TCP 443 are different sockets, so
an HTTPS server never collides with media on UDP 443. HTTP/3 does, because that
is UDP 443 as well, and Caddy serves
it by default.

The SFU binds this port at startup and refuses to start if something else holds
it. That is deliberate: the alternative is coming up healthy and handing out
candidates on ports nobody opened.

<Callout type="warn" title="The port range is gone">
`ICE_UDP_PORT_MIN` and `ICE_UDP_PORT_MAX` were removed. If they are still set
anywhere, they are ignored. A range never bought capacity — one muxed port
carries far more peers than a machine has CPU and upload bandwidth for. If you
genuinely outgrow one SFU, run a second one.
</Callout>

### MAX_PEERS

A guardrail on the machine rather than a limit the port imposes. When the count
of peers across every room reaches it, the next client to join is refused with
"there are no seats left in this voice server".

This is separate from the Gryt server's own `VOICE_MAX_USERS`, which caps a
single server's voice channels. The SFU's cap covers every server sharing it.

## Endpoints

| Endpoint | What it does |
|----------|--------------|
| `GET /health` | `{"status":"healthy","service":"sfu","version":…,"timestamp":…}` |
| `GET /metrics` | Prometheus |
| `GET /` | WebSocket upgrade. Anything else gets 400 and a sentence saying so |
| `WS /server` | The Gryt server's control connection |
| `WS /client` | A participant's connection |

### Health has a starting state

Before the SFU reports healthy it sends a STUN Binding Request and waits for a
reply, retrying up to eight times with a backoff that grows to five seconds.
Until that succeeds, `/health` answers **503** with `{"status":"starting",
"detail":"verifying UDP connectivity"}`.

This exists because Docker Desktop on Windows and macOS can finish starting a
container before its UDP forwarding rules are in place, which used to break ICE
on the very first `docker compose up` and look like a configuration problem.
With `DISABLE_STUN=true` or no STUN servers, the probe is skipped and the SFU is
ready immediately. If all eight attempts fail it gives up and reports healthy
anyway, with a warning in the log.

### Metrics

Four gauges, resynced from real state every fifteen seconds:
`gryt_sfu_rooms_active`, `gryt_sfu_peers_active`,
`gryt_sfu_websocket_connections_active`, `gryt_sfu_tracks_active`. Standard Go
runtime metrics come along with them.

## The WebSocket protocol

Raw WebSocket, not Socket.IO. Every message is
`{ "event": "<name>", "data": "<json string>" }` — note that `data` is a string
containing JSON, not a nested object.

There are two kinds of connection, and they are told apart by path. `/server` is
the Gryt server's control channel. `/client` (and anything else) is a
participant.

### The server's connection

The Gryt server connects once and registers each voice room. Every message
after registration carries `server_id` and `server_password`, and is dropped
without a reply if they do not match.

| Event | Direction | Payload |
|-------|-----------|---------|
| `server_register` | server → SFU | `{ server_id, server_password, room_id }` |
| `disconnect_user` | server → SFU | `{ room_id, user_id, server_id, server_password }` |
| `user_audio_control` | server → SFU | `{ room_id, user_id, server_id, server_password, is_muted, is_deafened }` |
| `sync_request` | server → SFU | `{ server_id, server_password }` |
| `sync_response` | SFU → server | `{ rooms: [{ room_id, user_ids }] }` |
| `peer_joined` / `peer_left` | SFU → server | `{ room_id, user_id }` |
| `keep_alive` | either | `{ timestamp }` |

`sync_request` is how the server reconciles after a restart: it asks who is
actually connected rather than trusting its own record.

### A participant's connection

The first message has to be `client_join`. Anything else and the connection is
closed with "Expected client_join event".

| Event | Direction | Payload |
|-------|-----------|---------|
| `client_join` | client → SFU | `{ room_id, server_id, server_password, user_token, user_id }` |
| `room_joined` | SFU → client | — |
| `room_error` | SFU → client | The reason, as a string |
| `offer` | SFU → client | SDP |
| `answer` | client → SFU | SDP |
| `candidate` | both | `RTCIceCandidateInit` JSON |
| `renegotiate` | client → SFU | — |
| `set_layer` | client → SFU | `{ track_id, max_temporal_layer }` |
| `keep_alive` | client → SFU | `{ timestamp }` |

`max_temporal_layer` is `-1` for every layer, or `0`, `1`, `2` for T0, T0+T1 and
T0+T1+T2. It only does anything on a stream that actually carries the Dependency
Descriptor extension.

## A call, start to finish

1. The Gryt server registers the room over `/server`.
2. A client connects to `/client` and sends `client_join`. The SFU checks the
   room exists and the server credentials match, then checks `MAX_PEERS`.
3. A peer connection is created with receive-only transceivers for audio, video,
   screen-share video and screen-share audio.
4. The SFU offers, the client answers, candidates go both ways.
5. Each track the client publishes gets a `LayerForwarder`: one goroutine
   reading RTP, parsing the Dependency Descriptor if present, and writing to one
   local track per receiver.
6. Each forwarder sends the sender a PLI every two seconds, and relays any PLI
   or FIR from a receiver as a PLI to the sender. FIR is translated to PLI
   rather than passed through.
7. On leave, tracks, forwarders and the peer connection are torn down, and the
   server is told with `peer_left`.

Rooms that have been empty for thirty minutes are removed by a sweep that runs
every five minutes.

## Layout

```
sfu/
├── cmd/sfu/            # main: config, codecs, the mux, HTTP, cleanup loops
└── internal/
    ├── config/         # every environment variable, in one place
    ├── readiness/      # the STUN probe /health gates on
    ├── recovery/       # panic containment around every goroutine and handler
    ├── room/           # rooms, registration, peer bookkeeping, cleanup
    ├── signaling/      # offer/answer coordination, RTCP relay
    ├── svc/            # Dependency Descriptor parser and LayerForwarder
    ├── track/          # track lifecycle
    ├── webrtc/         # peer connection management
    ├── websocket/      # the two connection kinds and their handlers
    └── metrics/        # the four gauges
```

## Running it

```bash
cd packages/sfu
cp env.example .env
go run ./cmd/sfu
```

```bash
go test ./...
go run -race ./cmd/sfu
```

Most people never do this. `gryt` runs one SFU per machine and shares it between
every server there — see the [CLI docs](https://docs.gryt.chat/docs/cli).