104 lines
3.1 KiB
Markdown
104 lines
3.1 KiB
Markdown
|
# unisonus-livekit
|
|||
|
|
|
|||
|
|
WebRTC audio infrastructure for **Unisonus** — a choir/music streaming application. Provides a LiveKit server and a custom choir audio mixer agent that subscribes to participant audio streams, mixes them, and publishes the combined output back to the room.
|
|||
|
|
|
|||
|
|
## Architecture
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Participants (up to 6)
|
|||
|
|
│ audio tracks
|
|||
|
|
▼
|
|||
|
|
choir-mixer agent
|
|||
|
|
├── subscribes to all participant audio
|
|||
|
|
├── normalises RMS per stream (target: -20 dBFS)
|
|||
|
|
├── noise-gates signals below -40 dBFS
|
|||
|
|
├── sums all streams
|
|||
|
|
├── soft-limits via tanh (no hard clipping)
|
|||
|
|
└── publishes mixed audio track back to room
|
|||
|
|
│
|
|||
|
|
▼
|
|||
|
|
All room participants receive the mix
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Services
|
|||
|
|
|
|||
|
|
| Service | Image | Ports | Description |
|
|||
|
|
|---------|-------|-------|-------------|
|
|||
|
|
| `livekit` | `livekit/livekit-server:latest` | `7880` (HTTP/WS), `7881` (RTC TCP), `50000-50100/udp` | LiveKit WebRTC media server |
|
|||
|
|
| `choir-mixer` | Built from `./choir-mixer/Dockerfile` | — | Python agent: subscribes, mixes, publishes |
|
|||
|
|
|
|||
|
|
## Audio Mixer Details
|
|||
|
|
|
|||
|
|
**File:** `choir-mixer/main.py` + `choir-mixer/mixer.py`
|
|||
|
|
|
|||
|
|
| Parameter | Value |
|
|||
|
|
|-----------|-------|
|
|||
|
|
| Sample rate | 48 kHz |
|
|||
|
|
| Channels | Mono (1) |
|
|||
|
|
| Frame duration | 20 ms (960 samples/frame) |
|
|||
|
|
| Max simultaneous streams | 6 |
|
|||
|
|
| Target loudness | -20 dBFS (RMS) |
|
|||
|
|
| Noise gate | -40 dBFS (silence below this threshold) |
|
|||
|
|
| Limiter | tanh soft-limit (smooth saturation, no hard clipping) |
|
|||
|
|
| Stale frame timeout | 60 ms (frames older than this are discarded from mix) |
|
|||
|
|
|
|||
|
|
The mixer runs at ~50 Hz (every 20 ms). Each cycle it:
|
|||
|
|
1. Collects the most recent frame from each active participant (discarding stale frames)
|
|||
|
|
2. Normalises each stream to -20 dBFS
|
|||
|
|
3. Noise-gates signals below -40 dBFS (suppresses background noise and keyboard clicks)
|
|||
|
|
4. Sums all streams into one
|
|||
|
|
5. Applies tanh soft limiting to prevent clipping
|
|||
|
|
6. Publishes the result as a mono 48kHz audio track
|
|||
|
|
|
|||
|
|
## LiveKit Server Config
|
|||
|
|
|
|||
|
|
**File:** `livekit.yaml`
|
|||
|
|
|
|||
|
|
| Setting | Value |
|
|||
|
|
|---------|-------|
|
|||
|
|
| HTTP/WS port | 7880 |
|
|||
|
|
| RTC TCP port | 7881 |
|
|||
|
|
| RTC UDP range | 50000–50100 |
|
|||
|
|
| Node IP | 192.168.0.241 |
|
|||
|
|
| External IP | Disabled (LAN only) |
|
|||
|
|
| Room empty timeout | 300 s |
|
|||
|
|
| Max participants | 50 |
|
|||
|
|
| Log level | info |
|
|||
|
|
|
|||
|
|
## Environment Variables
|
|||
|
|
|
|||
|
|
| Variable | Required | Description |
|
|||
|
|
|----------|----------|-------------|
|
|||
|
|
| `LIVEKIT_API_KEY` | Yes | LiveKit API key (from `livekit.yaml` `keys:`) |
|
|||
|
|
| `LIVEKIT_API_SECRET` | Yes | LiveKit API secret |
|
|||
|
|
|
|||
|
|
Copy `.env.example` to `.env` and fill in the values. The API key/secret pair must match what is configured in `livekit.yaml`.
|
|||
|
|
|
|||
|
|
## Running
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
docker compose up -d
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
To check mixer logs:
|
|||
|
|
```bash
|
|||
|
|
docker compose logs -f choir-mixer
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Dependencies
|
|||
|
|
|
|||
|
|
**Python** (choir-mixer):
|
|||
|
|
- `livekit` — LiveKit Agents SDK
|
|||
|
|
- `livekit-rtc` — RTC bindings
|
|||
|
|
- `numpy` — audio processing
|
|||
|
|
|
|||
|
|
## Deployment
|
|||
|
|
|
|||
|
|
Runs on `192.168.0.241` (albert-MacBookPro Linux server).
|
|||
|
|
|
|||
|
|
| Endpoint | URL |
|
|||
|
|
|----------|-----|
|
|||
|
|
| LiveKit signaling | `ws://192.168.0.241:7880` |
|
|||
|
|
| LiveKit RTC TCP | `192.168.0.241:7881` |
|
|||
|
|
| LiveKit RTC UDP | `192.168.0.241:50000-50100` |
|