Continuous replication with local fsync-backed checkpoints, multi-worker lease failover, and decoupled S3 archiving. Built for MySQL DBAs who need deterministic RPO and real operational control — not fragile shell scripts.
DURABILITY Local fsync precedes checkpoint advance (zero optimistic progress)
ARCHIVE Decoupled S3 upload (upload failure never stalls binlog capture)
CLUSTER Multi-worker lease ownership with automatic failover & zero split-brain
OBSERVE True lag vs source binlog tip, Prometheus metrics, and embedded Web UI
Operator Console & Interfaces
BinlogServer embeds a production web console and comprehensive REST API directly into the release binary. No separate frontend service to build, deploy, or maintain.
Install & First Backup Path
Packaged release archives contain precompiled statically-linked binaries (binlog-server, migrate), SQL migrations, and production config templates. Operators do not need Go installed.
Fetch the official static binary, verify its SHA256 checksum, unpack the archive, generate a task key, and boot local evaluation.
VER=0.5.83 OS=linux ARCH=amd64
curl -fsSL -O "https://github.com/Fanduzi/BinlogServer/releases/download/v${VER}/binlog-server_${VER}_${OS}_${ARCH}.tar.gz"
curl -fsSL -O "https://github.com/Fanduzi/BinlogServer/releases/download/v${VER}/checksums.txt"
sha256sum -c checksums.txt --ignore-missing
tar -xzf "binlog-server_${VER}_${OS}_${ARCH}.tar.gz"
cd "binlog-server_${VER}_${OS}_${ARCH}"
# Loopback bind allows unauthenticated evaluation.
# Generate a 0600 task key first. The command prints the path and key id, not the key.
./binlog-server task-key generate --out ./task.key
export BINLOG_SERVER_STANDALONE_TASK_KEY_FILE="$PWD/task.key"
export BINLOG_SERVER_LISTEN_ADDR=127.0.0.1:8080
export BINLOG_SERVER_DATA_DIR=./data
./binlog-server
Check health probe status and submit your MySQL source specification with explicit retention and start mode.
# Check health probe
curl -fsS http://127.0.0.1:8080/healthz
# Submit backup task (start from source tip)
curl -fsS -X POST http://127.0.0.1:8080/api/tasks \
-H 'Content-Type: application/json' \
-d '{
"name": "prod-mysql-01",
"cluster_key": "prod-cluster-main",
"source": {
"host": "192.168.1.50",
"port": 3306,
"user": "repl",
"password": "your_repl_password",
"flavor": "mysql"
},
"start": { "mode": "LATEST" },
"storage": { "retention_days": 7 }
}'
Trigger stream capture and immediately inspect real master lag, file rotation, and GTID coordinates in the embedded Web UI.
# Start continuous replication (use task id from step 2)
curl -i -X POST http://127.0.0.1:8080/api/tasks/<task-id>/start
# Open Web Console in browser
# http://127.0.0.1:8080/ui/
# Or query live task state via API
curl -fsS http://127.0.0.1:8080/api/tasks
Transition to production with non-negotiable security: AES-256 key, bearer auth token, and dedicated metadata MySQL.
# 1. Initialize dedicated metadata database
export META_DSN='binlog_meta:pass@tcp(10.0.0.15:3306)/meta?parseTime=true'
./migrate up --dsn "$META_DSN" --path ./migrations
# 2. Start production daemon with encryption key
export PRODUCTION=true
export BINLOG_SERVER_META_DSN="$META_DSN"
export BINLOG_SERVER_API_AUTH_BEARER_TOKEN="your-secure-token"
export BINLOG_SERVER_ENCRYPTION_KEY="$(openssl rand -hex 16)"
./binlog-server \
--config config.production.example.yaml \
--encryption-key "$BINLOG_SERVER_ENCRYPTION_KEY"
⚠️ Security notice: Loopback binds allow unauthenticated local testing. Any non-loopback listen address (including :8080 or 0.0.0.0:8080) strictly fail-closes at startup unless api.auth.enabled, protect_api, and protect_metrics are enabled.
MariaDB sources must set "flavor":"mariadb". The Console default and the MySQL Quick Start example use mysql. That flavor reads @@server_uuid, which MariaDB does not have, and the task fails immediately.
Architectural Guarantees
DBAs cannot afford optimistic progress or unhandled edge cases. Every component in BinlogServer is built around predictable durability and operational transparency.
Checkpoint offsets advance only after local fsync succeeds on disk. A process crash or sudden reboot never causes phantom progress.
S3-compatible upload is asynchronous and best-effort. Cloud storage throttling or outages never stall local replication streams.
Distributed lease locks backed by metadata MySQL ensure only one worker dumps a source at any time, with automatic timeout failover.
Non-loopback binds mandate authentication. PRODUCTION=true refuses startup without a 32-byte AES key and encrypts source passwords.
Deployment Topologies
Whether testing locally on a laptop or running hundreds of replication tasks across an enterprise cluster, choose the right architecture without changing binlog format or client protocols.
Single process. Without meta_dsn, tasks run in-memory for testing. With meta_dsn, tasks, checkpoints, and sealed segments persist across restarts for simple single-node deployments.
Production recommended. Dedicated Control Plane nodes serve API/UI/Scheduler, while a pool of headless Workers claim task leases and perform binlog replication with automatic crash failover.
A hybrid node running both the Control Plane API and Worker runner backed by a shared metadata MySQL. Ideal for small, cost-effective high-availability clusters.
Data Path & Lifecycle
Every step in the pipeline adheres to explicit transactional boundaries, making disaster recovery and auditing completely transparent.
Support LATEST (start at source tip), FILE_POS (specific binlog file and offset), or GTID sets without losing consistency.
Replication lag is evaluated directly against the source binlog position (SHOW MASTER STATUS, or SHOW BINARY LOG STATUS on MySQL 8.4+), avoiding misleading zeros during idle periods.
Configurable retention (1..3650 days) cleans up expired sealed files on disk. Upload failures trigger backoff and explicit retry APIs.
Positioning
Products are easier to trust when they draw hard boundaries. BinlogServer is a specialized MySQL binlog backup control plane, not a bloated generic data platform.
| Dimension | BinlogServer Design | Why DBAs Trust It |
|---|---|---|
| Durability Contract | Checkpoint advances only after local fsync succeeds on disk. | Zero phantom progress; resumes safely on restart. |
| Archive Decoupling | S3 upload runs asynchronously with explicit retry endpoints. | Cloud storage downtime never stalls binlog replication. |
| High Availability | MySQL lease locks with heartbeat expiration and automatic takeover. | Prevents split-brain binlog dumpers on the same source. |
| Product Scope | Focused purely on binlog capture, storage, and PITR foundation. | No heavy JVM dependencies, Kafka clusters, or complex schema registries. |
Fit Assessment
Evaluate BinlogServer against your real operational requirements:
If your mandate is ensuring MySQL binlogs are captured reliably, persisted with rock-solid checkpoints, and archived without risking primary database operations, BinlogServer is the exact tool for the job.
Operational Evidence
We do not ask DBAs to trust marketing claims. BinlogServer validates its failure modes and recovery paths through comprehensive automated E2E test suites.
Dual-worker lease takeover, worker crash recovery, and control plane failover tested against live MySQL engines.
S3 upload disconnection and backpressure tests prove capture loops continue writing local files without interruption.
Production mode asserts valid 32-byte AES-256 keys and encrypted source credentials before binding network sockets.
Dashboard and task summaries use SQL GROUP BY aggregations with bounded LIMIT/OFFSET paging to protect metadata databases.
Before Production
Isolate the metadata database, turn on authentication and encryption, and decide how long sealed binlogs stay on local disk.
Ensure the metadata MySQL instance is dedicated and never added to the backup task set. Run ./migrate up before starting.
Use config.production.example.yaml with BINLOG_SERVER_API_AUTH_BEARER_TOKEN and supply 32-byte --encryption-key.
Set storage.retention_days (1..3650) to manage local disk space while streaming sealed binlogs to object storage.
What changed in each version: release notes and CHANGELOG.
Read the step-by-step deployment guide, inspect the real production configuration, and replace brittle cron jobs with an authoritative backup engine.