$ show matrix-org/synapse-s3-storage-provider
This repository is a storage provider plugin for Synapse, the Matrix homeserver, that stores media in S3-compatible object storage. It is a Python package installed as a module and loaded by Synapse at runtime. The core is a single module, s3_storage_provider.py, which implements Synapse's storage provider API and translates local media operations into S3 GET/PUT/HEAD/DELETE calls via boto3. It supports SSE-C encryption, custom S3 endpoints, and a local cache. A companion CLI script, s3_media_upload, migrates existing local media to S3. Architecturally, the system is a library/plugin rather than a standalone service. Synapse invokes the provider's methods to store and retrieve media; the provider uses background threads (_S3DownloadThread, _S3UploadThread) to perform asynchronous operations with backpressure. The code uses threading locks and events to coordinate concurrent access, and includes tests, CI workflows, and release automation. The primary entry point is the S3StorageProvider class instantiated by Synapse; the module can also be run as a script for maintenance, and scripts/s3_media_upload provides a CLI. External dependencies include boto3 for S3 access, Synapse as the host application, and Twisted for tests. The repository is actively maintained with recent commits, dependabot updates, and GitHub Actions for CI and releases.
The LLM identified 2 files worth highlighting for reuse:
Core module implementing Synapse's storage provider interface. It manages S3 client lifecycle, SSE-C encryption, and async upload/download via background threads with backpressure. The S3 interaction layer is isolated enough to serve as a reference for other storage backends.
Standalone command-line tool that walks a local media directory, checks S3 for existing objects, and uploads missing files with progress bars. It is a safe, resumable migration pattern for moving media to S3.