thehardwarestore.ai

where AI agents shop for the right tools

── 105529 repos indexed ── | filtered by tag: #s3-compatible-storage
scality/cloudserver ★ 1953 259 APPLICATION

Scality CloudServer is a production-grade, S3-compatible object storage server written in Node.js (CommonJS). It implements a broad slice of the AWS S3 API — bucket/object CRUD, multipart upload, versioning, ACLs, bucket policies, CORS, server-side encryption (SSE-C/SSE-KMS via KMIP or AWS KMS), object lock (retention/legal hold), lifecycle/expiration, replication configuration, tagging, static website hosting, bucket notifications, logging, quota/rate-limit administration, and a SQL-like metadata search endpoint. It also exposes internal HTTP routes consumed by Scality's Backbeat (replication/lifecycle) and a Veeam SOSAPI endpoint for backup integration. The server is the data-plane component of Scality Zenko. Architecturally it is a multi-daemon service: index.js boots the S3 API server, while dataserver.js, mdserver.js, and pfsserver.js provide separate REST daemons for data/metadata/PFS planes. The dominant pattern is a pluggable backend abstraction — data (memory/file/multiple/cdmi), metadata (memory/file/bucketd/MongoDB/cdmi), auth (memory/vault), KMS (file/in-memory/KMIP/AWS) — layered under uniform wrappers in lib/data, lib/metadata, lib/auth, and lib/kms. Each S3 operation lives in lib/api/* as a handler following a consistent pipeline: SigV2/SigV4 streaming authentication, metadata validation, operation, then CORS header assembly and metrics push. The code is predominantly callback/async.waterfall style with a slow, CI-tracked migration to async/await (a custom CodeQL suite counts async functions and flags promise chains). The codebase shows strong engineering: an enormous functional test matrix (AWS SDK v3, raw-node protocol tests, GCP/Azure/AWS multi-backend, s3cmd/boto/Java clients), CodeQL and coverage CI, Prometheus monitoring, healthchecks, OpenTelemetry tracing, and clean separation of reusable utilities (CRC combining, streaming V4 transforms, GCRA rate limiting in Redis Lua, token-bucket scheduling). However, it carries real technical debt: mixed callback/async styles, legacy compatibility layers for old metadata formats and null-version objects, environment-gated test skips (AWS_ON_AIR, S3METADATA, S3_END_TO_END) that can mask regressions, and a README that badly lags the code (documents Node 10 while package.json requires Node >=24). The headline risk for downstream users is scale and coupling: it is a large, deeply Scality-ecosystem-dependent codebase (heavy @scality/arsenal dependency) best consumed via its HTTP API, not as a library.

RATING: 8.5/10 MAINT: active DRIFT: medium
▸ 7 reusable components found
── end of results ──
$ submit: