$ show yoshuawuyts/futures-concurrency
This repository is a Rust library providing structured concurrency primitives for async programming. It offers operations such as `join`, `race`, `try_join`, `race_ok` for futures, and `merge`, `chain`, `zip` for streams, supporting tuples up to 12 elements, arrays, and `Vec` (with alloc feature). It also includes `FutureGroup` and `StreamGroup` for dynamic collections, and a `ConcurrentStream` trait for processing streams concurrently with backpressure and bounded concurrency. The library is runtime-agnostic (works with tokio, async-std, smol, etc.) and supports `no_std` environments. The code extensively uses pin projection, `ManuallyDrop`, and custom waker arrays with bit-set readiness tracking to achieve efficient polling and cancellation. It provides benchmark comparisons against `futures-rs` demonstrating competitive performance. There are no data pipelines, notebooks, or research elements; this is a pure library for async concurrency.
The LLM identified 3 files worth highlighting for reuse:
A `ChunkedVec` that stores elements in fixed-size heap-allocated chunks, never moving elements after insertion. It provides stable indices, making it safe for storing pinned futures/streams. The struct is `Unpin` regardless of element type. It supports `insert`, `remove_in_place`, `get`, `get_mut`, and automatic growth. This is a foundational utility for `FutureGroup` and `StreamGroup`, but can be extracted and reused in any scenario requiring pinned storage with dynamic growth and removal.
A `ReadinessVec` that tracks which futures/streams are ready to be polled using a `FixedBitSet`. It maintains a count of ready indices and supports `set_ready`, `clear_ready`, `any_ready`, `set_all_ready`, and `resize`. The bit-set approach is memory-efficient and allows O(1) checks. This pattern avoids redundant wakeups by only waking the parent waker when a previously non-ready future becomes ready. It can be reused in any executor or combinator that needs to efficiently track readiness of a dynamic collection of tasks.
A `MaybeDone` enum that wraps a future and allows taking its output (success or error) after completion. It has three states: `Future` (not yet completed), `Done` (completed with output), and `Gone` (output taken). It provides specialized `take_ok` and `take_err` methods for `Result`-returning futures. This is a simple but reusable pattern for implementing combinators that need to inspect or take the output of a future after polling it to completion, without consuming the wrapper.