dragonfly

Dragonfly v2.1

This release is important because starting from this version, it changes how teams can use our software in production.

Starting from v2.1.0 the project moves from BUSL-1.1 to the Dragonfly Source-Available License (DSAL).

For more details, please read https://www.dragonflydb.io/blog/introducing-dragonfly-enterprise

Besides this much of the work is hardening. ACL enforcement is stricter, credentials are redacted from logs, and many crashes and deadlocks are fixed across blocking commands, tiered storage, Lua and search. Search gains index aliases, multi-attribute fields and punctuation-aware queries. There is also an experimental append-only-file (AOF) persistence mode, and a number of throughput and memory improvements.

Highlights

• Dragonfly's licence changes from BSL 1.1 to DSAL, starting with v2.1 (#8739).
• The RESP IoLoopV2 is now enabled by default (#8576).
• Search gains FT.ALIAS* commands, alias-aware queries and multiple attributes per field (#8556, #8649, #8604).

Commands

• Without --bind, Dragonfly now listens on :: on dual-stack hosts, serving IPv4 and IPv6 from one listener (#8682).
• Added CLIENT TRACKINGINFO (#8497).
• Added the lua_max_stall_ms flag, which kills Lua scripts that block their thread before writing (#8426).
• Added the --list_compress_policy flag. --list_compress_dict_threshold is deprecated (#8008).
• DEBUG OBJHIST now covers JSON, streams and sketches (#8656).
• Added the io_threads and max_connected_clients_per_thread Prometheus metrics (#8701).
• INFO expired_keys and evicted_keys are now cumulative across FLUSHDB/FLUSHALL and reset only by CONFIG RESETSTAT (#8562).

Performance

• During replica full sync, writes to buckets the snapshot has not yet reached skip forced early serialization. This now also works with a second replica and for EXPIRE-family commands (#8112).
• DFLYCLUSTER GETSLOTINFO collects stats per shard round, with bounded memory (#8715).
• The background sweep that flushes unowned slots after a migration is now throttled (#8641).
• Inline single-shard transactions are faster, with [about 5% fewer cycles and about 8% fewer instructions] (#8609).
• Added the opt-in --shard_sidekiq_queues flag so Sidekiq queues and their private queues share a shard (#8642).
• Single-shard BLMOVE/BRPOPLPUSH completes in one hop when data is available (#8638).
• Reduced the CPU overhead of cache-mode bumps (#8589).
• Added edge_depth to --list_compress_policy (#8334).
• Double formatting now uses dragonbox, [about 1.8–3.6x faster] (#8461).
• OAHTable now shrinks its bucket array automatically (#8454).
• V2 connections now flush replies right after a pipeline squash (#8431).
• V2 parse-loop lookups are cached per parsing segment (#8230).
• MGET and Memcache multi-get return large values zero-copy, and the borrow threshold is lowered to 8KB (#8420).
• CLUSTER SLOTS and CLUSTER SHARDS batch their replies, [from about 7000 socket writes to fewer than 100 for 1024 ranges] (#8390).
• Shard shutdown frees the whole heap in one operation (#8336).
• SCAN now packs returned keys into a single buffer (#8314).
• SCAN expiry checks and key extraction are faster (#8297).
• The journal is turned off automatically on shards where nothing uses it (#8259).
• Faster ASCII string decoding (#8295).
• Tiered storage's external allocator frees pages in O(1) and [halves descriptor RAM from about 128MB to 64MB per TB] (#8527).

Replication

• Added Prometheus metrics for snapshot and replica-disconnect state (#8740).
• S3 snapshot upload sockets are now paced to snapshot_egress_limit_bytes. Added --s3_upload_traffic_mark (#8719).

Search

• FT.CREATE now stores the INDEXMISSING, INDEXEMPTY and UNF flags. FT.INFO output is aligned with the reference (#8720).
• FT.ALTER and FT.DROPINDEX accept index aliases (#8683).
• Search commands resolve aliases created with FT.ALIASADD (#8649).
• Added FT.ALIASADD, FT.ALIASUPDATE and FT.ALIASDEL (#8556).
• Aliases are persisted in snapshots and sent to replicas on full sync (#8624).
• The schema is keyed by alias, so a HASH field can have several attributes (#8604).
• JSON indexes accept the same path under several aliases (#8651).
• Queries accept punctuation inside words, such as @email:*example.com* (#8419).
• FT.INFO percent_indexed reports real build progress (#8428).

Security

• CONFIG SET values and ACL SETUSER tokens are no longer exposed in logs, SLOWLOG or error replies (#8659).
• Fixed ACL category mismatches with Redis (#8308).

Metrics & Observability

• Prometheus dragonfly_rdb_serialize_total, dragonfly_save_in_progress and dragonfly_replica_disconnects_total{reason} are exported (#8740).
• Added V2 RESP IO loop and pipeline squash counters (#8557).
• pipeline_latency_seconds is split by a type label (read or other) (#8535).

AOF Persistence (Experimental)

• Added manifest-driven startup for --aof, with adoption of an existing snapshot as the base (#8502).