μWheel: Aggregate Management for Streams and Queries
2024 (English)In: DEBS 2024 - Proceedings of the 18th ACM International Conference on Distributed and Event-Based Systems, Association for Computing Machinery, Inc , 2024, p. 54-65Conference paper, Published paper (Refereed)
Abstract [en]
Aggregate management is equally significant for both streaming and query workloads. However, the prevalent approach of separating stream processing and query analysis impairs performance, hinders aggregate reuse, increases resource demands, and lowers data freshness. μWheel addresses this problem by unifying aggregate management needs within a single system optimized for continuous event streams. μWheel pre-aggregates and indexes timestamped data arriving out-of-order, enabling the sharing of aggregates across arbitrary time intervals while respecting low watermarks. Our performance analysis demonstrates that μWheel dramatically outperforms current aggregate sharing techniques for high-volume streaming, particularly when handling numerous concurrent window slides. Crucially, μWheel also delivers performance comparable to specialized pre-aggregation indexes for supporting ad-hoc queries and does so with significantly reduced storage requirements. μWheel’s efficiency stems from its compact wheel-based data layout, featuring implicit timestamps, a query-agnostic time hierarchy, and a query optimizer designed to minimize aggregate operations.
Place, publisher, year, edition, pages
Association for Computing Machinery, Inc , 2024. p. 54-65
Keywords [en]
Digital storage; Aggregate management; Embedded analytic; Event streams; Management IS; Performance; Query analysis; Resource demands; Reuse; Stream processing; Time-stamped data; Aggregates
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:ri:diva-74814DOI: 10.1145/3629104.3666031Scopus ID: 2-s2.0-85200696289OAI: oai:DiVA.org:ri-74814DiVA, id: diva2:1892506
Conference
18th ACM International Conference on Distributed and Event-Based Systems, DEBS 2024Villeurbanne, France. 25 June 2024 through 28 June 2024
Note
This work has been supported by Vinnova (Grant No.: 2022-03036),the Swedish Foundation of Strategic Research (Grant No.: BD15-0006), and Wallenberg AI NEST (DataBound Computing).
2024-08-272024-08-272025-09-23Bibliographically approved