Arcon: Continuous and deep data stream analyticsShow others and affiliations
2019 (English)In: ACM International Conference Proceeding Series, Association for Computing Machinery , 2019Conference paper, Published paper (Refereed)
Abstract [en]
Contemporary end-to-end data pipelines need to combine many diverse workloads such as machine learning, relational operations, stream dataflows, tensor transformations, and graphs. For each of these workload types, there exists several frontends (e.g., SQL, Beam, Keras) based on different programming languages as well as different runtimes (e.g., Spark, Flink, Tensorflow) that optimize for a particular frontend and possibly a hardware architecture (e.g., GPUs). The resulting pipelines suffer in terms of complexity and performance due to excessive type conversions, materialization of intermediate results, and lack of cross-framework optimizations. Arcon aims to provide a unified approach to declare and execute tasks across frontend-boundaries as well as enabling their seamless integration with event-driven services at scale. In this demonstration, we present Arcon and through a series of use-case scenarios demonstrate that its execution model is powerful enough to cover existing as well as upcoming real-time computations for analytics and application-specific needs. © 2019 Copyright held by the owner/author(s).
Place, publisher, year, edition, pages
Association for Computing Machinery , 2019.
Keywords [en]
Data flow analysis, Information analysis, Object oriented programming, Program processors, Application specific, Framework optimization, Hardware architecture, Intermediate results, Real-time computations, Relational operations, Seamless integration, Tensor transformation, Pipelines
National Category
Natural Sciences
Identifiers
URN: urn:nbn:se:ri:diva-40454DOI: 10.1145/3350489.3350492Scopus ID: 2-s2.0-85072806432ISBN: 9781450376600 (print)OAI: oai:DiVA.org:ri-40454DiVA, id: diva2:1361272
Conference
13th International Workshop on Real-Time Business Intelligence and Analytics, BIRTE 2019, in conjunction with the VLDB 2019 Conference, 26 August 2019
2019-10-152019-10-152023-06-07Bibliographically approved