Detecting trading trends in streaming financial data using Apache Flink

Emmanouil Kritharakis, Shengyao Luo, Vivek Unnikrishnan, Karan Vombatkere
2022 Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems  
Modern financial analytics rely on high-volume streams of event notifications that report live market fluctuations based on supply and demand. Accurately identifying trends or breakout patterns based on the Exponential Moving Average (EMA) in the development of an instrument's price early on is an important challenge, so as to buy while the price is low and sell before a downtrend begins. This paper aims to solve the above challenge with a distributed, event-streaming solution built using
more » ... Flink. We present and implement a solution that leverages customized window operators to calculate the EMA and find breakout patterns, using event generation parallelism to facilitate the rapid processing of the input stream uses sinks to collect and output results, and scales easily on a distributed Flink cluster. We empirically test our design on metrics specified by the benchmarking platform for the DEBS 2022 Grand Challenge and observe a throughput of 45 batches per second and an average latency of 120 ms. CCS CONCEPTS • Information systems → Data stream mining; • Computing methodologies → Distributed computing methodologies.
doi:10.1145/3524860.3539647 fatcat:5nfmj2o4g5e5hhgoebrwdxi7pu