AQPrius: Offline Approximate Query Processing Enhanced by Error Assessment using Bootstrap Sampling
AQPrius is an offline approximate query processing (AQP) engine that can efficiently answer complex analytic queries on large datasets. Unlike existing systems that employ the online AQP schemes, AQPrius employs the offline AQP scheme which has two advantages: (1) it doesn’t require high-end hardware or expensive auxiliary data structures such as indices or hash tables; (2) the synopses collected are reusable for future queries on the same database which can significantly save computing resources. However, the error assessment for offline AQP systems is still a challenging problem. The contributions of this research are four-fold. First, AQPrius is an offline AQP engine that can quickly answer common analytic queries including selection conditions, join conditions, and aggregate functions. It can speed up complex query processing on big data. Second, AQPrius enables error assessment using a non-parametric statistic method, namely bootstrap sampling, that can provide the standard error of query estimation. Third, using the standard error by bootstrap sampling, we extend the traditional offline AQP system from providing a single-point query estimation to a range estimation which is a bounded answer presented as a confidence interval (CI). Finally, the system is developed using the Rust programming language which can prevent many security issues and potential vulnerabilities. We evaluate AQPrius using the well-known TPC-H benchmarks. The experimental results show that AQPrius can rapidly generate accurate bounded query answers for various test queries with selection and join conditions.
System Architecture
System Runtime Demonstration
Publication:
- AQPrius: Offline Approximate Query Processing Enhanced by Error Assessment using Bootstrap Sampling
A. Yu, S. Maharjan, L. Kerns, X.-J. Min, A. Arslanyilmaz, and M. Zhu
International Journal of Computer Science and Security, Vol. 18, Issue 3, pages 30-47, Oct 2024. paper
Source code: