With BI-Processes, a wide range of tasks from ETL to analytics is described and executed in modern Data-Warehouse and Business Intelligence Systems. Nowadays, lots of these processes have become so big and complex that they require high-performance hardware, large distributed data sources and multiple processing steps. Although computing systems have become more powerful, they also have become more prone to failures due to physical effects (i.e. bit flips in main memory) or the sheer number of involved subsystems. While errors that lead to a crash of the whole process are a problem, they are easily noticeable and can be naively countered by restarting the whole process. A much more challenging problem is errors that occur unnoticed. These will not crash the complete process but will falsify its intermediate results etc. This means, the BI-process will terminate normally but will eventually provide incorrect answers that can have unpredictable negative consequences for the organization that runs them. The goal of this topic is to make BI-processes more resilient, by developing methods and practices that allow the identification of such errors. To realize this resilience even for large and complex BI-processes, the main idea of this topic is to create “shadow-processes” i.e. miniaturized and compact versions of the original process that act as a checksum and can be used for constant and effective error-detection.

Main Advisor at Technische Universität Dresden (TUD)
Co‐advisor at Poznan University of Technology (PUT)