Practical Course: Cloud-Based Data Processing
Content
This lab module provides hands-on experience with distributed data processing, cloud computing, and distributed databases. It gives students practical experience in implementing concepts covered in the Cloud-Based Data Processing lecture. Attending the lecture is recommended but not required for completing the Praktikum.
We will implement and evaluate data-intensive systems on cloud infrastructure: distributed query execution over partitioned datasets, stateful aggregation and shared state, fault-tolerant key-value stores using consensus (Raft), and cloud-native pipelines using serverless functions and caching. Using cloud services (containers, VMs, Functions, storage), we design, implement, deploy, and benchmark realistic cloud setups, and document their design and performance.
Organization
Preliminary meeting: 10.02.2026, 16:00 (online on BBB)
Grade will be based on take-home coding assignments and an end-term project.
Previous Knowledge Expected
The course is aimed at Master-level students who have experience in programming in a systems-level programming language (Rust, C/C++).
Knowledge from the Cloud-Based Data Processing lecture would be helpful, but is not required. It is recommended to take this Praktikum alongside the lecture.
Additionally, knowledge from some of the following (or similar) courses is expected:
- Introduction to Databases (IN0008)
- Basic Principles: Operating Systems and Systems Software (IN0009)
- Introduction to Computer Networks and Distributed Systems (IN0010)
Objective
After the completion of this course students should:
- deploy and operate containerized services in a cloud environment, including using object storage and basic monitoring.
- design and implement distributed, data-intensive applications using coordinator/worker patterns and shared state.
- apply replication and consensus algorithms (e.g. Raft) to build fault-tolerant, strongly consistent services.;
- construct event-driven/serverless pipelines that respond to data and integrate with existing components.
- be able to reliably measure performance, analyse experiment results, and make scientific conclusions;
- know how to debug code for performance, use tools like perf, analyse traces and flamegraphs;
- use caching, partitioning, and scheduling strategies to improve performance and scalability while understanding their trade-offs.
- be able to understand and discuss cloud papers.
Recommended Reading
All of the resources from the Cloud-Based Data Processing lecture will be useful.
- "Designing Data-Intensive Applications" by Martin Klepmann;
- "Distributed Systems" (3rd edition) by Andrew S. Tanenbaum & Maarten Van Steen