Building an Open Data Lakehouse Using Apache Iceberg

Course Information

Course Name

DENG-251: Building an Open Data Lakehouse Using Apache Iceberg

Exam code

CDP-3002

Duration

4 Days

Certification

Cloudera Data Engineer

Overview

The Open Data Lakehouse is a modern data architecture that enables versatile analytics on streaming and stored data within cloud-native object stores. This architecture can span hybrid and multi-cloud environments.

This course introduces Apache Ozone, a hybrid storage service addressing the limitations of HDFS. You’ll also explore Apache Iceberg, an open-table format optimized for petabyte-scale datasets. The course covers Iceberg’s benefits, architecture, read/write operations, streaming, and advanced features like time travel, partition evolution, and Data-as-Code. Over 25 hands-on labs and a capstone project will equip you with the skills to build an efficient, performant Open Data Lakehouse within your own environment.

Audience Profile

This course is designed for data professionals within organizations using Cloudera Data Warehouse or Cloudera Data Engineering solutions. If you’re building an Open Data Lakehouse powered by Apache Iceberg, this course will provide the knowledge and skills you need. Ideal roles include Data Engineers, Hive/Impala SQL Developers, Kafka Streaming Engineers, Data Scientists, and CDP Admins.

Prerequisites

A basic understanding of HDFS and experience with Hive and Spark are prerequisites.

At Course Completion

Course Outline

Module 1: Open Data Lakehouse Fundamentals

• Understand core Open Data Lakehouse concepts and benefits.

• Introduction to Apache Ozone and its integration within the CDP Ecosystem.

Module 2: Apache Ozone Mastery

• Configure Ozone, use CLI commands, and transfer data between HDFS and Ozone.

• Integrate Ozone into applications.

Module 3: Apache Iceberg Expertise

• Explore Iceberg’s integration with CDP, architecture, and data lakehouse design principles.

• Master data management, governance, and optimization best practices.

• Understand snapshots and time travel queries.

• Design tables strategically (external/managed, copy-on- write, merge-on-read).

• Employ advanced features: change data capture (CDC), schema/partition evolution, hidden partitions.

Module 4: Data-as-Code and Compliance

• Implement zero-copy cloning, table branching, and tagging for QA, ML models, and auditing.

• Optimize ETL/ELT data loading and achieve GDPR compliance with Iceberg’s write-audit-publish (WAP).

Module 5: Hive to Iceberg Migration

• Understand catalog differences and migration strategies.

• Manage late-arriving data effectively.

Module 6: Iceberg Administration

• Perform table maintenance tasks.

• Configure and manage access control settings.

Module 7: Capstone Project

• Apply all concepts by implementing an Open Data Lakehouse use case in CDP.

• Develop a comprehensive Open Data Lakehouse implementation runbook.

 All Cloudera certification courses are conducted by certified trainers from Iverson.

Digital Methods acts as the official training partner and assists with program consultation, registration, coordination, scheduling, and administrative arrangements to ensure a seamless and well-managed training experience.