Question bank
CDP Data Engineer
41 CDP Data Engineer questions with the correct answer and a written explanation, organised across 5 exam domains.
- 41 questions
- 5 domains
- Foundation level
certilumQuestion index
Jump straight to any question. Answers and explanations are inline.
Data Ingestion
11 questions in this domain
- Question 1Single Choice
Which Apache tool is best suited for simple log aggregation from many servers directly to HDFS with minimal overhead?
- AApache NiFi
- BApache Kafka
- CApache FlumeCorrect answer
- DApache Spark Structured Streaming
Why this is the answer
Flume is designed for reliable, agent-based log collection. NiFi is overkill for simple log aggregation, Kafka requires separate producers, and Spark is for processing, not direct ingestion.
- Question 2Multiple Select
Which two file formats native to CDP provide support for schema evolution, allowing fields to be added or removed with defaults?
- AAvroCorrect answer
- BParquetCorrect answer
- CJSON
- DORC
Why this is the answer
Avro has native schema evolution with defaults and aliases. Parquet supports adding nullable columns. JSON is schemaless and does not enforce evolution; ORC has limited evolution similar to Parquet but Avro and Parquet are the primary ones discussed in the exam.
- Question 3Interactive
A NiFi processor has a relationship named 'failure' that is not connected to any downstream component. What is the most likely consequence?
- AThe FlowFile will be automatically terminated and lost.
- BThe processor will stop processing because it cannot route the FlowFile.Correct answer
- CThe processor will continue but log a warning.
- DThe connection will be automatically created to a dead letter queue.
Why this is the answer
If a processor has a relationship that is not connected, it will not be able to route FlowFiles to that relationship, causing the processor to stop or produce a 'stuck' state. Auto-termination is only if explicitly configured.
- Question 4Single Choice
Which URI scheme should be used when connecting to Azure Data Lake Storage Gen2 from a CDP cluster?
- Awasb://
- Babfss://Correct answer
- Cs3a://
- Dgs://
Why this is the answer
ABFS (abfss) is the modern connector for Azure Data Lake Storage Gen2. wasb is for older Azure Blob Storage (Gen1). s3a is for AWS S3, gs for GCS.
- Question 5Multiple Select
Which two approaches are commonly used to implement a dead letter queue (DLQ) during data ingestion?
- AWriting bad records to a separate Kafka topicCorrect answer
- BStoring failed records in a dedicated HDFS directoryCorrect answer
- CIgnoring invalid records silently
- DLogging errors to a file but continuing the pipeline
Why this is the answer
A DLQ can be a separate Kafka topic or a HDFS directory that stores records that fail validation. Ignoring or only logging does not preserve the records for later analysis.
- Question 6Interactive
In a Spark Structured Streaming job, the following code is used inside `foreachBatch`: badRecordsDF.write.mode(
- ADeduplication of records
- BError isolation and storageCorrect answer
- CSchema evolution handling
- DPerformance tuning through caching
Why this is the answer
Writing invalid records to a separate location (here /dlq/) is the classic pattern of a dead letter queue, isolating errors for later analysis without stopping the pipeline.
- Question 7Single Choice
Which Kafka producer property is essential to achieve exactly-once delivery semantics?
- Aacks=all
- Benable.idempotence=trueCorrect answer
- Cretries=Integer.MAX_VALUE
- Dcompression.type=snappy
Why this is the answer
enable.idempotence=true makes the producer idempotent, which is a prerequisite for exactly-once semantics. acks=all is needed but not sufficient alone; retries and compression are unrelated to idempotency.
- Question 8Multiple Select
Which two scheduling strategies are available for processors in Apache NiFi?
- ATIMER_DRIVENCorrect answer
- BCRON_DRIVEN
- CEVENT_DRIVENCorrect answer
- DPOLL_DRIVEN
Why this is the answer
NiFi processors support TIMER_DRIVEN (periodic), EVENT_DRIVEN (triggered by events), and CRON_DRIVEN. CRON_DRIVEN is also valid, but since the question asks for two, TIMER_DRIVEN and EVENT_DRIVEN are the most common. However, CRON_DRIVEN is also a valid scheduling strategy. To avoid ambiguity, we will select TIMER_DRIVEN and EVENT_DRIVEN as per the playbook's mention.
- Question 9Interactive
Given this Avro schema snippet, what compatibility rule does the 'default' field support? {\"name\": \"age\", \"type\": \"int\", \"default\": 0}
- ABackward compatibilityCorrect answer
- BForward compatibility
- CFull compatibility
- DNo compatibility
Why this is the answer
A field with a default value allows new readers to read old data (which lacks that field) by using the default. This is the definition of backward compatibility.
- Question 10Single Choice
Which Flume channel type ensures that events are persisted to disk and can survive a process restart?
- AMemory Channel
- BFile ChannelCorrect answer
- CJDBC Channel
- DSpillable Memory Channel
Why this is the answer
File Channel writes events to a write-ahead log on disk, providing durability. Memory Channel is volatile, JDBC is not standard, Spillable Memory Channel is a hybrid but less durable than File.
- Question 11Multiple Select
Two key advantages of using Parquet over JSON for analytical workloads are:
- ABetter compression ratiosCorrect answer
- BFull support for schema evolution
- CColumnar storage enabling predicate pushdownCorrect answer
- DHuman-readable format
Why this is the answer
Parquet uses columnar storage which improves compression and enables predicate pushdown for faster queries. Schema evolution is limited compared to Avro, and human-readability is not an advantage.
Data Pipeline Orchestration
6 questions in this domain
- Question 12Single Choice
What is the primary definition language for Apache Oozie workflows?
- APython
- BXMLCorrect answer
- CYAML
- DJSON
Why this is the answer
Oozie uses XML with Oozie Expression Language (EL) to define workflows and actions.
- Question 13Multiple Select
Which of the following are valid scheduling mechanisms in Apache Oozie? (Select two.)
- ACron expressions
- BCoordinator jobs with frequencyCorrect answer
- CPython timedelta
- DBundle jobsCorrect answer
Why this is the answer
Oozie uses coordinator jobs (time-based) and bundle jobs (dataset-based) for scheduling. Cron is used by Airflow.
- Question 14Interactive
In the provided Oozie workflow XML, what type of node is 'mr_action'?
- AStart node
- BAction nodeCorrect answer
- CKill node
- DEnd node
Why this is the answer
The <action> element defines an action node that performs a MapReduce job or other operation.
- Question 15Single Choice
What is the default number of retries for an Oozie action?
- A3
- B0Correct answer
- C1
- D5
Why this is the answer
Oozie actions have a default retry-max of 0, meaning no automatic retries are performed.
- Question 16Multiple Select
Which two components are part of YARN resource management in CDP Private Cloud Base? (Select two.)
- AResourceManagerCorrect answer
- BNameNode
- CNodeManagerCorrect answer
- DApplicationMaster
Why this is the answer
YARN ResourceManager (RM) and NodeManager (NM) are the primary components; NameNode belongs to HDFS.
- Question 17Interactive
In the Airflow DAG snippet, what does the 'schedule_interval' parameter define?
- AThe time interval between DAG runsCorrect answer
- BThe start date of the DAG
- CThe maximum duration of a DAG run
- DThe number of retries for tasks
Why this is the answer
schedule_interval sets how frequently the DAG should be executed (e.g., '@daily' means once per day).
Data Security and Governance
4 questions in this domain
- Question 18Single Choice
What is the primary purpose of Apache Ranger in CDP?
- AData cataloging and lineage tracking
- BFine-grained access control and authorizationCorrect answer
- CData encryption at rest
- DCluster monitoring and metrics
Why this is the answer
Apache Ranger provides centralized authorization, auditing, and data masking for Hadoop ecosystem components.
- Question 19Multiple Select
Which two components are required to secure data access in CDP? (Select two.)
- AKerberosCorrect answer
- BTLSCorrect answer
- CData masking
- DLoad balancing
Why this is the answer
Kerberos authenticates users and services; TLS encrypts data in transit.
- Question 20Interactive
What does the following Atlas REST API call represent?
- ACreating a new data entity
- BAdding a classification to an entityCorrect answer
- CSearching for entities by classification
- DDeleting a classification from an entity
Why this is the answer
The API call adds the 'PII' classification to the specified Hive table entity.
- Question 21Single Choice
In Cloudera Manager, where are audit logs stored by default?
- ACloudera Manager's embedded PostgreSQL database
- BApache SolrCorrect answer
- CApache HDFS only
- DThe local file system of the management host
Why this is the answer
Cloudera Manager uses Apache Solr to index and store audit logs for searchability.
Data Storage and Management
8 questions in this domain
- Question 22Single Choice
What is the primary purpose of Hive partitioning?
- ATo organize data into subdirectories based on column values, enabling partition pruningCorrect answer
- BTo compress data at the partition level for storage efficiency
- CTo distribute data evenly across cluster nodes using hash keys
- DTo encrypt sensitive data within each partition
Why this is the answer
Hive partitioning divides data into subdirectories based on column values, allowing queries to skip irrelevant partitions via partition pruning.
- Question 23Multiple Select
Which of the following are true about Apache Parquet and Apache ORC? (Select two.)
- AParquet is natively supported by Impala; ORC is not.Correct answer
- BORC supports Hive ACID transactions; Parquet does not.Correct answer
- CBoth formats support nested types equally well.
- DORC is the default compression codec used by Parquet.
Why this is the answer
Impala supports Parquet natively but not ORC. ORC enables Hive ACID; Parquet lacks ACID support.
- Question 24Interactive
Consider the Hive DDL statement. What does the code accomplish?
- AIt sets a retention policy of 30 days on the table.
- BIt configures a lifecycle transition rule.
- CIt defines a retention policy at the table level.Correct answer
- DIt disables deletion for the table.
Why this is the answer
The ALTER TABLE statement with TBLPROPERTIES sets a retention policy that defines how long data must be kept before deletion.
- Question 25Single Choice
Which cloud object store provides strong consistency suitable for Hive ACID workloads?
- AAmazon S3
- BAzure Data Lake Storage Gen2Correct answer
- CGoogle Cloud Storage (for new objects only)
- DAll cloud object stores provide eventual consistency
Why this is the answer
Azure ADLS Gen2 offers strong consistency natively, unlike S3's eventual consistency, making it suitable for Hive ACID transactions.
- Question 26Multiple Select
Which two statements correctly describe Apache Ranger and Apache Atlas?
- ARanger enforces access control policies; Atlas tracks data lineage.Correct answer
- BRanger stores classification tags; Atlas defines security policies.
- CAtlas captures lineage through Hive hooks; Ranger audits are sent to Atlas.Correct answer
- DBoth tools are used for data replication between clusters.
Why this is the answer
Ranger provides centralized security policy enforcement; Atlas manages metadata and lineage. Audits from Ranger can be logged in Atlas.
- Question 27Interactive
Analyze the Hive DDL snippet. What does the bucketing clause do?
- AIt divides data into buckets based on a hash of customer_id.Correct answer
- BIt creates partitions on the customer_id column.
- CIt specifies the storage format for the table.
- DIt sets the compression codec for each bucket.
Why this is the answer
The CLUSTERED BY clause uses a hash of customer_id to assign rows to a fixed number of buckets, enabling efficient sampling and join optimization.
- Question 28Single Choice
Which compression codec is fastest in terms of speed but offers a moderate compression ratio?
- ASnappyCorrect answer
- BZstd
- CGzip
- DLZ4
Why this is the answer
Snappy is designed for fast compression/decompression with moderate ratios, while Zstd offers better ratios at similar speeds.
- Question 29Multiple Select
Which two statements about Replication Manager and DistCp are correct?
- AReplication Manager can handle Hive ACID tables; DistCp cannot.Correct answer
- BBoth tools automatically replicate Ranger policies.
- CDistCp performs bulk file copies via MapReduce jobs.Correct answer
- DReplication Manager is only for cloud-to-cloud replication.
Why this is the answer
Replication Manager supports ACID tables and metadata sync; DistCp is a lower-level bulk copy tool that lacks ACID handling.
Data Transformation
12 questions in this domain
- Question 30Single Choice
In Apache Spark, when does a transformation like `filter()` or `select()` actually execute?
- AImmediately when the function is called.
- BOnly when an action like `count()` or `show()` is invoked.Correct answer
- CDuring the creation of the SparkSession.
- DAfter every transformation, the data is materialized on disk.
Why this is the answer
Spark transformations are lazy; they build a DAG of operations but do not execute until an action triggers computation.
- Question 31Multiple Select
Which of the following are characteristics of narrow transformations in Spark? (Select two.)
- AThey cause data to be shuffled across partitions.
- BEach partition depends on only one input partition.Correct answer
- CThey are always faster than wide transformations.
- DExamples include `filter()` and `map()`.Correct answer
Why this is the answer
Narrow transformations do not require shuffle; each output partition depends on at most one input partition. Examples: `filter`, `map`, `select`.
- Question 32Interactive
Examine the following Spark code snippet. Determine whether the last line is a transformation or an action.
- ATransformation
- BActionCorrect answer
- CBoth
- DNeither
Why this is the answer
The `show()` method is an action because it triggers execution and returns output to the driver.
- Question 33Single Choice
What is the primary role of the Catalyst Optimizer in Spark SQL?
- AIt manages memory and disk storage for DataFrames.
- BIt applies rule-based and cost-based optimizations to query plans.Correct answer
- CIt serializes data between the JVM and Python.
- DIt handles user authentication and authorization.
Why this is the answer
Catalyst is Spark's query optimizer that applies rule-based and cost-based optimizations (e.g., predicate pushdown, constant folding) to improve execution plans.
- Question 34Multiple Select
Which of the following are considered built-in Spark SQL functions that should be preferred over UDFs for better performance? (Select two.)
- Asubstring()Correct answer
- Budf()
- Cwhen() and otherwise()Correct answer
- Dpandas_udf()
Why this is the answer
`substring()` and `when()/otherwise()` are built-in functions that Catalyst can optimize. UDFs and Pandas UDFs add overhead and may block optimizations.
- Question 35Interactive
Given the Spark transformation below, identify whether it is a narrow or wide transformation.
- ANarrowCorrect answer
- BWide
- CIt depends on the data size
- DCannot be determined
Why this is the answer
`filter()` is a narrow transformation because it operates independently on each partition without shuffling data across nodes.
- Question 36Single Choice
Which method should you use after a filter operation to reduce the number of partitions without causing a full shuffle?
- Arepartition()
- Bcoalesce()Correct answer
- CpartitionBy()
- Dcache()
Why this is the answer
`coalesce()` reduces the number of partitions without a full shuffle, making it efficient after a filter reduces data size. `repartition()` triggers a full shuffle.
- Question 37Multiple Select
Which storage levels can be used with Spark's `persist()` method? (Select two.)
- AMEMORY_ONLYCorrect answer
- BDISK_ONLY
- CMEMORY_AND_DISK_SERCorrect answer
- DREGISTER_ONLY
Why this is the answer
Spark provides several storage levels including `MEMORY_ONLY`, `MEMORY_AND_DISK_SER`, `MEMORY_ONLY_SER`, etc. `DISK_ONLY` is not a standard level (though `MEMORY_AND_DISK` exists).
- Question 38Interactive
Complete the following Spark SQL query by filling in the missing hint to force a broadcast join when the dimension table is small.
- ABROADCASTCorrect answer
- BMERGE
- CSHUFFLE_HASH
- DREPLICATE_NL
Why this is the answer
The `BROADCAST` hint (or `/*+ BROADCAST(dim) */`) tells Spark to use a broadcast hash join, which is efficient when one table is small enough to fit in executor memory.
- Question 39Single Choice
Which engine in Cloudera Data Platform runs dedicated daemons and does not use YARN for resource management?
- AHive LLAP
- BImpalaCorrect answer
- CSpark SQL
- DTez
Why this is the answer
Impala uses its own admission control and static resource pools; it does not run inside YARN containers. Hive LLAP uses YARN for its daemons.
- Question 40Multiple Select
Which of the following are features of Apache Impala as described in the CDP-3002 exam guide? (Select two.)
- AIt supports full ACID transactions (INSERT, UPDATE, DELETE).
- BIt uses a distributed MPP execution model with in-memory processing.Correct answer
- CIt can read but not write to Hive ACID tables.Correct answer
- DIt relies on YARN for dynamic resource allocation.
Why this is the answer
Impala is an MPP engine that executes queries in-memory. It can read ACID tables but cannot perform transactional writes. It does not use YARN.
- Question 41Interactive
Complete the following window function statement to assign a unique rank to each employee within their department, ordered by salary descending.
- AROW_NUMBER()Correct answer
- BRANK()
- CDENSE_RANK()
- DLAG()
Why this is the answer
`ROW_NUMBER()` assigns a unique sequential integer to each row within a partition, starting at 1, without skipping numbers even if there are ties (the order among ties is non-deterministic).
Frequently asked questions
Everything about the CDP Data Engineer question bank and how to use it.