Question bank
Databricks Certified Data Engineer Professional
47 Databricks Certified Data Engineer Professional questions with the correct answer and a written explanation, organised across 10 exam domains.
- 47 questions
- 10 domains
- Foundation level
certilumQuestion index
Jump straight to any question. Answers and explanations are inline.
Cost & Performance Optimisation
6 questions in this domain
- Question 1Single Choice
What is the default value of spark.sql.shuffle.partitions in Spark?
- A100
- B200Correct answer
- C400
- D1000
Why this is the answer
The default is 200. This controls the number of partitions after shuffle operations.
- Question 2Multiple Select
Which of the following are part of Spark executor sizing?
- Aspark.executor.coresCorrect answer
- Bspark.executor.memoryCorrect answer
- Cspark.executor.instancesCorrect answer
- Dspark.sql.shuffle.partitions
Why this is the answer
Executor sizing includes cores, memory, and instance count. Shuffle partitions are a separate tuning area.
- Question 3Interactive
Given the following configuration, what is the most likely performance issue?
- AToo few partitions cause memory spillCorrect answer
- BToo many partitions cause scheduling overhead
- CNot enough memory per executor
- DData skew due to key distribution
Why this is the answer
Setting partitions to 10 on a 100-core cluster creates large partitions that may exceed memory and spill to disk.
- Question 4Single Choice
What is the primary purpose of the Photon engine in Databricks?
- AQuery execution via native C++ vectorizationCorrect answer
- BStorage layout optimization using Z-ordering
- CData compression during write operations
- DStreaming job reliability with checkpointing
Why this is the answer
Photon is a native C++ vectorized query engine that speeds up SQL and DataFrame operations.
- Question 5Multiple Select
Which of the following operations does Photon accelerate?
- AAggregationsCorrect answer
- BJoinsCorrect answer
- CPython UDFs
- DWindow functionsCorrect answer
Why this is the answer
Photon excels at CPU-bound operations like aggregations, joins, and windows. Python UDFs cause fallback.
- Question 6Interactive
This table creation uses which Delta Lake feature?
- APartitioning
- BBucketing
- CLiquid ClusteringCorrect answer
- DZ-ordering
Why this is the answer
Liquid Clustering uses CLUSTER BY to dynamically reorganize data for multiple columns.
Data Governance
3 questions in this domain
- Question 7Single Choice
How many Unity Catalog metastores can be assigned to a single Databricks workspace?
- AOneCorrect answer
- BTwo
- CAs many as needed
- DNone
Why this is the answer
Only one metastore can be assigned per workspace; however, one metastore can serve multiple workspaces.
- Question 8Multiple Select
Which two of the following are securable objects in Unity Catalog?
- AMetastoreCorrect answer
- BCluster
- CSchemaCorrect answer
- DNotebook
Why this is the answer
Metastore and Schema are securable objects in the Unity Catalog hierarchy; Clusters and Notebooks are not.
- Question 9Interactive
A data engineer runs the following SQL command. Which object is being granted privileges?
- ACatalog
- BSchema
- CTableCorrect answer
- DMetastore
Why this is the answer
The GRANT command specifies 'ON TABLE sales.customers', so the table 'sales.customers' is the object receiving SELECT.
Data Ingestion & Acquisition
3 questions in this domain
- Question 10Single Choice
A data pipeline ingests 50,000 small JSON files per minute from an S3 bucket. Which Auto Loader mode should be used to minimize latency?
- ADirectory listing mode with a low polling interval of 1 minute
- BFile notification mode using S3 bucket notifications and SQSCorrect answer
- CDirectory listing mode with the default polling interval of 10 minutes
- DFile notification mode but without setting up a schema location
Why this is the answer
File notification mode uses cloud-native event services (e.g., SQS, Event Grid) to detect new files with low latency (<1 minute), making it ideal for high-throughput scenarios. Directory listing mode has higher latency and is not suitable for thousands of files per minute.
- Question 11Multiple Select
Which schema evolution modes are available for Auto Loader when ingesting files with varying structures?
- ArescueCorrect answer
- BrescueAndEvolveCorrect answer
- CfailOnNewColumns
- DmergeSchema
Why this is the answer
Auto Loader supports three evolution modes: 'none' (fail on mismatch), 'rescue' (write mismatched data to _rescued_data), and 'rescueAndEvolve' (add new columns and rescue mismatches). 'failOnNewColumns' and 'mergeSchema' are not valid values for cloudFiles.schemaEvolutionMode; 'mergeSchema' is a Delta Lake write option.
- Question 12Interactive
Consider the following Auto Loader streaming code. The pipeline is failing due to schema mismatch errors. Which single change should be made to resolve the issue?
- AAdd the option .option("cloudFiles.schemaLocation", "s3://bucket/schemaCache")Correct answer
- BChange the cloudFiles.format option from "json" to "parquet"
- CSet the maxFilesPerTrigger option to 1000
- DUse spark.read instead of spark.readStream
Why this is the answer
Without a schema location, Auto Loader re-infers schema every micro-batch, causing failures when file schemas differ. Setting cloudFiles.schemaLocation caches the schema and enables evolution control. The format option is already correct; maxFilesPerTrigger controls throughput, not schema; switching to batch read loses incremental capabilities.
Data Modelling
3 questions in this domain
- Question 13Single Choice
Which schema is recommended as the best practice for the gold layer of the Medallion architecture on Databricks?
- AStar schemaCorrect answer
- BSnowflake schema
- CThird normal form (3NF)
- DFlat table (no dimensions)
Why this is the answer
The document states that star schemas are the gold standard for the gold layer, optimized for BI and AI workloads.
- Question 14Multiple Select
Which of the following are types of Slowly Changing Dimensions (SCD) described in the document?
- AType 1 (Overwrite)Correct answer
- BType 2 (Add new row)Correct answer
- CType 3 (Add new column)Correct answer
- DType 4 (History table only)
Why this is the answer
The document elaborates on SCD Types 1, 2, and 3. Type 4 is not mentioned.
- Question 15Interactive
Review the Delta Lake MERGE statement below. Which SCD type does it implement? (The code uses WHEN MATCHED THEN UPDATE SET * and WHEN NOT MATCHED THEN INSERT *)
- ASCD Type 1 (Overwrite)Correct answer
- BSCD Type 2 (Add new row)
- CSCD Type 3 (Add new column)
- DSCD Type 0 (No change)
Why this is the answer
The MERGE with WHEN MATCHED THEN UPDATE overwrites existing rows, matching the Type 1 definition (no history kept).
Data Sharing and Federation
3 questions in this domain
- Question 16Single Choice
What is the name of the open protocol developed by Databricks specifically designed for secure, governed, and real-time data sharing across heterogeneous platforms?
- ADelta SharingCorrect answer
- BApache Spark
- CUnity Catalog
- DLakehouse Federation
Why this is the answer
Delta Sharing is an open protocol developed by Databricks for secure, governed, and real-time data sharing across heterogeneous platforms without copying data.
- Question 17Multiple Select
Consider the following statements regarding Delta Sharing shares in Databricks and choose the ones that are accurate. (Select two options.)
- AA share is a logical container for tables, views, or volumes.Correct answer
- BA share can include tables from multiple catalogs within the same metastore.
- CA share is created using the CREATE SHARE command.Correct answer
- DA share can be renamed after creation using ALTER SHARE RENAME.
Why this is the answer
Shares are created with CREATE SHARE, can contain tables/views/volumes from a single metastore, and are immutable – they cannot be renamed.
- Question 18Interactive
After creating a recipient using CREATE RECIPIENT, which SQL command must be executed to allow the recipient to read a share? Refer to the code block.
- AGRANT SELECT ON SHARE sales_share TO RECIPIENT partner_rec;Correct answer
- BGRANT USAGE ON CATALOG sales_catalog TO RECIPIENT partner_rec;
- CGRANT INSERT ON SHARE sales_share TO RECIPIENT partner_rec;
- DGRANT CREATE ON SHARE sales_share TO RECIPIENT partner_rec;
Why this is the answer
After creating a recipient, you must grant SELECT on the share to allow read-only access. Other privileges are either insufficient or irrelevant for read access.
Data Transformation, Cleansing, and Quality
5 questions in this domain
- Question 19Single Choice
What does DLT expect_all_or_drop do when a row violates a constraint?
- AThe violating row is kept but flagged in the event log.
- BThe violating row is dropped from the output table.Correct answer
- CThe entire batch fails and is rolled back.
- DThe pipeline stops and an alert is sent.
Why this is the answer
expect_all_or_drop removes rows that fail any constraint from the target table while logging the violation in the event log.
- Question 20Multiple Select
Which two statements are correct about Great Expectations integration in Databricks?
- AGreat Expectations can validate Spark DataFrames using Pandas conversion.Correct answer
- BDLT built-in expectations are identical to Great Expectations in functionality.
- CThe mostly parameter allows a percentage of failures per expectation.Correct answer
- DGreat Expectations cannot be used with Databricks notebooks.
Why this is the answer
Great Expectations supports Pandas validation and the mostly parameter sets a threshold for acceptable failures.
- Question 21Interactive
What is the result of the PySpark null handling operation?
- AAll nulls in 'name' are replaced with 'Unknown', others unchanged.Correct answer
- BAll nulls in all columns are replaced with 'Unknown'.
- CThe code throws an error because 'name' column has no nulls.
- DOnly the first null is replaced; subsequent nulls remain.
Why this is the answer
df.na.fill(value, subset) replaces only nulls in specified columns, so only nulls in 'name' are replaced.
- Question 22Single Choice
Which of the following Spark operations is a narrow transformation?
- AgroupBy()
- Bfilter()Correct answer
- Cjoin()
- Drepartition()
Why this is the answer
Filter is a narrow transformation because it operates per partition without redistributing data across the cluster.
- Question 23Multiple Select
Which two techniques are examples of irreversible data anonymization?
- ADynamic column masking
- BHashing with saltCorrect answer
- CGeneralization like age rangesCorrect answer
- DRow filters on Unity Catalog tables
Why this is the answer
Hashing with salt and generalization irreversibly transform data, satisfying anonymization requirements.
Debugging and Deploying
5 questions in this domain
- Question 24Single Choice
Which Spark UI tab is primarily used to view task-level metrics like duration and shuffle spill?
- AJobs
- BStagesCorrect answer
- CStorage
- DEnvironment
Why this is the answer
The Stages tab shows task-level metrics including duration, shuffle read/write, and spill, making it the primary debugging tab.
- Question 25Multiple Select
Which of the following are symptoms of data skew in a Spark job? (Select all that apply.)
- AA few tasks take significantly longer than the median.Correct answer
- BAll tasks have similar execution times.
- CHigh shuffle spill (memory and disk) for certain tasks.Correct answer
- DThe Storage tab shows cached data evenly distributed.
Why this is the answer
Data skew manifests as a long tail of task durations and high spill metrics due to imbalanced partitions.
- Question 26Interactive
Examine the following code snippet used for unit testing a Spark transformation. What does this test verify?
- AThat the transformation renames columns correctly.
- BThat null names are replaced with 'unknown' and the id column is preserved.Correct answer
- CThat the DataFrame schema contains exactly two columns.
- DThat the transformation filters out rows with null names.
Why this is the answer
The test inputs a DataFrame with a null name and expects the output to have 'unknown' and the same id, verifying the cleaning logic.
- Question 27Single Choice
Which Databricks feature provides Git integration for version-controlling notebooks and code?
- ADatabricks CLI
- BDatabricks ReposCorrect answer
- CDatabricks Workflows
- DDBFS
Why this is the answer
Databricks Repos provides Git integration, allowing users to link remote repositories and manage branches directly in the workspace.
- Question 28Multiple Select
Which of the following are valid methods to isolate environments (dev, staging, prod) in Databricks? (Select all that apply.)
- AUsing separate workspaces for each environment.Correct answer
- BStoring all notebooks in a single folder and relying on user permissions.
- CUsing Unity Catalog catalogs (e.g., dev_catalog, prod_catalog) within a single workspace.Correct answer
- DHardcoding environment-specific database names in notebooks.
Why this is the answer
Separate workspaces and Unity Catalog catalogs provide proper isolation; folder permissions alone are insufficient for security and data isolation.
Developing Code for Data Processing using Python and
9 questions in this domain
- Question 29Single Choice
What is the primary abstraction for distributed data processing in Apache Spark?
- ARDDCorrect answer
- BDataFrame
- CDataset
- DSparkContext
Why this is the answer
RDD stands for Resilient Distributed Dataset and is the fundamental data structure of Apache Spark, representing an immutable, partitioned collection of records that can be processed in parallel.
- Question 30Multiple Select
Which of the following are characteristics of a Spark DataFrame? (Select all that apply.)
- AIt is distributed across a clusterCorrect answer
- BIt has row and column structureCorrect answer
- CIt is stored on a single machine
- DIt supports lazy evaluationCorrect answer
Why this is the answer
DataFrames in Spark are distributed collections of data organized into named columns, and they leverage lazy evaluation for performance optimization. They are not stored on a single machine but partitioned across the cluster.
- Question 31Interactive
Analyze the following PySpark code block and determine the type of operation performed.
- ATransformationCorrect answer
- BAction
- CShuffle
- DPersist
Why this is the answer
The code shows a filter operation, which is a narrow transformation that returns a new DataFrame without triggering any computation until an action is called.
- Question 32Single Choice
Which method is used to read a Parquet file into a DataFrame in Databricks?
- Aspark.read.parquet()Correct answer
- Bspark.read.csv()
- Cspark.read.json()
- Dspark.read.text()
Why this is the answer
The spark.read.parquet() method is the standard way to read Parquet files into a DataFrame in PySpark. Parquet is a columnar storage format commonly used with Spark.
- Question 33Multiple Select
Which of the following are Spark actions? (Select all that apply.)
- Acount()Correct answer
- Bcollect()Correct answer
- Cfilter()
- Dshow()Correct answer
Why this is the answer
count(), collect(), and show() are actions that force Spark to evaluate transformations and return results. filter() is a transformation that creates a new DataFrame without computation.
- Question 34Interactive
Identify the error in the following PySpark code if any. If there is an error, select 'Error'. Otherwise, select 'Valid'.
- AErrorCorrect answer
- BValid
- CMissing import
- DSyntax warning
Why this is the answer
The code uses `spark` without defining it. In a Databricks notebook, `spark` is automatically available, but this snippet incorrectly uses `SparkSession` with a lowercase `s`. The correct way is `SparkSession.builder.getOrCreate()`, but typically `spark` is already defined. However, the line `spark = SparkSession.builder.getOrCreate()` is valid only if SparkSession is imported. Here, neither import nor definition exists; thus it would error.
- Question 35Single Choice
What does the `select()` transformation do in PySpark?
- AIt selects specific columns from a DataFrameCorrect answer
- BIt filters rows based on a condition
- CIt renames columns
- DIt sorts the DataFrame
Why this is the answer
The `select()` method returns a new DataFrame that contains only the specified columns (or expressions). It is a transformation that does not trigger computation.
- Question 36Multiple Select
Which of the following are valid ways to create a DataFrame in PySpark? (Select all that apply.)
- AReading from a file using spark.readCorrect answer
- BConverting an RDD using toDF()Correct answer
- CUsing the range() method on SparkSessionCorrect answer
- DCreating an empty DataFrame with a schemaCorrect answer
Why this is the answer
All listed options are valid ways to create a DataFrame in PySpark. You can read from files, convert RDDs, use spark.range(), or create an empty DataFrame with a defined schema.
- Question 37Interactive
What is the output of the following PySpark code? Assume the DataFrame `df` has 100 rows with column `col` of type integer.
- A100Correct answer
- B10
- C0
- DError
Why this is the answer
The `count()` action returns the number of rows in the DataFrame, which is 100. It triggers the computation of all transformations and returns the result.
Ensuring Data Security and Compliance
5 questions in this domain
- Question 38Single Choice
What is the primary purpose of a row filter in Unity Catalog?
- AIt automatically removes entire rows from query results based on the caller's identity or attributes.Correct answer
- BIt masks specific column values to hide sensitive data from unauthorized users.
- CIt restricts users from inserting new rows into a table.
- DIt encrypts row data at rest to prevent data breaches.
Why this is the answer
Row filters are SQL functions that return a Boolean expression. When a query runs, rows evaluating to FALSE or NULL are suppressed. This filters rows based on the caller (e.g., CURRENT_USER()), not column masking, write restrictions, or encryption.
- Question 39Multiple Select
Which statements accurately describe column masks in Unity Catalog? (Select two.)
- AColumn masks are SQL expressions that transform or hide column values at query time.Correct answer
- BColumn masks prevent users from seeing the column in DESCRIBE TABLE output.
- CColumn masks can show different values based on the caller's group membership.Correct answer
- DColumn masks are applied to INSERT and UPDATE statements to protect data integrity.
Why this is the answer
Column masks are applied at query time to SELECT results. They transform values (e.g., mask SSN) and can use IS_MEMBER to differentiate users. Masks do not affect schema viewing (DESCRIBE) nor write operations.
- Question 40Interactive
Review the following SQL statement and identify what it accomplishes:
- AIt attaches a row filter named 'region_filter' to the 'sales' table, so that every SELECT on the table will invoke the filter.Correct answer
- BIt creates a new view named 'region_filter' over the 'sales' table.
- CIt alters the 'sales' table to include a new column named 'region_filter'.
- DIt removes an existing row filter from the 'sales' table.
Why this is the answer
The ALTER TABLE ... SET ROW FILTER syntax attaches a previously created row filter function to a table. This ensures all queries against the table automatically apply the filter logic.
- Question 41Single Choice
What privilege is required to attach a row filter or column mask to a table in Unity Catalog?
- AAPPLYCorrect answer
- BMODIFY
- CSELECT
- DCREATE
Why this is the answer
The APPLY privilege is specifically designed for attaching row filters and column masks. MODIFY allows changing table data but not security policies. SELECT is for reading data, and CREATE is for creating objects.
- Question 42Multiple Select
Which two components are essential for achieving network isolation in Databricks?
- ADeploying the workspace into a customer-managed VPC/VNet.Correct answer
- BConfiguring IP access lists to allow only corporate IP ranges.Correct answer
- CUsing only managed VPC for simplicity and cost savings.
- DEnabling public network access for all users.
Why this is the answer
Network isolation requires a customer-managed VPC to control subnets and security groups, and IP access lists to restrict workspace UI and API access. Managed VPC does not support no-public-IP or Private Link. Public network access defeats isolation.
Monitoring and Alerting
5 questions in this domain
- Question 43Single Choice
What does the /api/2.1/jobs/runs/list endpoint return?
- APaginated run metadata including state and timingsCorrect answer
- BDetailed job definition and cluster configuration
- CA list of all job IDs in the workspace
- DThe Spark UI event logs for the last run
Why this is the answer
The /api/2.1/jobs/runs/list endpoint returns paginated metadata about job runs, including state, start time, duration, and other run-level information.
- Question 44Multiple Select
Which comparison operators are valid for a Databricks dashboard alert? (Select all that apply)
- A=Correct answer
- B!=Correct answer
- C<Correct answer
- D>Correct answer
Why this is the answer
Dashboard alerts support the comparison operators =, !=, <, >, <=, >=, as well as a 'no data' condition. The four listed are all valid.
- Question 45Interactive
Refer to the Spark UI stage output below. Based on this information, which of the following statements are correct? (Select all that apply)
- AThe stage exhibits partition skewCorrect answer
- BThe stage is running efficiently with no issues
- CA straggler task is presentCorrect answer
- DThe task duration is uniformly distributed
Why this is the answer
A very long max task duration compared to median indicates a straggler task, which is a sign of partition skew.
- Question 46Single Choice
What type of data is stored in the operationMetrics column of the DESCRIBE HISTORY output?
- AA JSON object with numeric metrics like numOutputRows and numAddedFilesCorrect answer
- BA list of file paths that were modified by the operation
- CThe SQL text of the operation that was executed
- DThe username and timestamp of the operation
Why this is the answer
operationMetrics is a JSON column containing key-value pairs of numeric metrics such as numOutputRows, numAddedFiles, and numDeletedFiles.
- Question 47Multiple Select
Which of the following are predefined sink types for custom Spark metrics on Databricks? (Select all that apply)
- AGraphiteCorrect answer
- BJMXCorrect answer
- CConsole
- DPrometheus
Why this is the answer
Databricks predefines Graphite and JMX as Spark metric sinks. Console and Prometheus are not included by default.
Frequently asked questions
Everything about the Databricks Certified Data Engineer Professional question bank and how to use it.