A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.

Which approach would allow using DLT expectations to validate all expected records are present in this table?

• A. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table
• B. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
• C. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table
• D. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null

A query is taking too long to run. After investigating the Spark UI, the data engineer discovered a significant amount of disk spill. The compute instance being used has a core-to-memory ratio of
1:2. What are the two steps the data engineer should take to minimize spillage? (Choose two.)

• A. Reduce spark.sql.files.maxPartitionBytes.
• B. Choose a compute instance with more network bandwidth.
• C. Choose a compute instance with a higher core-to-memory ratio.
• D. Increase spark.sql.files.maxPartitionBytes.
• E. Choose a compute instance with more disk space.

When monitoring a complex workload, being able to see the query plan is critical to understanding what the workload is doing. Where can the visualization of the query plan be found?

• A. In the Query Profiler, under the Stages tab
• B. In the Spart UI, under the Jobs tab
• C. In the Spark UI, under the SQL/DataFrame tab
• D. In the Query Profiler, under Query Source

A data engineer needs to capture pipeline settings from an existing in the workspace, and use them to create and version a JSON file to create a new pipeline. Which command should the data engineer enter in a web terminal configured with the Databricks CLI?

• A. Stop the existing pipeline; use the returned settings in a reset command
• B. Use list pipelines to get the specs for all pipelines; get the pipeline spec from the return results parse and use this to create a pipeline
• C. Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command
• D. Use the alone command to create a copy of an existing pipeline; use the get JSON command to get the pipeline definition; save this to git

An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the field pk_id.
For auditing purposes, the data governance team wishes to maintain a full record of all values that have ever been valid in the source system. For analytical purposes, only the most recent value for each record needs to be recorded. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour.
Which solution meets these requirements?

• A. Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.
• B. Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
• C. Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
• D. Ingest all log information into a bronze table; use merge into to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.
• E. Use merge into to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.