🔍

DATA-ENGINEER-PROFESSIONAL — questions

Page 6 of 6 · 110 total questions.

DATA-ENGINEER-PROFESSIONAL Question 101

Single page →
Topic 1 · Question 101

Which statement describes the default execution mode for Databricks Auto Loader?

  • ACloud vendor-specific queue storage and notification services are configured to track newly arriving files; new files are incrementally and idempotently loaded into the target Delta Lake table.
  • BNew files are identified by listing the input directory; the target table is materialized by directly querying all valid files in the source directory.
  • CWebhooks trigger a Databricks job to run anytime new data arrives in a source directory; new data are automatically merged into target tables using rules inferred from the data.
  • DNew files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table. (correct answer)
Reveal answer & explanation
Correct answer: D

The correct answer is D. Option D: New files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table.

Explanation

Delta Lake adds ACID transactions, schema enforcement, time travel, and reliable batch and streaming operations to a data lake.

DATA-ENGINEER-PROFESSIONAL Question 102

Single page →
Topic 1 · Question 102

An upstream source writes Parquet data as hourly batches to directories named with the current date. A nightly batch job runs the following code to ingest all data from the previous day as indicated by the date variable: Assume that the fields customer_id and order_id serve as a composite key to uniquely identify each order. If the upstream system is known to occasionally produce duplicate entries for a single order hours apart, which statement is correct?

Exhibit 1 for question 102
  • AEach write to the orders table will only contain unique records, and only those records without duplicates in the target table will be written.
  • BEach write to the orders table will only contain unique records, but newly written records may have duplicates already present in the target table. (correct answer)
  • CEach write to the orders table will only contain unique records; if existing records with the same key are present in the target table, these records will be overwritten.
  • DEach write to the orders table will run deduplication over the union of new and existing records, ensuring no duplicate records are present.
Reveal answer & explanation
Correct answer: B

The correct answer is B. Option B: Each write to the orders table will only contain unique records, but newly written records may have duplicates already present in the target table.

DATA-ENGINEER-PROFESSIONAL Question 103

Single page →
Topic 1 · Question 103

A junior data engineer on your team has implemented the following code block. The view new_events contains a batch of records with the same schema as the events Delta table. The event_id field serves as a unique key for this table. When this query is executed, what will happen with new records that have the same event_id as an existing record?

Exhibit 1 for question 103
  • AThey are merged.
  • BThey are ignored. (correct answer)
  • CThey are updated.
  • DThey are inserted.
Reveal answer & explanation
Correct answer: B

The correct answer is B. Option B: They are ignored.

DATA-ENGINEER-PROFESSIONAL Question 104

Single page →
Topic 1 · Question 104

A team of data engineers are adding tables to a DLT pipeline that contain repetitive expectations for many of the same data quality checks. One member of the team suggests reusing these data quality rules across all tables defined for this pipeline. What approach would allow them to do this?

  • AAdd data quality constraints to tables in this pipeline using an external job with access to pipeline configuration files.
  • BUse global Python variables to make expectations visible across DLT notebooks included in the same pipeline.
  • CMaintain data quality rules in a separate Databricks notebook that each DLT notebook or file can import as a library.
  • DMaintain data quality rules in a Delta table outside of this pipeline's target schema, providing the schema name as a pipeline parameter. (correct answer)
Reveal answer & explanation
Correct answer: D

The correct answer is D. Option D: Maintain data quality rules in a Delta table outside of this pipeline's target schema, providing the schema name as a pipeline parameter.

DATA-ENGINEER-PROFESSIONAL Question 105

Single page →
Topic 1 · Question 105

A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy. The user attempts and fails to accomplish this by adding an expectation to the report table definition. Which approach would allow using DLT expectations to validate all expected records are present in this table?

Exhibit 1 for question 105
  • ADefine a temporary table that performs a left outer join on validation_copy and report, and define an expectation that no report key values are null
  • BDefine a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table
  • CDefine a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table (correct answer)
  • DDefine a function that performs a left outer join on validation_copy and report, and check against the result in a DLT expectation for the report table
Reveal answer & explanation
Correct answer: C

The correct answer is C. Option C: Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table

DATA-ENGINEER-PROFESSIONAL Question 106

Single page →
Topic 1 · Question 106

A user new to Databricks is trying to troubleshoot long execution times for some pipeline logic they are working on. Presently, the user is executing code cell-by-cell, using display() calls to confirm code is producing the logically correct results as new transformations are added to an operation. To get a measure of average time to execute, the user is running each cell multiple times interactively. Which of the following adjustments will get a more accurate measure of how code is likely to perform in production?

  • AThe Jobs UI should be leveraged to occasionally run the notebook as a job and track execution time during incremental code development because Photon can only be enabled on clusters launched for scheduled jobs.
  • BThe only way to meaningfully troubleshoot code execution times in development notebooks is to use production-sized data and production-sized clusters with Run All execution. (correct answer)
  • CProduction code development should only be done using an IDE; executing code against a local build of open source Spark and Delta Lake will provide the most accurate benchmarks for how code will perform in production.
  • DCalling display() forces a job to trigger, while many transformations will only add to the logical query plan; because of caching, repeated execution of the same logic does not provide meaningful results.
Reveal answer & explanation
Correct answer: B

The correct answer is B. Option B: The only way to meaningfully troubleshoot code execution times in development notebooks is to use production-sized data and production-sized clusters with Run All execution.

DATA-ENGINEER-PROFESSIONAL Question 107

Single page →
Topic 1 · Question 107

Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push-down?

  • AIn the Executor’s log file, by grepping for "predicate push-down"
  • BIn the Stage’s Detail screen, in the Completed Stages table, by noting the size of data read from the Input column
  • CIn the Query Detail screen, by interpreting the Physical Plan (correct answer)
  • DIn the Delta Lake transaction log. by noting the column statistics
Reveal answer & explanation
Correct answer: C

The correct answer is C. Option C: In the Query Detail screen, by interpreting the Physical Plan

DATA-ENGINEER-PROFESSIONAL Question 108

Single page →
Topic 1 · Question 108

A data engineer needs to capture pipeline settings from an existing setting in the workspace, and use them to create and version a JSON file to create a new pipeline. Which command should the data engineer enter in a web terminal configured with the Databricks CLI?

  • AUse list pipelines to get the specs for all pipelines; get the pipeline spec from the returned results; parse and use this to create a pipeline
  • BStop the existing pipeline; use the returned settings in a reset command
  • CUse the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command (correct answer)
  • DUse the clone command to create a copy of an existing pipeline; use the get JSON command to get the pipeline definition; save this to git
Reveal answer & explanation
Correct answer: C

The correct answer is C. Option C: Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command

DATA-ENGINEER-PROFESSIONAL Question 109

Single page →
Topic 1 · Question 109

Which REST API call can be used to review the notebooks configured to run as tasks in a multi-task job?

  • A/jobs/runs/list
  • B/jobs/list
  • C/jobs/runs/get
  • D/jobs/get (correct answer)
Reveal answer & explanation
Correct answer: D

The correct answer is D. Option D: /jobs/get

DATA-ENGINEER-PROFESSIONAL Question 110

Single page →
Topic 1 · Question 110

A Data Engineer wants to run unit tests using common Python testing frameworks on Python functions defined across several Databricks notebooks currently used in production. How can the data engineer run unit tests against functions that work with data in production?

  • ADefine and import unit test functions from a separate Databricks notebook
  • BDefine and unit test functions using Files in Repos (correct answer)
  • CRun unit tests against non-production data that closely mirrors production
  • DDefine unit tests and functions within the same notebook
Reveal answer & explanation
Correct answer: B

The correct answer is B. Option B: Define and unit test functions using Files in Repos

Explanation

Databricks Repos integrates Git-based source control with workspace development workflows.

Showing questions 101–110 of 110 · Page 6 of 6