Google Cloud Professional Data Engineer PDE — Topic 1 Question 263

Question

You maintain ETL pipelines. You notice that a streaming pipeline running on Dataflow is taking a long time to process incoming data, which causes output delays. You also noticed that the pipeline graph was automatically optimized by Dataflow and merged into one step. You want to identify where the potential bottleneck is occurring. What should you do?

Accepted Answer

Correct answer: A. The correct answer is A. Option A: Insert a Reshuffle operation after each processing step, and monitor the execution details in the Dataflow console. Dataflow runs serverless Apache Beam pipelines for stream and batch data processing with autoscaling. This option meets the real-time / low-latency performance requirement.

Google Cloud Professional Data Engineer PDE — Question 263