GENERATIVE-AI-ENGINEER-ASSOCIATE — all questions

91 practice questions with answers and explanations.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 1

Single page →

Topic 1 · Question 1 · Select all that apply

A Generative Al Engineer has created a RAG application to look up answers to questions about a series of fantasy novels that are being asked on the author’s web forum. The fantasy novel texts are chunked and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with the user’s query, and provided to an LLM for response generation. The Generative AI Engineer used their intuition to pick the chunking strategy and associated configurations but now wants to more methodically choose the best values. Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and parameters? (Choose two.)

AChange embedding models and compare performance.
BAdd a classifier for user queries that predicts which book will best contain the answer. Use this to filter retrieval.
CChoose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best performance metric. (correct answer)
DPass known questions and best answers to an LLM and instruct the LLM to provide the best token count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
ECreate an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric. (correct answer)

Reveal answer & explanation

Correct answer: C, E

The correct answer is C, E. Option C: Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best perfor... Option E: Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric.

Explanation

OPTIMIZE compacts small Delta files to improve data-skipping and query performance. Retrieval-augmented generation grounds model responses in retrieved enterprise data to improve relevance and reduce unsupported claims. Chunking divides source documents into retrievable passages whose size and overlap affect recall and answer context.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 2

Single page →

Topic 1 · Question 2

A Generative AI Engineer is designing a RAG application for answering user questions on technical regulations as they learn a new sport. What are the steps needed to build this RAG application and deploy it?

AIngest documents from a source –> Index the documents and saves to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> Evaluate model –> LLM generates a response –> Deploy it using Model Serving
BIngest documents from a source –> Index the documents and save to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model –> Deploy it using Model Serving (correct answer)
CIngest documents from a source –> Index the documents and save to Vector Search –> Evaluate model –> Deploy it using Model Serving
DUser submits queries against an LLM –> Ingest documents from a source –> Index the documents and save to Vector Search –> LLM retrieves relevant documents –> LLM generates a response –> Evaluate model –> Deploy it using Model Serving

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Ingest documents from a source –> Index the documents and save to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model –> Deploy it usi...

Explanation

Databricks Model Serving exposes models through scalable REST endpoints with managed deployment infrastructure. Databricks Vector Search builds and serves similarity indexes for retrieval-augmented generation applications.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 3

Single page →

Topic 1 · Question 3

A Generative AI Engineer just deployed an LLM application at a digital marketing company that assists with answering customer service inquiries. Which metric should they monitor for their customer service LLM application in production?

ANumber of customer inquiries processed per unit of time (correct answer)
BEnergy usage per query
CFinal perplexity scores for the training of the model
DHuggingFace Leaderboard values for the base LLM

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Number of customer inquiries processed per unit of time

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 4

Single page →

Topic 1 · Question 4

A Generative AI Engineer is building a Generative AI system that suggests the best matched employee team member to newly scoped projects. The team member is selected from a very large team. The match should be based upon project date availability and how well their employee profile matches the project scope. Both the employee profile and project scope are unstructured text. How should the Generative Al Engineer architect their system?

ACreate a tool for finding available team members given project dates. Embed all project scopes into a vector store, perform a retrieval using team member profiles to find the best team member.
BCreate a tool for finding team member availability given project dates, and another tool that uses an LLM to extract keywords from project scopes. Iterate through available team members’ profiles and perform keyword matching to find the best available team member.
CCreate a tool to find available team members given project dates. Create a second tool that can calculate a similarity score for a combination of team member profile and the project scope. Iterate through the team members and rank by best score to select a team member.
DCreate a tool for finding available team members given project dates. Embed team profiles into a vector store and use the project scope and filtering to perform retrieval to find the available best matched team members. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Create a tool for finding available team members given project dates. Embed team profiles into a vector store and use the project scope and filtering to perform retrieval to find the available best matched team members.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 5

Single page →

Topic 1 · Question 5

A Generative AI Engineer is designing an LLM-powered live sports commentary platform. The platform provides real-time updates and LLM-generated analyses for any users who would like to have live summaries, rather than reading a series of potentially outdated news articles. Which tool below will give the platform access to real-time data for generating game analyses based on the latest game scores?

ADatabricksIQ
BFoundation Model APIs
CFeature Serving (correct answer)
DAutoML

Reveal answer & explanation

Correct answer: C

The correct answer is C. Option C: Feature Serving This option meets the real-time / low-latency performance requirement.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 6

Single page →

Topic 1 · Question 6

A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and would like to monitor the serving endpoint’s incoming requests and outgoing responses. The current approach is to include a micro-service in between the endpoint and the user interface to write logs to a remote server. Which Databricks feature should they use instead which will perform the same task?

AVector Search
BLakeview
CDBSQL
DInference Tables (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Inference Tables

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 7

Single page →

Topic 1 · Question 7

A Generative Al Engineer is tasked with improving the RAG quality by addressing its inflammatory outputs. Which action would be most effective in mitigating the problem of offensive text outputs?

AIncrease the frequency of upstream data updates
BInform the user of the expected RAG behavior
CRestrict access to the data sources to a limited number of users
DCurate upstream data properly that includes manual review before it is fed into the RAG system (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Curate upstream data properly that includes manual review before it is fed into the RAG system

Explanation

Retrieval-augmented generation grounds model responses in retrieved enterprise data to improve relevance and reduce unsupported claims.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 8

Single page →

Topic 1 · Question 8

A Generative Al Engineer is creating an LLM-based application. The documents for its retriever have been chunked to a maximum of 512 tokens each. The Generative Al Engineer knows that cost and latency are more important than quality for this application. They have several context length levels to choose from. Which will fulfill their need?

Acontext length 514; smallest model is 0.44GB and embedding dimension 768
Bcontext length 2048: smallest model is 11GB and embedding dimension 2560
Ccontext length 32768: smallest model is 14GB and embedding dimension 4096
Dcontext length 512: smallest model is 0.13GB and embedding dimension 384 (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: context length 512: smallest model is 0.13GB and embedding dimension 384

Explanation

An embedding maps content into a numeric vector so semantically similar items can be retrieved by distance.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 9

Single page →

Topic 1 · Question 9

A small and cost-conscious startup in the cancer research field wants to build a RAG application using Foundation Model APIs. Which strategy would allow the startup to build a good-quality RAG application while being cost-conscious and able to cater to customer needs?

ALimit the number of relevant documents available for the RAG application to retrieve from
BPick a smaller LLM that is domain-specific (correct answer)
CLimit the number of queries a customer can send per day
DUse the largest LLM possible because that gives the best performance for any general queries

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Pick a smaller LLM that is domain-specific

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 10

Single page →

Topic 1 · Question 10 · Select all that apply

A Generative Al Engineer is responsible for developing a chatbot to enable their company’s internal HelpDesk Call Center team to more quickly find related tickets and provide resolution. While creating the GenAI application work breakdown tasks for this project, they realize they need to start planning which data sources (either Unity Catalog volume or Delta table) they could choose for this application. They have collected several candidate data sources for consideration: call_rep_history: a Delta table with primary keys representative_id, call_id. This table is maintained to calculate representatives’ call resolution from fields call_duration and call start_time. transcript Volume: a Unity Catalog Volume of all recordings as a *.wav files, but also a text transcript as *.txt files. call_cust_history: a Delta table with primary keys customer_id, cal1_id. This table is maintained to calculate how much internal customers use the HelpDesk to make sure that the charge back model is consistent with actual service use. call_detail: a Delta table that includes a snapshot of all call details updated hourly. It includes root_cause and resolution fields, but those fields may be empty for calls that are still active. maintenance_schedule – a Delta table that includes a listing of both HelpDesk application outages as well as planned upcoming maintenance downtimes. They need sources that could add context to best identify ticket root cause and resolution. Which TWO sources do that? (Choose two.)

Acall_cust_history
Bmaintenance_schedule
Ccall_rep_history
Dcall_detail (correct answer)
Etranscript Volume (correct answer)

Reveal answer & explanation

Correct answer: D, E

The correct answer is D, E. Option D: call_detail Option E: transcript Volume

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 11

Single page →

Topic 1 · Question 11

What is the most suitable library for building a multi-step LLM-based workflow?

APandas
BTensorFlow
CPySpark
DLangChain (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: LangChain

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 12

Single page →

Topic 1 · Question 12

When developing an LLM application, it’s crucial to ensure that the data used for training the model complies with licensing requirements to avoid legal risks. Which action is NOT appropriate to avoid legal risks?

AReach out to the data curators directly before you have started using the trained model to let them know.
BUse any available data you personally created which is completely original and you can decide what license to use.
COnly use data explicitly labeled with an open license and ensure the license terms are followed.
DReach out to the data curators directly after you have started using the trained model to let them know. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Reach out to the data curators directly after you have started using the trained model to let them know.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 14

Single page →

Topic 1 · Question 14

A Generative Al Engineer is creating an LLM system that will retrieve news articles from the year 1918 and related to a user's query and summarize them. The engineer has noticed that the summaries are generated well but often also include an explanation of how the summary was generated, which is undesirable. Which change could the Generative Al Engineer perform to mitigate this issue?

ASplit the LLM output by newline characters to truncate away the summarization explanation.
BTune the chunk size of news articles or experiment with different embedding models.
CRevisit their document ingestion logic, ensuring that the news articles are being ingested properly.
DProvide few shot examples of desired output format to the system and/or user prompt. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Provide few shot examples of desired output format to the system and/or user prompt.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 15

Single page →

Topic 1 · Question 15

A Generative Al Engineer has developed an LLM application to answer questions about internal company policies. The Generative AI Engineer must ensure that the application doesn’t hallucinate or leak confidential data. Which approach should NOT be used to mitigate hallucination or confidential data leakage?

AAdd guardrails to filter outputs from the LLM before it is shown to the user
BFine-tune the model on your data, hoping it will learn what is appropriate and not (correct answer)
CLimit the data available based on the user’s access level
DUse a strong system prompt to ensure the model aligns with your needs.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Fine-tune the model on your data, hoping it will learn what is appropriate and not

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 16

Single page →

Topic 1 · Question 16

A Generative Al Engineer interfaces with an LLM with prompt/response behavior that has been trained on customer calls inquiring about product availability. The LLM is designed to output “In Stock” if the product is available or only the term “Out of Stock” if not. Which prompt will work to allow the engineer to respond to call classification labels correctly?

ARespond with “In Stock” if the customer asks for a product.
BYou will be given a customer call transcript where the customer asks about product availability. The outputs are either “In Stock” or “Out of Stock”. Format the output in JSON, for example: {“call_id”: “123”, “label”: “In Stock”}.
CRespond with “Out of Stock” if the customer asks for a product.
DYou will be given a customer call transcript where the customer inquires about product availability. Respond with “In Stock” if the product is available or “Out of Stock” if not. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: You will be given a customer call transcript where the customer inquires about product availability. Respond with “In Stock” if the product is available or “Out of Stock” if not.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 17

Single page →

Topic 1 · Question 17

A Generative Al Engineer is tasked with developing a RAG application that will help a small internal group of experts at their company answer specific questions, augmented by an internal knowledge base. They want the best possible quality in the answers, and neither latency nor throughput is a huge concern given that the user group is small and they’re willing to wait for the best answer. The topics are sensitive in nature and the data is highly confidential and so, due to regulatory requirements, none of the information is allowed to be transmitted to third parties. Which model meets all the Generative Al Engineer’s needs in this situation?

ADolly 1.5B
BOpenAI GPT-4
CBGE-large
DLlama2-70B (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Llama2-70B

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 18

Single page →

Topic 1 · Question 18

A Generative Al Engineer would like an LLM to generate formatted JSON from emails. This will require parsing and extracting the following information: order ID, date, and sender email. Here’s a sample email: They will need to write a prompt that will extract the relevant information in JSON format with the highest level of output accuracy. Which prompt will do that?

AYou will receive customer emails and need to extract date, sender email, and order ID. You should return the date, sender email, and order ID information in JSON format.
BYou will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in JSON format.Here’s an example: {“date”: “April 16, 2024”, “sender_email”: “[email protected]”, “order_id”: “RE987D”} (correct answer)
CYou will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in a human-readable format.
DYou will receive customer emails and need to extract date, sender email, and order IReturn the extracted information in JSON format.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: You will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in JSON format.Here’s an example: {“date”: “April 16, 2024”, “sender_email”: “[email protected]”,...

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 19

Single page →

Topic 1 · Question 19

A Generative AI Engineer has been asked to build an LLM-based question-answering application. The application should take into account new documents that are frequently published. The engineer wants to build this application with the least cost and least development effort and have it operate at the lowest cost possible. Which combination of chaining components and configuration meets these requirements?

AFor the application a prompt, a retriever, and an LLM are required. The retriever output is inserted into the prompt which is given to the LLM to generate answers. (correct answer)
BThe LLM needs to be frequently with the new documents in order to provide most up-to-date answers.
CFor the question-answering application, prompt engineering and an LLM are required to generate answers.
DFor the application a prompt, an agent and a fine-tuned LLM are required. The agent is used by the LLM to retrieve relevant content that is inserted into the prompt which is given to the LLM to generate answers.

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: For the application a prompt, a retriever, and an LLM are required. The retriever output is inserted into the prompt which is given to the LLM to generate answers. This option delivers the requirement at the lowest cost.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 20

Single page →

Topic 1 · Question 20

A Generative AI Engineer is creating an agent-based LLM system for their favorite monster truck team. The system can answer text based questions about the monster truck team, lookup event dates via an API call, or query tables on the team’s latest standings. How could the Generative AI Engineer best design these capabilities into their system?

AIngest PDF documents about the monster truck team into a vector store and query it in a RAG architecture.
BWrite a system prompt for the agent listing available tools and bundle it into an agent system that runs a number of calls to solve a query. (correct answer)
CInstruct the LLM to respond with “RAG”, “API”, or “TABLE” depending on the query, then use text parsing and conditional statements to resolve the query.
DBuild a system prompt with all possible event dates and table information in the system prompt. Use a RAG architecture to lookup generic text questions and otherwise leverage the information in the system prompt.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Write a system prompt for the agent listing available tools and bundle it into an agent system that runs a number of calls to solve a query.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 21

Single page →

Topic 1 · Question 21

A Generative AI Engineer has been asked to design an LLM-based application that accomplishes the following business objective: answer employee HR questions using HR PDF documentation. Which set of high level tasks should the Generative AI Engineer's system perform?

ACalculate averaged embeddings for each HR document, compare embeddings to user query to find the best document. Pass the best document with the user query into an LLM with a large context window to generate a response to the employee.
BUse an LLM to summarize HR documentation. Provide summaries of documentation and user query into an LLM with a large context window to generate a response to the user.
CCreate an interaction matrix of historical employee questions and HR documentation. Use ALS to factorize the matrix and create embeddings. Calculate the embeddings of new queries and use them to find the best HR documentation. Use an LLM to generate a response to the employee question based upon the documentation retrieved.
DSplit HR documentation into chunks and embed into a vector store. Use the employee question to retrieve best matched chunks of documentation, and use the LLM to generate a response to the employee based upon the documentation retrieved. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Split HR documentation into chunks and embed into a vector store. Use the employee question to retrieve best matched chunks of documentation, and use the LLM to generate a response to the employee based upon the docum...

Showing questions 1–20 of 91 · Page 1 of 5