GENERATIVE-AI-ENGINEER-ASSOCIATE — questions

Page 4 of 5 · 91 total questions.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 62

Single page →

Topic 1 · Question 62

A Generative AI Engineer has built an LLM-based system that will automatically translate user text between two languages. They now want to benchmark multiple LLM’s on this task and pick the best one. They have an evaluation set with known high quality translation examples. They want to evaluate each LLM using the evaluation set with a performant metric. Which metric should they choose for this evaluation?

ABLEU metric (correct answer)
BNDCG metric
CROUGE metric
DRECALL metric

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: BLEU metric

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 63

Single page →

Topic 1 · Question 63

A Generative AI Engineer is using an LLM to classify species of edible mushrooms based on text descriptions of certain features. The model is returning accurate responses in testing and the Generative AI Engineer is confident they have the correct list of possible labels, but the output frequently contains additional reasoning in the answer when the Generative AI Engineer only wants to return the label with no additional text. Which action should they take to elicit the desired behavior from this LLM?

AUse few shot prompting to instruct the model on expected output format (correct answer)
BUse zero shot prompting to instruct the model on expected output format
CUse zero shot chain-of-thought prompting to prevent a verbose output format
DUse a system prompt to instruct the model to be succinct in its answer

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Use few shot prompting to instruct the model on expected output format

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 64

Single page →

Topic 1 · Question 64

A Generative AI Engineer is working with a retail company that wants to enhance its customer experience by automatically handling common customer inquiries. They are working on an LLM-powered AI solution that should improve response times while maintaining a personalized interaction. They want to define the appropriate input and LLM task to do this. Which input/output pair will do this?

AInput: Customer service chat logs; Output: Group the chat logs by users, followed by summarizing each user’s interactions, then respond (correct answer)
BInput: Customer service chat logs; Output: Find the answers to similar questions and respond with a summary
CInput: Customer reviews; Output: Classify review sentiment
DInput: Customer reviews; Output: Group the reviews by users and aggregate per-user average rating, then respond

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Input: Customer service chat logs; Output: Group the chat logs by users, followed by summarizing each user’s interactions, then respond

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 65

Single page →

Topic 1 · Question 65

A Generative AI Engineer is developing a RAG application and would like to experiment with different embedding models to improve the application performance. Which strategy for picking an embedding model should they choose?

APick an embedding model with multilingual support to support potential multilingual user questions
BPick the most recent and most performant open LLM released at the time
CPick an embedding model trained on related domain knowledge (correct answer)
DPick the embedding model ranked highest on the Massive Text Embedding Benchmark (MTEB) leaderboard hosted by HuggingFace

Reveal answer & explanation

Correct answer: C

The correct answer is C. Option C: Pick an embedding model trained on related domain knowledge

Explanation

An embedding maps content into a numeric vector so semantically similar items can be retrieved by distance.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 66

Single page →

Topic 1 · Question 66

A Generative AI Engineer wants their finetuned LLMs in their prod Databricks workspace available for testing in their dev workspace as well. All of their workspaces are Unity Catalog enabled and they are currently logging their models into the Model Registry in MLflow. What is the most cost-effective and secure option for the Generative AI Engineer to accomplish their goal?

AUse an external model registry which can be accessed from all workspaces.
BUse MLflow to log the model directly into Unity Catalog, and enable READ access in the dev workspace to the model. (correct answer)
CSetup a duplicate training pipeline in dev, so that an identical model is available in dev.
DSetup a script to export the model from prod and import it to dev.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Use MLflow to log the model directly into Unity Catalog, and enable READ access in the dev workspace to the model.

Explanation

Unity Catalog provides centralized governance, fine-grained permissions, lineage, discovery, and auditing across Databricks data and AI assets. MLflow tracks experiments, packages models, manages model versions, and supports governed deployment workflows. This option delivers the requirement at the lowest cost.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 67

Single page →

Topic 1 · Question 67

A Generative AI Engineer has just deployed an LLM application at a manufacturing company that assists with answering customer service inquiries. They need to identity the key enterprise metrics to monitor the application in production. Which is NOT a metric they will implement for their customer service LLM application in production?

AMassive Multi-task Language Understanding (MMLU) score (correct answer)
BNumber of customer inquiries processed per unit of time
CFactual accuracy of the response
DTime taken for LLM to generate a response

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Massive Multi-task Language Understanding (MMLU) score

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 68

Single page →

Topic 1 · Question 68

Generative AI Engineer is helping a cinema extend its website’s chat bot to be able to respond to questions about specific showtimes for movies currently playing at their local theater. They already have the location of the user provided by location services to their agent, and a Delta table which is continually updated with the latest showtime information by location. They want to implement this new capability in their RAG application. Which option will do this with the least effort and in the most performant way?

ACreate a Feature Serving Endpoint from a FeatureSpec that references an online store synced from the Delta table. Query the Feature Serving Endpoint as part of the agent logic / tool implementation. (correct answer)
BQuery the Delta table directly via a SQL query constructed from the user’s input using a text-to-SQL LLM in the agent logic / tool implementation.
CSet up a task in Databricks Workflows to write the information in the Delta table periodically to an external database such as MySQL and query the information from there as part of the agent logic / tool implementation.
DWrite the Delta table contents to a text column, then embed those texts using an embedding model and store these in the vector index. Look up the information based on the embedding as part of the agent logic / tool implementation.

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Create a Feature Serving Endpoint from a FeatureSpec that references an online store synced from the Delta table. Query the Feature Serving Endpoint as part of the agent logic / tool implementation. This option is the most fully managed / serverless choice, so it meets the requirement for the least operational overhead.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 69

Single page →

Topic 1 · Question 69

Generative AI Engineer needs to build an LLM application that can understand medical documents, including recently published ones. They want to select an open model available on HuggingFace’s model hub. Which step is most appropriate for selecting an LLM?

APick any model in the Mistral family, as Mistral models are good with all types of use cases
BSelect a model based on the highest number of downloads, as this indicates popularity, reliability, and general suitability
CSelect a model that is most recently uploaded, as this indicates the model is the newest and highly likely to be the most performant
DCheck for the model and training data description to identify if the model is trained on any medical data. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Check for the model and training data description to identify if the model is trained on any medical data.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 70

Single page →

Topic 1 · Question 70

Generative AI Engineer is building a RAG application that answers questions about technology-related news articles. The source documents may contain a significant amount of irrelevant content, such as advertisements, sports news, or entertainment news. Which approach is NOT advisable for building a RAG application focused on answering technology-only questions?

AInclude in the system prompt that the application is not supposed to answer any questions unrelated to technology.
BFilter out irrelevant news articles in the retrieval process.
CKeep all news articles because the RAG application needs to understand non-technological content to avoid answering questions about them. (correct answer)
DFilter out irrelevant news articles in the upstream document database.

Reveal answer & explanation

Correct answer: C

The correct answer is C. Option C: Keep all news articles because the RAG application needs to understand non-technological content to avoid answering questions about them.

Explanation

Retrieval-augmented generation grounds model responses in retrieved enterprise data to improve relevance and reduce unsupported claims.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 71

Single page →

Topic 1 · Question 71

A Generative AI Engineer is building a RAG application that will rely on context retrieved from source documents that are currently in HTML format. They want to develop a solution using the least amount of lines of code. Which Python package should be used to extract the text from the source documents?

Apytesseract
Bnumpy
Cpypdf2
Dbeautifulsoup (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: beautifulsoup

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 72

Single page →

Topic 1 · Question 72

A Generative AI Engineer is building a RAG application for answering employee questions on company policies. What are the steps needed to build this RAG application and deploy it?

AIngest documents from a source -> Index the documents and saves to Vector Search -> User submits queries against an LLM -> LLM retrieves relevant documents -> Evaluate model -> LLM generates a response -> Deploy it using Model Serving
BUser submits queries against an LLM -> Ingest documents from a source -> Index the documents and save to Vector Search -> LLM retrieves relevant documents -> LLM generates a response -> Evaluate model -> Deploy it using Model Serving
CIngest documents from a source -> Index the documents and save to Vector Search -> Evaluate model -> Deploy it using Model Serving -> User submits queries against an LLM -> LLM retrieves relevant documents -> LLM generates a response
DIngest documents from a source -> Index the documents and save to Vector Search -> User submits queries against an LLM -> LLM retrieves relevant documents -> LLM generates a response -> Evaluate model -> Deploy it using Model Serving (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Ingest documents from a source -> Index the documents and save to Vector Search -> User submits queries against an LLM -> LLM retrieves relevant documents -> LLM generates a response -> Evaluate model -> Deploy it usi...

Explanation

Databricks Model Serving exposes models through scalable REST endpoints with managed deployment infrastructure. Databricks Vector Search builds and serves similarity indexes for retrieval-augmented generation applications.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 73

Single page →

Topic 1 · Question 73

A Generative AI Engineer who was prototyping an LLM system accidentally ran thousands of inference queries against a Foundation Model endpoint over the weekend. They want to take action to prevent this from unintentionally happening again in the future. What action should they take?

AUse prompt engineering to instruct the LLM endpoints to refuse too many subsequent queries.
BRequire that all development code which interfaces with a Foundation Model endpoint must be reviewed by a Staff level engineer before execution.
CBuild a pyfunc model which proxies to the Foundation Model endpoint and add throttling within the pyfune model.
DConfigure rate limiting on the Foundation Model endpoints. (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Configure rate limiting on the Foundation Model endpoints.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 74

Single page →

Topic 1 · Question 74

A Generative AI Engineer is setting up a Databricks Vector Search that will lookup news articles by topic within 10 days of the date specified. An example query might be “Tell me about monster truck news around January 5th 1992”. They want to do this with the least amount of effort. How can they set up their Vector Search index to support this use case?

ACreate separate indexes by topic and add a classifier model to appropriately pick the best index.
BInclude metadata columns for article date and topic to support metadata filtering. (correct answer)
CPass the query directly to the vector search index and return the best articles.
DSplit articles by 10 day blocks and return the block closest to the query.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Include metadata columns for article date and topic to support metadata filtering. This option is the most fully managed / serverless choice, so it meets the requirement for the least operational overhead.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 75

Single page →

Topic 1 · Question 75

A Generative AI Engineer developed an LLM application using the pay-per-token Foundation Model API. Now that the application is ready to be deployed, they would like to ensure the model endpoint can serve high incoming volumes of requests in production. What should the Generative AI Engineer consider?

ASwitch to using External Models instead
BThrottle the incoming batch of requests manually to avoid rate limiting issues
CChange to a model with a fewer number of parameters in order to reduce hardware constraint issues
DDeploy the model using provisioned throughput as it comes with performance guarantees (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Deploy the model using provisioned throughput as it comes with performance guarantees

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 76

Single page →

Topic 1 · Question 76

A Generative AI Engineer at a home appliance company has been asked to design an LLM based application that accomplishes the following business objective: answer customer questions on home appliances using the associated instruction manuals. Which set of high-level tasks should the Generative AI Engineer’s system perform?

ASplit instruction manuals into chunks and embed into a vector store. Use the question to retrieve best matched chunks of manual, and use the LLM to generate a response to the user based upon the manual retrieved. (correct answer)
BCreate an interaction matrix of historical user questions and appliance instruction manuals. Use ALS to factorize the matrix and create embeddings. Calculate the embeddings of new queries and use them to find the best manual. Use an LLM to generate a response to the question based upon the manual retrieved.
CCalculate averaged embeddings for each instruction manual, compare embeddings to user query to find the best manual. Pass the best manual with user query into an LLM with a large context window to generate a response to the employee.
DUse an LLM to summarize all of the instruction manuals. Provide summaries of each manual and user query into an LLM with a large context window to generate a response to the user.

Reveal answer & explanation

Correct answer: A

The correct answer is A. Option A: Split instruction manuals into chunks and embed into a vector store. Use the question to retrieve best matched chunks of manual, and use the LLM to generate a response to the user based upon the manual retrieved.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 77

Single page →

Topic 1 · Question 77

A Generative AI Engineer is developing an LLM application to interact with users to provide personalized movie recommendations. Given the potential for malicious user inputs, which technique would be most effective in safeguarding the application?

AReduce the time that the users can interact with the LLM
BIncrease the amount of compute that powers the LLM to process input faster
CAsk the LLM to remind the user that the input is malicious but continue the conversation with the user
DImplement a safety filter that detects any harmful inputs and ask the LLM to respond that it is unable to assist (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Implement a safety filter that detects any harmful inputs and ask the LLM to respond that it is unable to assist

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 78

Single page →

Topic 1 · Question 78

A Generative AI Engineer received the following business requirements for an internal chatbot. The internal chatbot needs to know what types of questions the user asks and route them to appropriate models to answer the questions. For example, the user might ask about historical failure rates of a specific electrical part. Another user might ask about how to troubleshoot a piece of electrical equipment. Available data sources include a database of electrical equipment PDF manuals and also a table with information on when an electrical part experiences failure. Which workflow supports such a chatbot?

AParse the electrical equipment PDF manuals into a table of question and response pairs. That way, the same chatbot can query tables easily to answer questions about both historical failure rates and equipment troubleshooting.
BThe chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it’s a historical failure rate question, send the query to a text-to-SQL model. If it’s a troubleshooting question, then send the query to another model that summarizes the equipment-specific document and generates the response. (correct answer)
CThere should be two different chatbots handling different types of user queries.
DThe table with electrical part failures should be converted into a text document first. That way, the same chatbot can use the same document retrieval process to generate answers regardless of question types.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: The chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it’s a historical failure rate question, send the query...

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 79

Single page →

Topic 1 · Question 79

A Generative AI Engineer is building a system that will answer questions on currently unfolding news topics. As such, it pulls information from a variety of sources including articles and social media posts. They are concerned about toxic posts on social media causing toxic outputs from their system. Which guardrail will limit toxic outputs?

AReduce the amount of context items the system will include in consideration for its response.
BUse only approved social media and news accounts to prevent unexpected toxic data from getting to the LLM. (correct answer)
CLog all LLM system responses and perform a batch toxicity analysis monthly.
DImplement rate limiting.

Reveal answer & explanation

Correct answer: B

The correct answer is B. Option B: Use only approved social media and news accounts to prevent unexpected toxic data from getting to the LLM.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 80

Single page →

Topic 1 · Question 80

A Generative AI Engineer has created a RAG application which can help employees interpret HR documentation. The prototype application is now working with some positive feedback from internal company testers. Now the Generative AI Engineer wants to formally evaluate the system’s performance and understand where to focus their efforts to further improve the system How should the Generative AI Engineer evaluate the system?

AUse ROUGE score to comprehensively evaluate the quality of the final generated answers.
BUse an LLM-as-a-judge to evaluate the quality of the final answers generated.
CCurate a dataset that can test the retrieval and generation components of the system separately. Use MLflow’s built in evaluation metrics to perform the evaluation on the retrieval and generation components. (correct answer)
DBenchmark multiple LLMs with the same data and pick the best LLM for the job.

Reveal answer & explanation

Correct answer: C

The correct answer is C. Option C: Curate a dataset that can test the retrieval and generation components of the system separately. Use MLflow’s built in evaluation metrics to perform the evaluation on the retrieval and generation components.

Explanation

MLflow tracks experiments, packages models, manages model versions, and supports governed deployment workflows.

GENERATIVE-AI-ENGINEER-ASSOCIATE Question 81

Single page →

Topic 1 · Question 81

A Generative AI Engineer is using LangChain to assist a museum in classifying documents and using this code: Their code results in an error. What do they need to change in order to fix this template?

AProvide an LLM argument to PromptTemplate()
BProvide template and LLM arguments to PromptTemplate()
COmit PromptTemplate(), it is only used for multi-part templates
DProvide a template argument to PromptTemplate() (correct answer)

Reveal answer & explanation

Correct answer: D

The correct answer is D. Option D: Provide a template argument to PromptTemplate()

Showing questions 61–80 of 91 · Page 4 of 5