Q001 - Question
A web shop wants a model to recommend other catalog items as soon as a customer selects a product. The recommendations must appear when the product details page loads. Select two statements that apply to the deployment design.
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Multiple choice
- A. Deploy the model to a batch endpoint and run scoring once a week.
- B. Deploy the model to a real-time endpoint.
- C. Use a compute cluster that the workspace scales down to 0 nodes when no data is waiting to be processed.
- D. Plan for compute that is always on, because the model must be available for immediate predictions.
B and D are correct.
Explanation: The evidence supports B and D because a model that must score new data as it comes in, such as for a website, needs real-time predictions. A real-time endpoint uses compute that is always available and can return results almost immediately. You can't pause or stop that compute, so it is always on.
A is incorrect: Batch scoring suits predictions that are consumed at certain times, such as a weekly forecast. It does not return recommendations when a customer selects a product.
C is incorrect: Scaling a compute cluster down to 0 nodes applies to batch scoring, where compute is provisioned when scoring is triggered. A real-time endpoint must stay available.
Q002 - Question
An organization needs one cloud service to train, register, deploy, and manage its own machine learning models. Data scientists want a UI-based experience, and engineers want a code-first experience that uses the Python SDK or CLI. The service must also provide on-demand compute for training jobs and built-in support for evaluating responsible AI metrics. Which service should the organization choose?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Azure Machine Learning
- B. Foundry Tools
- C. Microsoft Fabric
- D. Azure Databricks
A is correct.
Explanation: The evidence supports A because Azure Machine Learning is a cloud service for training, deploying, and managing machine learning models. You can use the studio for a UI-based experience, or the Python SDK or CLI for a code-first experience. It also provides on-demand compute and built-in support for visualizing and evaluating responsible AI metrics.
B is incorrect: Foundry Tools is a collection of prebuilt models offered as APIs. Some models can be customized, but the requirement is to train and manage the organization's own models.
C is incorrect: Microsoft Fabric is an integrated analytics platform for workflows between data analysts, data engineers, and data scientists, with results visualized in Power BI reports. It is not the service described for the registration, deployment, and responsible AI evaluation requirements.
D is incorrect: Azure Databricks is a data analytics platform that uses distributed Spark compute. You can train models in it, but the requirements point to Azure Machine Learning.
Q003 - Question
A team will train a model in Azure Machine Learning on a smaller tabular dataset. The team wants a cost-effective compute choice. It also wants to learn from each training run whether to scale compute up or down. Which approach should the team use?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Select GPU compute for all training jobs, because GPUs are required for tabular data.
- B. Distribute training by using Spark compute, because it works without changes to the training scripts.
- C. Start with CPU compute, and monitor training time and compute utilization for each run.
- D. Select memory optimized compute, because it is the only option for datasets with fewer rows.
C is correct.
Explanation: The evidence supports C because for smaller tabular datasets, a CPU is sufficient and cost-effective. Each time you train a model, you should monitor how long it takes and how much compute your code uses to know whether to scale compute up or down.
A is incorrect: GPUs are more powerful and efficient for unstructured data such as images or text. For tabular data, use a GPU only if CPU compute is insufficient.
B is incorrect: Spark compute can distribute model training, but it requires you to rewrite your training scripts. It is also not needed for a smaller dataset.
D is incorrect: Memory optimized compute has a high memory-to-CPU ratio and suits larger datasets or notebook work. General purpose compute has a balanced ratio and suits testing and development with smaller datasets.
Q004 - Question
A training script calls mlflow.autolog(). The command job references a Conda environment that installs only numpy, pandas, scikit-learn, and matplotlib with pip. The team wants Azure Machine Learning to store the tracked parameters, metrics, and artifacts with each job run. What should they do?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Keep the environment as is, because mlflow.autolog() installs the required packages on the compute.
- B. Add the mlflow and azureml-mlflow pip packages to the Conda environment, and reference that environment in the job.
- C. Add only the mlflow pip package to the environment, because azureml-mlflow is needed only to view results in the studio.
- D. Replace mlflow.autolog() with mlflow.log_param(), because logging functions do not require extra packages.
B is correct.
Explanation: The evidence supports B because to use MLflow during a training job, the mlflow and azureml-mlflow pip packages must be installed on the compute that runs the script. Include both packages in the Conda environment, register the environment, and refer to it when you submit the job. The tracked parameters, metrics, and artifacts are then stored with the job run.
A is incorrect: Adding mlflow.autolog() to the script enables autologging, but the packages must still be installed on the compute through the environment.
C is incorrect: Both the mlflow and azureml-mlflow packages must be installed. Installing only one does not meet the requirement.
D is incorrect: The mlflow.log_* functions are also MLflow tracking options. Both options need the packages in the environment.
Q005 - Question
A data scientist has a notebook that reads diabetes.csv, runs df.head() and df.describe(), prints several status messages, and then trains a model. The team plans to run the code every week in an automation pipeline. They want to reduce cost and compute time and confirm that the code works before it is integrated into the pipeline. Which approach should the data scientist use?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Keep all exploratory statements and add more print statements so each step is confirmed in every run.
- B. Submit the converted script to the automation pipeline first and review any errors from the pipeline runs.
- C. Remove the exploratory code that has no purpose in production, then run the script with python train.py in the terminal of the compute instance.
- D. Keep the exploratory code in the script and change the compute to a larger size to offset the extra run time.
C is correct.
Explanation: The evidence supports C because exploratory statements such as df.describe() are useful in a notebook but add cost and compute time when the code runs regularly. Remove code that has no purpose in production. Then run the script in the terminal of the compute instance. Print output and errors appear in the terminal, so you can verify that the script works before you integrate it with automation.
A is incorrect: Extra print statements add code to every run. Keep print statements only if you plan to review the script output.
B is incorrect: Test the script before you integrate it with automation pipelines. The terminal of the compute instance is a simple way to do this.
D is incorrect: A larger compute does not remove the nonessential code. The goal is to avoid running code that has no purpose, not to provide more compute for it.
Q006 - Question
IoT devices send temperature measurements as JSON objects at regular intervals. A team wants to train a forecasting model on one table that contains the average temperature per machine per minute. The team wants the data preparation steps to run on a schedule. Which design meets the requirement?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Send the raw JSON objects directly to a batch endpoint for scoring.
- B. Select GPU compute in Azure Machine Learning so that the JSON objects are aggregated during training.
- C. Use automated machine learning to convert the JSON objects to a table before any data is extracted.
- D. Create a data ingestion pipeline that extracts the JSON objects, converts them to a table, aggregates the data per machine per minute, and stores the prepared data in a serving layer such as Azure Blob Storage.
D is correct.
Explanation: The evidence supports D because a data ingestion pipeline is a sequence of tasks that move and transform data, and you can trigger it manually or schedule it. In this scenario, the pipeline extracts the JSON objects, converts them to a table, aggregates the data, and stores it in a serving layer.
A is incorrect: A batch endpoint is used to score new data with a trained model. It does not prepare training data.
B is incorrect: Compute selection affects how much processing power is available for training. It does not convert JSON objects to a table or create per-minute aggregates.
C is incorrect: Automated machine learning runs training jobs with different algorithms and parameters. The data must first be extracted from the source and made available in the required format.
Q007 - Question
A team submits a command job with the command "python train.py --training_data diabetes.csv". The script hard-codes a different file path and does not use the value passed in the command. The team wants to reuse the same script with different datasets. What should they change?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Change the display_name of the command job to match the dataset name.
- B. Change the experiment_name of the command job to match the dataset name.
- C. Change the compute of the command job so that it reads the dataset.
- D. Define a --training_data argument in the script by using a library such as argparse, and assign the value to a variable that reads the data.
D is correct.
Explanation: The evidence supports D because to use parameters in a script, you must use a library such as argparse to read the arguments passed to the script and assign them to variables. The script must define each parameter it expects. After the script reads --training_data, you can pass a different path in the command for each job.
A is incorrect: The display_name is the name of the individual job. It does not pass a value to the script.
B is incorrect: The experiment_name specifies which experiment the job belongs to. The script still needs code to read the argument.
C is incorrect: The compute parameter specifies where the command runs. It does not change how the script receives arguments.
Q008 - Question
A team submits several command jobs that run train.py from the ./src folder on a compute cluster named aml-cluster. In Azure Machine Learning studio, they want all of these jobs grouped under one experiment, and they want to find each individual job by name. Which command function parameters should they configure for this purpose?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Use the same experiment_name for all jobs and set display_name to identify each individual job.
- B. Use the same compute for all jobs and set environment to identify each individual job.
- C. Use the same code folder for all jobs and set command to identify each individual job.
- D. Use the same environment for all jobs and set compute to identify each individual job.
A is correct.
Explanation: The evidence supports A because the experiment_name parameter specifies the experiment that a job belongs to. All jobs with the same experiment name are grouped under the same experiment. The display_name parameter is the name of the individual job, so you can use it to find a specific job in the studio.
B is incorrect: The compute parameter specifies which compute runs the command. It does not group jobs into an experiment. The environment parameter specifies the packages to install on the compute. It does not name a job.
C is incorrect: The code parameter specifies the folder that includes the script. The command parameter specifies which file to run. Neither parameter groups jobs or names them.
D is incorrect: The environment parameter specifies packages, not experiment grouping. The compute parameter selects the compute to run the command. It does not provide a job name.
Q009 - Question
A data science team has historical weekly sales records. The team must build a model that predicts the number of units that will be sold in each of the next several weeks. Before the team selects algorithms and evaluation metrics, it must identify the machine learning task. Which task should the team choose?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Classification
- B. Time-series forecasting
- C. Computer vision
- D. Natural language processing (NLP)
B is correct.
Explanation: The evidence supports B because the model must predict future numerical values from time-series data, which describes time-series forecasting. The task determines which algorithms you can use to train the model, and the available performance metrics also depend on the task.
A is incorrect: Classification predicts a categorical value, such as whether a patient has diabetes. A unit count is a numerical value.
C is incorrect: Computer vision classifies images or detects objects in images. The data in this scenario is sales records, not images.
D is incorrect: Natural language processing (NLP) extracts insights from text. The data in this scenario is numerical sales history, not text.
Q010 - Question
Different team members logged runs of the same model in separate experiments in an Azure Machine Learning workspace. A data scientist uses MLflow in a notebook to retrieve only runs where the num_boost_round parameter is 100, with the most recent runs first. Which two actions should the data scientist take? Select two.
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Multi-select
- A. Call mlflow.search_runs with search_all_experiments=True to search runs across all experiments in the workspace.
- B. Call mlflow.search_experiments with view_type=ViewType.ALL to return the runs that have the matching parameter.
- C. Call mlflow.get_experiment_by_name for one experiment and search only that experiment by using its experiment ID.
- D. Use filter_string="params.num_boost_round='100'" and order_by=["start_time DESC"] in the mlflow.search_runs call.
A and D are correct.
Explanation: The evidence supports A and D because you can search runs across multiple experiments, which is useful when different people log the same model in separate experiments. Use search_all_experiments=True to search all experiments in the workspace. The filter_string parameter finds runs with a specific hyperparameter value. The order_by parameter changes the default sort order, so order_by=["start_time DESC"] returns the most recent runs first.
B is incorrect: mlflow.search_experiments returns experiments, and ViewType.ALL includes archived experiments. To retrieve runs, use mlflow.search_runs.
C is incorrect: Searching with the ID of one experiment limits the results to that experiment. The runs are logged in separate experiments, so this action misses runs from the others.
Q011 - Question
A team enables streaming (stream: true) on a conversational agent. They plan to compare it with the non-streaming baseline. Which two actions should they take in the experiment design? Select two answers.
Domain: Optimize generative AI systems and model performance (10–15%) Type: Multi select
- A. Expect streaming to reduce the total end-to-end response time, and use that as the only performance metric.
- B. Measure time-to-first-token and compare it with a threshold defined for streaming responses.
- C. Set the success thresholds after reviewing the results so that they match what the variants achieved.
- D. Define the quality, cost, and performance thresholds before running the experiment.
B and D are correct.
Explanation: Streaming does not change total response time, but it changes how quickly users see the agent start to respond. Time-to-first-token measures that perceived responsiveness. Defining thresholds before you run experiments, such as a time-to-first-token target, prevents you from rationalizing disappointing results.
A is incorrect: Streaming does not change total response time. End-to-end response time alone does not show the improvement that streaming provides.
C is incorrect: Setting thresholds after you see the results allows you to rationalize results. Define success criteria before you run the experiment.
Q012 - Question
A team builds an evaluation dataset from customer support tickets and live agent conversation logs. The logs contain customer names, email addresses, and phone numbers. They also contain few unusual requests and no prompt injection attempts. Which TWO actions should the team take before it runs evaluations in Microsoft Foundry?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Multiple choice
- A. Anonymize the personal data, for example by using Azure Language PII detection to identify and redact it.
- B. Keep the dataset to production logs only, because production data provides sufficient edge cases and adversarial examples.
- C. Add edge cases and adversarial examples that domain experts create manually to cover the missing scenario types.
- D. Make edge cases about 50% of the dataset so unusual scenarios are covered comprehensively.
A and C are correct.
Explanation: The evidence supports A and C because preparing evaluation data includes removing personal data such as names, email addresses, and phone numbers using tools like Azure Language PII detection. Furthermore, production data rarely includes enough edge cases or adversarial examples, which are best created manually using domain expertise.
B is incorrect: Production data provides authentic common scenarios and variations, but it rarely includes sufficient edge cases or adversarial examples.
D is incorrect: A balanced dataset uses about 5-10% edge cases, not 50%. Common scenarios should make up 60-70% to represent the quality baseline.
Q013 - Question
Adventure Works wants a GitHub Actions workflow that evaluates prompt changes before they merge. The workflow must authenticate to Microsoft Foundry without stored keys and post metrics as a pull request comment. Which configuration meets these requirements?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Trigger on pull_request events for the prompts and config paths. Grant id-token: write, contents: read, and pull-requests: write. Use azure/login with federated credentials.
- B. Trigger on push events to the main branch. Grant id-token: write and use azure/login with federated credentials, then post results after the merge.
- C. Trigger on pull_request events for the prompts and config paths. Grant only contents: read and use azure/login with federated credentials.
- D. Run the evaluation script locally for each change and paste the metrics into the pull request by hand.
A is correct.
Explanation: The evidence supports A because a pull_request trigger scoped to the prompt and configuration paths runs the evaluation before changes merge. The id-token: write permission supports federated identity credentials for keyless authentication with azure/login. The pull-requests: write permission lets the workflow post the metrics as a pull request comment.
B is incorrect: A push to main runs after the merge. The workflow must run evaluations on pull requests so the results can guide the merge decision. This configuration also does not grant pull-requests: write.
C is incorrect: The contents: read permission alone does not provide id-token: write for federated authentication or pull-requests: write for posting comments.
D is incorrect: A manual local run does not create an automated quality gate and does not use the workflow to post results to the pull request.
Q014 - Question
A team that builds an agent on pretrained models, prompts, and data connectors must keep governance, security, and compliance in place during exploration, building, and deployment. The team also wants to balance speed of delivery with strict adherence to standards. Which part of the GenAIOps lifecycle addresses this requirement?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. The Explore loop, which defines the business need and designs the architecture.
- B. The Build loop, which develops the application and evaluates it against quality and safety targets.
- C. The Operationalize loop, which deploys the application for real-world use.
- D. The management loop, which overarches the other loops and focuses on governance, security, and compliance.
D is correct.
Explanation: The management loop overarches the Explore, Build, and Operationalize phases. It focuses on governance, security, and compliance, and provides a framework that balances speed in deliverables with strict adherence to standards.
A is incorrect: The Explore loop is where you define the business need and design the architecture, not the overarching loop for governance and compliance.
B is incorrect: The Build loop is where you develop the initial application and evaluate it iteratively, not the overarching loop for governance and compliance.
C is incorrect: The Operationalize loop is where you deploy the application for real-world use, not the overarching loop for governance and compliance.
Q015 - Question
A Python script creates a cloud evaluation in Microsoft Foundry. The JSONL dataset uses the field name "Question" (capitalized). The groundedness evaluator uses the data mapping "query": "{{item.question}}". The evaluation run ends with a status of failed. Model quota and dataset access are confirmed to be correct. What should you do?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Increase the polling interval so the run has more time to complete.
- B. Remove "query" from the required fields in the data schema so validation passes.
- C. Make the field name in the data mapping match the dataset field exactly, or rename the dataset field to match the mapping.
- D. Replace the built-in groundedness evaluator with a different built-in evaluator.
C is correct.
Explanation: The evidence supports C because field names in data_mapping are case-sensitive and must match the JSONL dataset exactly. A mapping of {{item.question}} does not match a dataset field named "Question", causing the evaluation to fail.
A is incorrect: Polling only checks the run status. It does not change the result of a run that failed because of an invalid mapping.
B is incorrect: The schema defines which fields must be present. Evaluators still need their inputs, and the mismatch between the mapping and the dataset remains.
D is incorrect: Another built-in evaluator still needs data mapped from your dataset fields, so the mapping error would remain.
Q016 - Question
An outdoor-gear retailer builds a shopper agent. The product catalog changes constantly as items come and go. The company also wants the agent to answer in a consistent, warm brand voice that matches past support chats. Which design meets both requirements?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. Fine-tune the model on the current catalog and retrain it each time stock changes.
- B. Use a reasoning model such as o1 so that the agent handles catalog and tone requirements.
- C. Use Retrieval Augmented Generation (RAG) for current catalog data and fine-tune the model on past support chats for tone.
- D. Use only prompt engineering and keep the catalog inside the command prompt.
C is correct.
Explanation: Retrieval Augmented Generation (RAG) lets the agent search external data, such as a constantly changing product catalog, so it always pulls the latest data. Fine-tuning retrains the model on task-specific examples, such as past support chats, to lock in a consistent brand voice and tone.
A is incorrect: A fine-tuned model requires retraining every time stock changes, making it unsuitable for a constantly changing catalog.
B is incorrect: Reasoning models excel at math and complex logic. A chat completion model is better suited for generating natural, conversational text in a specific tone.
D is incorrect: Prompt engineering refines the command prompt, but the evidence recommends RAG for current data and fine-tuning for consistent tone, rather than keeping the catalog inside the prompt.
Q017 - Question
Several reviewers will judge the quality of a shopper support agent before launch. The team wants different reviewers to grade the same response the same way. The team also wants to compare the agent's answers with human support staff answers without risking a live customer. Which two actions should the team take? (Select two.)
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Multiple select
- A. Define a rubric with named criteria, each with a description of what a good answer looks like.
- B. Write a display prompt that shoppers can click as an example question.
- C. Rely only on smoke tests, because they are fast to run after every update.
- D. Use shadow rating to run the agent silently alongside human support staff.
A and D are correct.
Explanation: A rubric is a scoring guide with named criteria and a description of what a good answer looks like, ensuring different reviewers grade the same response the same way. Shadow rating runs the agent silently alongside human support staff, allowing you to compare answers without risking a live customer.
B is incorrect: A display prompt is an example question a shopper sees and can click. It does not help reviewers grade responses consistently.
C is incorrect: Smoke tests are short checks with known good answers used to catch obvious breakage, not tools for standardizing reviewer grades or comparing against human staff.
Q018 - Question
Adventure Works has validated that its automated evaluators align with human ratings. The team now wants to design the production phase of its human-in-the-loop (HITL) process for prompt updates. Evaluators must score every test case, and human reviewers must have a focused role. Which design meets these requirements?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Run automated evaluation occasionally and use human evaluation at other times, with no shared workflow.
- B. Run the automated evaluation on every prompt update, send responses that score below the threshold plus a random 10% sample to human reviewers, and run monthly correlation checks.
- C. Have human reviewers score all 500 test cases for every prompt update and use automated scores only as a reference.
- D. Remove human reviewers from the process after validation and rely only on the automated gate.
B is correct.
Explanation: The evidence supports B because in the production phase, automation scores everything as an automated gate. Humans review responses that score below the threshold and a random 10% sample. Monthly correlation checks detect evaluator drift.
A is incorrect: HITL is a system where automation provides comprehensive coverage and humans provide strategic oversight, not running them separately without a shared workflow.
C is incorrect: Reviewing all 500 test cases manually takes days and reduces consistency as reviewers tire, failing to use automation for volume.
D is incorrect: Automated evaluation lacks human context and requires validation. Without human review and drift checks, you cannot detect misalignment as the system changes.
Q019 - Question
A retail team runs a full automated evaluation on its support agent. The agent scores high, but the test dataset contains only the questions that shoppers ask most often. The team wants scores it can trust before launch. What should the team do?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. Run the smoke tests after every small update and use those results as the launch score.
- B. Add rare edge cases to the dataset, such as ambiguous requests, out-of-stock items, and questions the agent should refuse.
- C. Remove the questions that the agent should refuse so that the dataset stays focused on common use.
- D. Replace the full automated evaluation with a rubric that has no test data.
B is correct.
Explanation: The evidence states that to trust evaluation scores, you must use a deliberate coverage mix. This means choosing test questions that reflect the full range of real use, including common questions and rare edge cases such as ambiguous requests, out-of-stock items, and questions the agent should refuse.
A is incorrect: Smoke tests are short, quick checks run after small updates to catch obvious breakage, not full automated evaluations that score the agent at scale.
C is incorrect: Questions the agent should refuse are a necessary part of the coverage mix. Removing them hides failures that only show up in those situations.
D is incorrect: A rubric is a scoring guide, but you still need test data to run a full automated evaluation and prove the agent works.
Q020 - Question
A team manages a customer-facing agent. They want to test GPT-4 mini and also a shorter system prompt. They need to know which change affects quality, cost, and response time. Which experiment design should they use?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Create one variant that uses GPT-4 mini and the shorter prompt together, and compare it to the baseline.
- B. Test GPT-4 mini only on the easiest test prompts to confirm that it responds faster.
- C. Run the baseline and a GPT-4 mini variant with the same prompt against the same test prompts. Test the shorter prompt as a separate variant.
- D. Run each variant against a different set of test prompts so that each variant is tested on new scenarios.
C is correct.
Explanation: Change one variable at a time so you can attribute any difference to a single modification. Running every variant against the same test prompts keeps the comparison controlled. After you validate each change, you can test combinations of the successful variants.
A is incorrect: Changing the model and the prompt together makes it impossible to tell which change caused a difference in quality, cost, or response time.
B is incorrect: Test prompts must cover diverse use cases and edge cases. Testing only easy prompts hides weaknesses on complex trip planning queries.
D is incorrect: Different test prompts per variant remove the shared basis for comparison. Each variant must run against the same test prompts.
Q021 - Question
A shopper agent is live on a website. The team wants to see which tools the agent called for each request, track latency, token usage, and error rates, and detect when answer quality drifts from the rubric. Which approach meets these requirements?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. Use tracing, collect the traces and metrics in Azure Monitor and Application Insights, and run continuous evaluation that re-scores live answers against the same rubric.
- B. Use the Azure Developer CLI (azd) to reprovision the environment from the AI app template after each deployment.
- C. Test shopper questions in the Chat playground in the Foundry portal and compare how natural the answers sound.
- D. Fine-tune the model on past support chats to keep the brand tone consistent.
A is correct.
Explanation: Tracing records each request, including which tools the agent called. Azure Monitor and Application Insights collect those traces alongside metrics like latency, token usage, and error rates. Continuous evaluation re-scores live answers against the rubric to detect if quality drifts.
B is incorrect: The Azure Developer CLI (azd) provisions and deploys the environment. It does not track live metrics, traces, or answer quality.
C is incorrect: The Chat playground in the Foundry portal is used to explore models before committing, not to monitor a live agent in production.
D is incorrect: Fine-tuning trains the model on examples to lock in a tone. It does not provide tracing, metrics, or continuous evaluation of live traffic.
Q022 - Question
Three evaluators independently score 10 agent responses on three metrics, which gives 30 scoring opportunities. The scores are within one point of each other in 21 of the 30 cases. The team's reliability target is 80% agreement within one point. What should the team do?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Review the responses with divergent scores, update the rubrics to clarify those cases, and conduct additional calibration training.
- B. Accept the scores, because 70% agreement is sufficient for optimization decisions.
- C. Average the three scores for each response and use the averages to compare variants.
- D. Replace the rubrics with a single overall quality score to reduce disagreement.
A is correct.
Explanation: 21 of 30 is 70% agreement within one point. This is below the 80% target. Review the divergent responses to find what caused the disagreement, clarify the rubrics, and conduct more calibration training.
B is incorrect: 70% is below the 80% target. Accepting the scores means optimization decisions rest on individual evaluator preferences.
C is incorrect: Averaging scores does not fix the different interpretations of the criteria. It hides the inconsistency instead of correcting it.
D is incorrect: A single overall score removes the defined score levels and examples that create a shared understanding. Rubrics with concrete examples at each level reduce ambiguity.
Q023 - Question
During an alignment investigation, the correlation between human scores and Intent Resolution evaluator scores is low. You calculate Cohen's kappa for two human evaluators who scored the same examples, and the result is 0.35. What should you do next?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Create a custom evaluator immediately to replace the built-in Intent Resolution evaluator.
- B. Lower the correlation threshold so the automated gate can continue to run.
- C. Add more adversarial examples to the dataset and rerun the shadow rating with the same human rubric.
- D. Resolve the inconsistency between human evaluators by training them and clarifying the rubric before you adjust the automated evaluators.
D is correct.
Explanation: The evidence supports D because a kappa value below 0.4 indicates poor agreement between human evaluators. You must resolve human inconsistency first by investing in evaluator training and clearer rubrics. If you automate inconsistent criteria, you scale the inconsistency.
A is incorrect: Custom evaluators are for cases where built-in evaluators miss domain-specific dimensions, but you must first confirm that the human scores are consistent.
B is incorrect: Lowering the threshold hides the misalignment and does not address its cause, leading to trust in automated scores that do not reflect consistent human judgment.
C is incorrect: Adding adversarial examples changes dataset coverage but does not fix the human disagreement. The same unclear rubric would still produce inconsistent human scores.
Q024 - Question
A team is preparing a calibration exercise for new human evaluators of a Trail Guide Agent. They have a rubric with defined score levels. How should they build and run the exercise?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Select 20 high-scoring responses and show the evaluators the intended scores before they score.
- B. Select five to eight responses that span the score range, including ambiguous ones. Have each evaluator score them independently, then compare scores and discuss differences.
- C. Have the evaluators score each response together as a group so that they reach one score immediately.
- D. Use the scores of the most experienced evaluator as the answer key and skip the group discussion.
B is correct.
Explanation: A calibration set of five to eight responses should include clear examples of low and high scores plus ambiguous responses between levels. Independent scoring shows how each evaluator interprets the rubric. Discussing scores that differ by more than one point identifies where to clarify the rubric.
A is incorrect: High-scoring responses only do not span the score range, and showing the intended scores in advance prevents you from seeing how evaluators interpret the rubric.
C is incorrect: Group scoring hides individual interpretation differences. Evaluators must score independently first.
D is incorrect: A single evaluator's scores do not reveal where the rubric is unclear. Discussion of differing scores is needed to update the rubric descriptions.
Q025 - Question
A team tests a smaller model on the branch experiment/gpt4o-mini-model. The evaluation.csv file shows an average quality score of 4.1. The success threshold is 4.2. The team has committed the results. What should they do next?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Merge the branch to main because the quality score is close to the threshold.
- B. Merge the branch to main and tag it, then revert the merge if customers report problems.
- C. Delete the branch without recording why, because the experiment failed.
- D. Document that quality fell below the threshold, then decide whether to keep the branch as a record or delete it.
D is correct.
Explanation: The experiment does not meet the quality threshold, so it is not promoted to main. Document why it failed before you decide what to do with the branch. You can keep the branch to show future teams what did not work, or delete it if the results are already committed and documented.
A is incorrect: A score of 4.1 is below the 4.2 threshold. Only experiments that meet the success criteria are merged to main.
B is incorrect: Merging a variant that does not meet the threshold promotes it to production without evidence that it is acceptable.
C is incorrect: Deleting the branch is acceptable only when the evaluation results are already committed and documented. The reason for the failure must be recorded.
Q026 - Question
A team has a model running in an Azure Machine Learning online endpoint for real-time scoring. They trained a new version and want to send a small share of requests to it. They also want to shift more traffic to the new version later by changing configuration, without creating a new endpoint. What should they do?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Register the new model with the same name as the existing model and rely on the incremented version number to replace the running deployment.
- B. Create a second deployment in the same online endpoint and update the traffic percentage in the endpoint configuration.
- C. Deploy the new model to a batch endpoint and compare its scores with the online endpoint results.
- D. Convert the new model to Open Neural Network Exchange (ONNX) and redeploy it over the existing deployment.
B is correct.
Explanation: Controlled rollout for online endpoints lets you create multiple deployments in one endpoint, route traffic to different deployments for A/B testing, and switch between deployments by updating the traffic percentage in the endpoint configuration.
A is incorrect: Registering a model with the same name only increments the version number in the registry. It does not split traffic between deployments.
C is incorrect: Batch endpoints support batch scoring. They do not provide real-time traffic splitting for an online endpoint.
D is incorrect: Converting to ONNX can change model performance, but it does not provide traffic routing, and redeploying over the existing deployment removes the ability to compare two deployments.
Q027 - Question
A group of data scientists must register new models and delete outdated models in an Azure Machine Learning registry. They must also use the assets in workspaces. They must not be able to create, update, or delete registries. Which role assignment meets the requirement?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. The built-in Reader role.
- B. The built-in Contributor role.
- C. A custom role that has registries/read, registries/assets/read, registries/assets/write, and registries/assets/delete permissions.
- D. The built-in Owner role.
C is correct.
Explanation: To let users read, create, and delete assets, grant the read permissions plus the assets/write and assets/delete permissions. A custom role that includes only these permissions does not include registries/write or registries/delete, so the users can't create, update, or delete registries.
A is incorrect: The built-in Reader role lets users only read and use assets. It does not allow them to create or delete assets.
B is incorrect: The built-in Contributor role allows users to create, update, and delete registries. This does not meet the restriction.
D is incorrect: The built-in Owner role also allows users to create, update, and delete registries. This does not meet the restriction.
Q028 - Question
Your team trains a model that needs a specific set of Python packages and environment variables. You must run the same training job on a compute instance and on a compute cluster. The package set must be versioned in the Azure Machine Learning workspace, and the setup must be reproducible and auditable. What should you create?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. A datastore that stores the list of required packages.
- B. A custom environment that uses a base Docker image and a Conda YAML file.
- C. A curated environment that you use as is, with the extra packages installed at run time.
- D. A component for each required package.
B is correct.
Explanation: An environment specifies the software packages, environment variables, and software settings for training and scoring scripts. Environments are managed and versioned entities in the workspace, and they support reproducible, auditable, and portable workflows across computes. A custom environment can use a base Docker image with a Conda YAML file for further customization.
A is incorrect: A datastore keeps the connection information for a storage service. It does not define packages or software settings for a job.
C is incorrect: Curated environments are intended to be used as is. Use a custom environment when you must control the packages and dependencies.
D is incorrect: A component is a self-contained piece of code that completes one step in a pipeline. It does not define the software environment for a job.
Q029 - Question
A platform team deploys a Foundry resource by using a Bicep template that sets agent capability settings. The template references an existing Azure Storage account, an existing Azure Cosmos DB account, and an existing Azure AI Search resource. The team wants to avoid a deployment that fails partway through because of missing permissions. Which two role assignments must the deployment principal hold before the deployment? Select two.
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Multiple choice
- A. A caller role on the Azure AI Search resource.
- B. Storage Blob Data Contributor on the referenced Azure Storage account.
- C. Data-plane roles for the project managed identity on each referenced resource.
- D. Cosmos DB Operator on the referenced Azure Cosmos DB account.
B and D are correct.
Explanation: When a template sets agent capability settings, the deployment principal needs Storage Blob Data Contributor on every referenced Azure Storage account. It also needs Cosmos DB Operator on every referenced Azure Cosmos DB account. A missing assignment fails the deployment partway through. Run az role assignment list to check the principal's current assignments before you deploy.
A is incorrect: Azure AI Search doesn't require a caller role for capability settings provisioning. The deployment principal still needs permission to create or update every resource that the template declares.
C is incorrect: Provisioning permissions and runtime permissions are separate. Grant the project managed identity its data-plane roles after the deployment succeeds, and then confirm that an agent can read and write.
Q030 - Question
A company plans a production generative AI application that uses a Foundry Model sold by Azure. Compliance requires inference data to be processed within a geographic data zone. The company also requires a customizable content filtering configuration. Which deployment option and type should the company use?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. A managed compute deployment on the Foundry project.
- B. A managed compute deployment that uses the company's existing Azure VM quota.
- C. A Serverless API deployment with the Global Standard deployment type.
- D. A Serverless API deployment with the Data Zone Standard deployment type.
D is correct.
Explanation: Serverless API deployments support regional, data zone, and global data processing. They also provide built-in and customizable content filtering. The Data Zone Standard deployment type keeps processing in a data zone (US, EU, or APAC), which meets the compliance requirement.
A is incorrect: Managed compute supports only global data processing. Content filtering isn't available in the public preview, and the preview isn't recommended for production workloads.
B is incorrect: Existing Azure VM quota can't be applied to a managed compute deployment. This option also has the same global-only processing and content filtering limits.
C is incorrect: Global Standard uses global processing, which doesn't meet the requirement to process data within a geographic data zone.
Q031 - Question
A company deploys a classification model to an Azure Machine Learning batch endpoint. The team wants to use Azure Machine Learning model monitoring to track data drift on the model inputs. Which action is required to support this?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Enable Azure Machine Learning model data collection on the batch endpoint so that production inference data is collected automatically.
- B. Use the AllowOnlyApprovedOutbound managed virtual network isolation setting so that monitoring jobs can reach the data.
- C. Collect the production inference data yourself and provide it to model monitoring.
- D. Set the reference data to the production data window so that no training data is needed.
C is correct.
Explanation: When you deploy a model to a batch endpoint or outside Azure Machine Learning, you are responsible for collecting production inference data. You can then use that data for model monitoring.
A is incorrect: Azure Machine Learning model data collection is the option for models deployed to online endpoints. For batch endpoints, you collect the data yourself.
B is incorrect: Model monitoring does not support the AllowOnlyApprovedOutbound managed virtual network isolation setting.
D is incorrect: The reference data window and the production data window should not overlap. Reference data can be training data or recent past production data, but it is still a separate comparison baseline.
Q032 - Question
A team runs a latency-sensitive production copilot with predictable traffic on a provisioned deployment. The team wants the discounted Azure Reservations rate for the PTU billing meter. The team has PTU quota in the target region. What should the team do?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. Create the provisioned deployment first to confirm that capacity is available, and then purchase the reservation.
- B. Purchase the reservation first, because a reservation guarantees capacity for the deployment.
- C. Delete and re-create the deployment with each traffic change to stay on hourly billing.
- D. Skip the capacity check, because PTU quota guarantees that capacity is available.
A is correct.
Explanation: Reservations are a discount on the PTU billing meter and don't guarantee capacity. Create the deployment first to confirm that capacity is available, and then purchase the reservation to lock in the discounted rate.
B is incorrect: A reservation doesn't guarantee capacity. Reservations and deployments are created independently, so you can buy a reservation and still be unable to deploy.
C is incorrect: Capacity might not be available when you need to scale back up. Continuous hourly billing at high utilization typically exceeds reservation pricing.
D is incorrect: PTU quota is a policy limit and doesn't guarantee capacity. If capacity in the region is insufficient for the requested PTU count, the deployment fails.
Q033 - Question
You evaluate an agent that calls tools. You need to detect behaviors that violate a policy of prohibited actions and detect exposure of sensitive information such as financial data or personal identifiers. You evaluate an agent target and can provide tool-call information. Which two evaluators should you configure? Select two.
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Multiple choice
- A. builtin.violence
- B. builtin.prohibited_actions
- C. builtin.indirect_attack
- D. builtin.sensitive_data_leakage
B and D are correct.
Explanation: builtin.prohibited_actions measures whether an agent engages in behaviors that violate explicitly disallowed actions or tool uses. builtin.sensitive_data_leakage measures an agent's vulnerability to exposing sensitive information. Both are in preview, support agent targets only, and take tool-call information through the tool_calls field or a messages array.
A is incorrect: builtin.violence measures violent or threatening language. It does not evaluate prohibited tool use or sensitive data exposure.
C is incorrect: builtin.indirect_attack measures whether a response fell for an indirect jailbreak attempt, and it applies to models only. It does not evaluate prohibited actions or sensitive data leakage.
Q034 - Question
You create an Azure Machine Learning workspace that uses a private endpoint. The workspace storage account and container registry are behind private endpoints. Docker image builds for environments must work. You use the Azure CLI with a YAML file to create the workspace. What should you configure?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Create a CPU compute cluster and set the image_build_compute property in the workspace YAML to its name.
- B. Rely on Azure Container Registry tasks compute to build the images.
- C. Set public_network_access to Enabled so that the default image build works.
- D. Remove the private endpoint from the workspace after you create it.
A is correct.
Explanation: When a workspace uses a private link, it can't use Azure Container Registry tasks compute for image building. By default, it uses a serverless compute cluster, but that works only when the dependent resources have no network restrictions. Because the storage account and container registry are behind private endpoints, set image_build_compute to a compute cluster name. Create that compute before you build images.
B is incorrect: A workspace that uses a private link can't use Azure Container Registry tasks compute for image building.
C is incorrect: The public_network_access property controls whether the workspace is reachable over the internet. It does not change how images are built when the dependent resources are under network restrictions.
D is incorrect: Removing the private endpoint removes the workspace from the virtual network and doesn't make it publicly accessible. It also defeats the purpose of the private endpoint.
Q035 - Question
Your team deployed a generative AI application to production. You want to detect system drift by running quality and safety evaluations against the same fixed test dataset at regular intervals. Which post-production monitoring approach should you use?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Continuous evaluation of production traffic at a sampled rate
- B. Scheduled evaluation that uses test datasets
- C. Scheduled red teaming
- D. Base model selection by using the Microsoft Foundry benchmark
B is correct.
Explanation: Scheduled evaluation runs quality and safety evaluations on a schedule by using test datasets. Use it to detect system drift, because the same dataset is evaluated each time.
A is incorrect: Continuous evaluation assesses production traffic at a sampled rate. It does not run a fixed test dataset at regular intervals.
C is incorrect: Scheduled red teaming runs adversarial tests to probe for safety and security vulnerabilities. It does not measure drift in quality and safety scores on a test dataset.
D is incorrect: The Microsoft Foundry benchmark compares models during base model selection. It is not a post-production monitoring approach.
Q036 - Question
Your organization has Azure Machine Learning workspaces in East US today. It plans to add workspaces in West US soon. Teams must register models, environments, and components once and use them from all workspaces. You use the Azure CLI to create the registry. Which design should you use?
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Single choice
- A. Register the online endpoint in the registry so that its scoring URI works from both workspaces.
- B. Create the registry with only East US, and change the primary region to West US when the new workspaces exist.
- C. Create a separate registry in each workspace and copy assets between them when needed.
- D. Create one registry with eastus as the primary location and include eastus and westus in the replication locations.
D is correct.
Explanation: A registry lets you create and use assets such as models, environments, and components across workspaces, including workspaces in different regions. You define a primary region and a set of other regions when you create the registry. List the regions where you have workspaces today and plan to add in the near future. The primary location appears in the location item and in the replication_locations list.
A is incorrect: Endpoints are resources, not assets. Resources are workspace specific, and an online endpoint has a scoring URI that is unique to one instance in one workspace.
B is incorrect: You can't change the primary region after you create the registry. You can update only the other regions.
C is incorrect: Registries exist to store assets in a central repository and share them across workspaces. A registry is not created inside a workspace, and copying assets between registries adds work that a single registry avoids.
Q037 - Question
A multi-step agent built with LangGraph sometimes responds slowly. Developers need to see each LLM call, tool invocation, and agent decision, and identify which step causes the delay. The data must be available in Azure Monitor Application Insights. Which Microsoft Foundry capability should you configure?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Built-in evaluators for coherence and fluency
- B. Azure Monitor alerts on output quality thresholds
- C. Distributed tracing that uses OpenTelemetry standards
- D. The AI red teaming agent
C is correct.
Explanation: Distributed tracing captures the execution flow of AI applications, including LLM calls, tool invocations, agent decisions, and inter-service dependencies. It is built on OpenTelemetry standards, integrates with Azure Monitor Application Insights, and supports LangGraph. Use it to find performance bottlenecks in multi-step agent behavior.
A is incorrect: Coherence and fluency evaluators measure response quality. They do not show the execution flow or where latency occurs.
B is incorrect: Azure Monitor alerts send notifications when outputs fail quality thresholds or produce harmful content. They do not show individual steps in the agent execution.
D is incorrect: The AI red teaming agent simulates attacks to find safety and security vulnerabilities before deployment. It does not diagnose performance bottlenecks.
Q038 - Question
A regulated workload requires that training data for a Microsoft Foundry fine-tuning job stays in the region of the current Foundry resource. The selected base model supports Standard-tier training. Which training type should you select?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Standard
- B. Global
- C. Developer
- D. Any tier, because the training type doesn't affect where training occurs
A is correct.
Explanation: Standard training occurs in the current Foundry resource's region and provides guarantees for data residency. It is the right choice when data must remain in a specific region.
B is incorrect: Global training is more affordable and has faster queue times because it uses capacity beyond your current region. Data and weights are copied to the region where training occurs, which conflicts with a residency requirement.
C is incorrect: Developer training uses idle capacity and has no data residency guarantees. It also has no latency or SLA guarantees, and jobs can be preempted and resumed later.
D is incorrect: The training type determines where training occurs and whether residency is guaranteed. Only Standard provides the residency guarantee.
Q039 - Question
You are building a RAG solution over proprietary documents. The documents contain many organization-specific terms that are unlikely to be in a general embedding model's vocabulary. No domain-specific embedding model is available, and top-ranked general models return poor vector search results on your content. What should you do?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Select the largest available embedding model, because larger models always perform better
- B. Apply dimensionality reduction to the embeddings to correct the vocabulary mismatch
- C. Fine-tune a general embedding model with your domain vocabulary, and evaluate it by running vector searches with real queries and content
- D. Select the top model on the Hugging Face leaderboard and skip testing on your own data
C is correct.
Explanation: When a model's vocabulary does not cover your terms, it can split unknown words into subwords whose meanings are far from the original word. This produces poorer vector matches. If no domain-specific model exists or it does not perform well, fine-tune a general embedding model with your domain vocabulary. Then evaluate retrieval performance with real-world queries and content. Fine-tuning can degrade performance if the training data is poor, so evaluation is required.
A is incorrect: Larger models are not always better. They can increase cost, including vector storage and compute, without improving performance on your data.
B is incorrect: Dimensionality reduction is a post-processing step that you apply after you generate embeddings to optimize storage and retrieval. It does not add missing vocabulary to the model.
D is incorrect: Leaderboard rankings are a starting point. Public benchmarks often use academic datasets and might not apply to your business data, so you must validate the model through experimentation.
Q040 - Question
A training job produces a model by using features from a managed feature store. The team will register the model in an Azure Machine Learning workspace. They need the workspace to show lineage between the model and the feature sets. They also need the online scoring script to look up feature values at inference time. How should the training job package the feature retrieval specification?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Save it in a subfolder of the model artifact and rename it to match the model name.
- B. Register it only as a separate data asset and have the scoring script load it from there.
- C. Omit it, because the get_offline_features() API accepts a list of features and no specification is needed.
- D. Place feature_retrieval_spec.yaml in the root folder of the model artifact and keep the file name unchanged.
D is correct.
Explanation: The feature retrieval specification must be in the root folder of the model artifact, and its file name can't be changed. Lineage between the model and the feature sets is tracked only if the specification exists in the model artifact. The scoring code also loads the feature list from the specification in the model artifact folder before it looks up features from the online store.
A is incorrect: The specification must be in the root folder, and its file name can't be changed.
B is incorrect: The scoring script loads the specification from the model artifact folder. A separate data asset does not provide model-to-feature-set lineage.
C is incorrect: The specification is optional only for generating training data with get_offline_features(). It is still required in the model artifact for lineage tracking and online inference.
Q041 - Question
You plan an evaluation run that scores entire multi-turn conversations. You set evaluation_level to conversation. One of the evaluators you selected supports only the turn level. You need the run to be valid. What should you do?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Keep all evaluators in one run. Each evaluator then uses its own supported level.
- B. Omit evaluation_level so that each evaluator uses the level it supports.
- C. Set evaluation_level to turn so that conversation-level scoring is applied automatically.
- D. Check supported_evaluation_levels in the evaluator catalog. Include only evaluators that support conversation in this run.
D is correct.
Explanation: Each evaluator supports specific evaluation levels, which are listed in the supported_evaluation_levels field in the evaluator catalog. All evaluators in a run must support the evaluation_level that you set. Verify the supported levels and remove evaluators that don't support conversation from the conversation-level run.
A is incorrect: You can't mix evaluators with incompatible levels in the same evaluation run. The run does not apply a different level per evaluator.
B is incorrect: If you omit evaluation_level, the default is turn. The run then evaluates individual responses, not entire conversations.
C is incorrect: Setting evaluation_level to turn evaluates individual agent responses. It does not score the full interaction.
Q042 - Question
A security team must restrict outbound traffic from Foundry Agents to a curated list of destinations to reduce the risk of data exfiltration. The list includes some fully qualified domain names (FQDNs). The team doesn't want to build or maintain its own virtual network. Which configuration should the team use?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. A managed virtual network in Allow internet outbound mode.
- B. A managed virtual network in Disabled mode.
- C. A managed virtual network in Allow only approved outbound mode with outbound rules.
- D. A custom virtual network with a customer-provided Azure Firewall.
C is correct.
Explanation: Allow only approved outbound mode restricts outbound traffic by using service tags, private endpoints, and optional FQDN rules. Microsoft provisions and manages the network, so the team doesn't build its own virtual network. When you add an FQDN rule, a managed Azure Firewall is created. FQDN rules support only ports 80 and 443, and the firewall adds cost.
A is incorrect: Allow internet outbound mode allows all outbound traffic to the internet. It doesn't limit destinations to a curated list.
B is incorrect: Disabled means managed virtual network isolation isn't enabled, unless you use a custom virtual network. It doesn't provide a managed network boundary.
D is incorrect: A custom virtual network requires the team to set up and maintain the network, such as subnet delegation. This conflicts with the requirement to avoid building and maintaining a virtual network.
Q043 - Question
You are monitoring a supervised fine-tuning job in Microsoft Foundry. Training loss keeps decreasing, but the validation loss and validation token accuracy diverge from the training metrics. This indicates possible overfitting. Which two changes should you try in the next fine-tuning job?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Multi-select
- A. Increase n_epochs so the model sees the training data more times
- B. Train with fewer epochs
- C. Increase learning_rate_multiplier above the recommended 0.02 to 0.2 range
- D. Use a smaller learning_rate_multiplier
B and D are correct.
Explanation: When training and validation metrics diverge, the model might be overfitting. Train with fewer epochs or use a smaller learning rate multiplier. A smaller learning rate can help avoid overfitting. You can also review checkpoints, which might provide snapshots from before overfitting.
A is incorrect: More epochs means more full passes through the training data. This can increase overfitting when validation metrics already diverge.
C is incorrect: Larger learning rates do not address overfitting. The recommended range to experiment with is 0.02 to 0.2, and a smaller value is what helps avoid overfitting.
Q044 - Question
A team trains a deep neural network in Azure Machine Learning. The training data is large, and the full model fits in the memory of each node in the compute cluster. The team wants the approach that is easiest to implement and sufficient for most use cases. Which approach should they use?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Single choice
- A. Use data parallelism. Divide the data into partitions, copy the model to each worker node, and synchronize the model parameters or gradients at the end of the batch computation.
- B. Use model parallelism because it is easier to implement than data parallelism.
- C. Use data parallelism, but let each worker node keep its own model without synchronizing parameters.
- D. Use data parallelism on nodes that are too small to hold the full model, because each node only needs part of the model.
A is correct.
Explanation: Data parallelism is the easiest distributed training approach to implement and is sufficient for most use cases. It partitions the data across worker nodes and copies the model to each node. Each node must be able to hold the entire model. Nodes synchronize parameters or gradients at the end of the batch computation so that they train a consistent model.
B is incorrect: Model parallelism is more complex to implement than data parallelism.
C is incorrect: Worker nodes must synchronize the model parameters or gradients so that they train a consistent model.
D is incorrect: With data parallelism, the entire model has to fit on each node. Splitting the model across nodes is model parallelism.
Q045 - Question
You deploy a fraud classification model to an Azure Machine Learning online endpoint and use the data collector to collect production inference data. You want to monitor accuracy, precision, and recall in production. Which TWO actions should you take?
Domain: Implement machine learning model lifecycle and operations (25–30%) Type: Multi-select
- A. Collect ground truth data as it becomes available, and maintain it as a data asset that has a unique ID per row matching the IDs in the model output data.
- B. Use recent past production data as the reference data for the model performance signal.
- C. Create an Event Grid event subscription that uses the Dataset drift detected event type to receive model monitoring events.
- D. Schedule the monitoring job on a serverless Spark compute pool that uses a supported instance type, such as Standard_E4s_v3.
A and D are correct.
Explanation: Model performance monitoring requires ground truth data with a unique ID for each row. The ID must match the ID in the model output data so the two data assets can be joined. Monitoring jobs run on serverless Spark compute pools, and Standard_E4s_v3 is a supported instance type.
B is incorrect: The model performance signal requires ground truth data as the reference data. Recent past production data is not a valid reference for this signal.
C is incorrect: Dataset drift detected applies to data drift v1, not Azure Machine Learning model monitoring. Select only Run status changed for model monitoring events.
Q046 - Question
Your team is adding grounding to an internal assistant. The team already has orchestration code that sends a single query to Azure AI Search and passes the results to a deployed LLM. Policy allows only generally available (GA) features in production. Simplicity and response speed matter more than advanced relevance. Which approach should you use?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Agentic retrieval with a knowledge base and LLM-based query planning
- B. Classic RAG with hybrid queries and optional semantic ranking
- C. Send all documentation to the LLM with each request and rely on its context window
- D. Keyword-only queries with no vector search and no semantic ranking
B is correct.
Explanation: Classic RAG fits this scenario. Your application sends a single query to Azure AI Search and orchestrates the handoff to the LLM separately. This approach uses GA features, is simpler, is faster because the LLM does not take part in query planning, and preserves your existing orchestration code. Hybrid queries combine keyword and vector search for better recall, and semantic ranking is optional.
A is incorrect: Agentic retrieval uses LLM-based query planning, which is in preview. It does not meet the GA-only requirement and adds components that your simplicity and speed priorities do not need.
C is incorrect: LLMs accept a limited number of tokens. Sending everything wastes tokens and can degrade quality. Retrieval should return relevant, concise results.
D is incorrect: Keyword search fails when queries do not match document terminology. Classic RAG improves recall by combining keyword and vector search, so keyword-only queries give up that benefit.
Q047 - Question
You plan to use the synthetic data generation preview in Microsoft Foundry to create Simple Q&A training data from a domain policy document. This is your first generation job for this document. How should you proceed?
Domain: Optimize generative AI systems and model performance (10–15%) Type: Single choice
- A. Generate the maximum of 1,000 samples first, so you only run the job once
- B. Add real customer records to the reference file so the generated examples look realistic
- C. Skip review and use the dataset directly in fine-tuning, because the output is already formatted as JSONL
- D. Generate a small sample first, review the outputs, and adjust the reference file or generation parameters before scaling up
D is correct.
Explanation: Start with a smaller sample size and review the outputs. Then adjust your reference file or generation parameters before you scale up. This helps you avoid unnecessary costs and confirms that the data meets your requirements. The sample count can be between 50 and 1,000. Because this is a preview, always validate outputs before production use.
A is incorrect: Generating 1,000 samples first can increase cost, and you would not know whether the reference file produces good data until after the full job completes.
B is incorrect: Do not include personally identifiable information (PII) or other sensitive data in the reference file. Keep the file clean and focused on the target domain.
C is incorrect: Although the output is formatted as JSONL files compatible with Foundry fine-tuning, the quality depends on the reference file. Preview functionality can change, so you must review and validate the data before use.
Q048 - Question
A service principal in a partner application must call the endpoints of one agent in a Foundry project. The project contains several other agents. The service principal must not build or modify agents, and it must not reach the endpoints of the other agents. Which role assignment meets the requirements with the least privilege?
Domain: Design and implement a GenAIOps infrastructure (20–25%) Type: Single choice
- A. Assign Foundry User to the service principal at the Foundry project scope.
- B. Assign Foundry Agent Consumer to the service principal at the agent scope by using Azure CLI.
- C. Assign Foundry Project Manager to the service principal at the Foundry resource scope.
- D. Assign Azure AI Developer to the service principal at the Foundry project scope.
B is correct.
Explanation: Foundry Agent Consumer is the least-privilege role for principals that only interact with agents. An agent-scope assignment grants endpoint access to the specified agent only. The Azure portal supports assigning this role only at the Foundry account scope, so use Azure CLI to assign it at the agent scope.
A is incorrect: Foundry User includes build and develop data actions, which the service principal doesn't need. At project scope, it also isn't limited to one agent.
C is incorrect: Foundry Project Manager can manage projects, publish agents, and assign the Foundry User role. This role is broader than the requirement, and the resource scope is wider than one agent.
D is incorrect: Azure AI Developer is scoped to Azure Machine Learning workspaces and Foundry hubs, not to Foundry projects. Use Foundry roles for Foundry project access.
Q049 - Question
You are writing a Bicep template for a Microsoft.MachineLearningServices/workspaces resource that uses API version 2025-12-01. You must plan for properties that are hard to change after deployment. Select two statements that are correct.
Domain: Design and implement an MLOps infrastructure (15–20%) Type: Multiple choice
- A. You can change the keyVault property after the workspace is created to move to a different key vault.
- B. You can't change the storageAccount property after the workspace is created.
- C. After you switch managedNetworkKind from V1 to V2, you can switch back to V1 if needed.
- D. You can switch managedNetworkKind from V1 to V2 to get granular access controls.
B and D are correct.
Explanation: The storageAccount property holds the ARM ID of the associated storage account and can't be changed once the workspace is created, so choose the account before you deploy. The managedNetworkKind property lets you switch from V1 to V2 for granular access controls.
A is incorrect: The keyVault property holds the ARM ID of the key vault associated with the workspace. It can't be changed once the workspace is created.
C is incorrect: After V2 is enabled for managedNetworkKind, you can't switch back to V1. Plan the change before you make it.
Q050 - Question
You evaluate a retrieval-augmented generation (RAG) application. You need a binary pass/fail result that shows whether responses are grounded in the retrieved context. You do not want to provide a model deployment for the evaluator. You also accept that the evaluator is in preview. Which built-in evaluator should you use?
Domain: Implement generative AI quality assurance and observability (10–15%) Type: Single choice
- A. Groundedness Pro (preview)
- B. Groundedness
- C. Relevance
- D. Document Retrieval
A is correct.
Explanation: Groundedness Pro measures whether the response is grounded in the retrieved context by using the Azure AI Content Safety service. It returns a binary pass/fail and does not require a model deployment. It is in preview, and Microsoft doesn't recommend preview features for production workloads.
B is incorrect: Groundedness uses a model-based judgment and returns a score from 1 to 5. It does not return a binary result.
C is incorrect: Relevance measures how relevant the response is to the query. It does not measure grounding in the retrieved context.
D is incorrect: Document Retrieval measures the accuracy of retrieval results given ground truth. It does not evaluate whether the response is grounded.