Operationalize machine learning and generative AI solutions

Outline

• Experiment with Azure Machine Learning
  • Preprocess data and configure featurization
  • Run an automated machine learning experiment
  • Evaluate and compare models
  • Configure MLflow for model tracking in notebooks
  • Train and track models in notebooks
  • Evaluate models with the Responsible AI dashboard
  • Exercise - Find the best classification model with Azure Machine Learning
  • Module assessment
• Perform hyperparameter tuning with Azure Machine Learning
  • Define a search space
  • Configure a sampling method
  • Configure early termination
  • Use a sweep job for hyperparameter tuning
  • Exercise - Run a sweep job
  • Module assessment
• Run pipelines in Azure Machine Learning
  • Create components
  • Create a pipeline
  • Run a pipeline job
  • Exercise - Run a pipeline job
  • Module assessment
• Trigger Azure Machine Learning jobs with GitHub Actions
  • Understand the business problem
  • Explore the solution architecture
  • Use GitHub Actions for model training
  • Exercise
  • Module assessment
• Trigger GitHub Actions with feature-based development
  • Understand the business problem
  • Explore the solution architecture
  • Trigger a workflow
  • Exercise
  • Module assessment
• Work with environments in GitHub Actions
  • Understand the business problem
  • Explore the solution architecture
  • Set up environments
  • Exercise
  • Module assessment
• Deploy a model with GitHub Actions
  • Understand the business problem
  • Explore the solution architecture
  • Model deployment
  • Exercise
  • Module assessment
• Plan and prepare a GenAIOps solution
  • Explore use cases for GenAIOps
  • Select the right generative AI model
  • Understand the development lifecycle of a language model application
  • Explore available tools and frameworks to implement GenAIOps
  • Exercise - Compare language models from the model catalog
  • Module assessment
• Manage prompts for agents in Microsoft Foundry with GitHub
  • Apply version control to prompts
  • Understand Microsoft Foundry agents and prompt versioning
  • Organize prompts in GitHub repositories
  • Develop safe prompt deployment workflows
  • Exercise - Develop prompt and agent versions
  • Knowledge check
• Evaluate and optimize AI agents through structured experiments
  • Design evaluation experiments
  • Apply Git-based workflows to optimization experiments
  • Apply evaluation rubrics for consistent scoring
  • Exercise - Evaluate and compare AI agent versions
  • Knowledge check
• Automate AI evaluations with Microsoft Foundry and GitHub Actions
  • Understand why automated evaluations matter
  • Align evaluators with human criteria
  • Create evaluation datasets
  • Implement batch evaluations with Python
  • Integrate evaluations into GitHub Actions
  • Exercise - Set up automated evaluations
  • Knowledge check
• Monitor your generative AI application
  • Why do you need to monitor?
  • Understand key metrics to monitor
  • Explore how to monitor with Azure
  • Integrate monitoring into your app
  • Interpret monitoring results
  • Exercise - Enable monitoring for a generative AI application
  • Knowledge check
• Analyze and debug your generative AI app with tracing
  • Why do you need to use tracing?
  • Identify what to trace in generative AI applications
  • Implement tracing in generative AI applications
  • Debug complex workflows with advanced tracing patterns
  • Make informed decisions with trace data analysis
  • Exercise - Enable tracing for a generative AI application
  • Knowledge check