Optimize ML Models and Deploy Human-in-the-Loop Pipelines

بواسطة: Coursera

Overview

In the third course of the Practical Data Science Specialization, you will learn a series of performance-improvement and cost-reduction techniques to automatically tune model accuracy, compare prediction performance, and generate new training data with human intelligence. After tuning your text classifier using Amazon SageMaker Hyper-parameter Tuning (HPT), you will deploy two model candidates into an A/B test to compare their real-time prediction performance and automatically scale the winning model using Amazon SageMaker Hosting. Lastly, you will set up a human-in-the-loop pipeline to fix misclassified predictions and generate new training data using Amazon Augmented AI and Amazon SageMaker Ground Truth.

Practical data science is geared towards handling massive datasets that do not fit in your local hardware and could originate from multiple sources. One of the biggest benefits of developing and running data science projects in the cloud is the agility and elasticity that the cloud offers to scale up and out at a minimum cost.

The Practical Data Science Specialization helps you develop the practical skills to effectively deploy your data science projects and overcome challenges at each step of the ML workflow using Amazon SageMaker. This Specialization is designed for data-focused developers, scientists, and analysts familiar with the Python and SQL programming languages and want to learn how to build, train, and deploy scalable, end-to-end ML pipelines - both automated and human-in-the-loop - in the AWS cloud.

Syllabus

  • Week 1: Advanced model training, tuning and evaluation
    • Train, tune, and evaluate models using data-parallel and model-parallel strategies and automatic model tuning.
  • Week 2: Advanced model deployment and monitoring
    • Deploy models with A/B testing, monitor model performance, and detect drift from baseline metrics.
  • Week 3: Data labeling and human-in-the-loop pipelines
    • Label data at scale using private human workforces and build human-in-the-loop pipelines.

Taught by

Antje Barth, Shelbee Eigenbrode, Sireesha Muppala and Chris Fregly

Optimize ML Models and Deploy Human-in-the-Loop Pipelines
الذهاب الي الدورة

Optimize ML Models and Deploy Human-in-the-Loop Pipelines

بواسطة: Coursera

  • Coursera
  • مجانية
  • الإنجليزية
  • متاح شهادة
  • متاح في أي وقت
  • الجميع
  • English, Arabic, French, Portuguese, German, Thai, Indonesian, Spanish, Kazakh, Hindi, Russian, Swedish, Korean, Greek, Italian, Chinese, Ukrainian, Japanese, Polish, Dutch, Turkish
8.1.2PHP Version186msRequest Duration2MBMemory UsageGET ar/الدورات/{slug}Route
    • Booting (114ms)
    • Application (71.54ms)
    • 1 x Booting (61.39%)
      114.47ms
      1 x Application (38.37%)
      71.54ms
      14 templates were rendered
      • public.courses.show (resources/views/public/courses/show.blade.php)3bladefile
        Params
        0
        course
        1
        links
        2
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.courses.partials.details (resources/views/public/courses/partials/details.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.layouts.main (resources/views/public/layouts/main.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.meta (resources/views/public/layouts/partials/meta.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.navbar (resources/views/public/layouts/partials/navbar.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.links (resources/views/public/auth/profile/partials/links.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.layouts.partials.flash-session (resources/views/public/layouts/partials/flash-session.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      uri
      GET ar/الدورات/{slug}
      middleware
      web, localize:ar
      controller
      App\Http\Controllers\CourseController@show
      as
      ar.courses.show
      namespace
      prefix
      /ar
      where
      file
      app/Http/Controllers/CourseController.php:17-35
      7 statements were executed4.1ms
      • select * from `courses` where `slug_ar` = 'optimize-ml-models-and-deploy-human-in-the-loop-pipelines' limit 1
        2.42ms/app/Http/Controllers/CourseController.php:20corspedia
        Metadata
        Bindings
        • 0. optimize-ml-models-and-deploy-human-in-the-loop-pipelines
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:20
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • update `courses` set `visitors` = `visitors` + 1, `courses`.`updated_at` = '2025-04-12 22:33:47' where `id` = 295
        830μs/app/Http/Controllers/CourseController.php:21corspedia
        Metadata
        Bindings
        • 0. 2025-04-12 22:33:47
        • 1. 295
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:21
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `topic_id`, `slug_en`, `slug_ar` from `subjects` where `subjects`.`id` in (4)
        160μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `slug_en`, `slug_ar` from `topics` where `topics`.`id` in (1)
        150μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 25. /app/Http/Controllers/CourseController.php:23
        • 26. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 27. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 28. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 29. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `institutions` where `institutions`.`id` in (34) and `institutions`.`deleted_at` is null
        180μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `providers` where `providers`.`id` in (2) and `providers`.`deleted_at` is null
        160μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `html_files` where `html_files`.`id` = 289 limit 1
        200μs/app/Models/Course.php:84corspedia
        Metadata
        Bindings
        • 0. 289
        Backtrace
        • 21. /app/Models/Course.php:84
        • 28. view::public.courses.show:29
        • 30. /vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:125
        • 31. /vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • 32. /vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
      App\Models\HtmlFile
      1
      App\Models\Provider
      1
      App\Models\Institution
      1
      App\Models\Topic
      1
      App\Models\Subject
      1
      App\Models\Course
      1
        _token
        ePK960n1OdlUmb4qsOAjPjQRyBp60MvaNl43b9M7
        locale
        ar
        _previous
        array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/optimi...
        _flash
        array:2 [ "old" => [] "new" => [] ]
        PHPDEBUGBAR_STACK_DATA
        []
        path_info
        /ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/optimize-ml-models-and-deploy-human-in-the-loop-pipelines
        status_code
        200
        
        status_text
        OK
        format
        html
        content_type
        text/html; charset=UTF-8
        request_query
        []
        
        request_request
        []
        
        request_headers
        0 of 0
        array:24 [ "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "priority" => array:1 [ 0 => "u=0, i" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "cf-ipcountry" => array:1 [ 0 => "US" ] "cf-connecting-ip" => array:1 [ 0 => "18.218.155.110" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "x-forwarded-for" => array:1 [ 0 => "18.218.155.110" ] "cf-ray" => array:1 [ 0 => "92f62ff31f8fd94a-ORD" ] "host" => array:1 [ 0 => "www.corspedia.com" ] "content-length" => array:1 [ 0 => "" ] "content-type" => array:1 [ 0 => "" ] ]
        request_server
        0 of 0
        array:50 [ "USER" => "www-data" "HOME" => "/var/www" "HTTP_SEC_CH_UA_MOBILE" => "?0" "HTTP_SEC_CH_UA" => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" "HTTP_CACHE_CONTROL" => "no-cache" "HTTP_PRAGMA" => "no-cache" "HTTP_UPGRADE_INSECURE_REQUESTS" => "1" "HTTP_PRIORITY" => "u=0, i" "HTTP_USER_AGENT" => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "HTTP_CF_IPCOUNTRY" => "US" "HTTP_CF_CONNECTING_IP" => "18.218.155.110" "HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "HTTP_SEC_FETCH_SITE" => "none" "HTTP_CF_VISITOR" => "{"scheme":"https"}" "HTTP_SEC_FETCH_MODE" => "navigate" "HTTP_SEC_FETCH_USER" => "?1" "HTTP_X_FORWARDED_PROTO" => "https" "HTTP_CDN_LOOP" => "cloudflare; loops=1" "HTTP_ACCEPT_ENCODING" => "gzip, br" "HTTP_SEC_FETCH_DEST" => "document" "HTTP_SEC_CH_UA_PLATFORM" => ""Windows"" "HTTP_X_FORWARDED_FOR" => "18.218.155.110" "HTTP_CF_RAY" => "92f62ff31f8fd94a-ORD" "HTTP_HOST" => "www.corspedia.com" "REDIRECT_STATUS" => "200" "SERVER_NAME" => "corspedia.com" "SERVER_PORT" => "443" "SERVER_ADDR" => "141.95.147.152" "REMOTE_USER" => "" "REMOTE_PORT" => "52770" "REMOTE_ADDR" => "172.69.17.192" "SERVER_SOFTWARE" => "nginx/1.18.0" "GATEWAY_INTERFACE" => "CGI/1.1" "HTTPS" => "on" "REQUEST_SCHEME" => "https" "SERVER_PROTOCOL" => "HTTP/2.0" "DOCUMENT_ROOT" => "/var/www/corspedia/public" "DOCUMENT_URI" => "/index.php" "REQUEST_URI" => "/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/optimize-ml-models-and-deploy-human-in-the-loop-pipelines" "SCRIPT_NAME" => "/index.php" "CONTENT_LENGTH" => "" "CONTENT_TYPE" => "" "REQUEST_METHOD" => "GET" "QUERY_STRING" => "" "SCRIPT_FILENAME" => "/var/www/corspedia/public/index.php" "PATH_INFO" => "" "FCGI_ROLE" => "RESPONDER" "PHP_SELF" => "/index.php" "REQUEST_TIME_FLOAT" => 1744497226.9939 "REQUEST_TIME" => 1744497226 ]
        request_cookies
        []
        
        response_headers
        0 of 0
        array:5 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "no-cache, private" ] "date" => array:1 [ 0 => "Sat, 12 Apr 2025 22:33:47 GMT" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6ImhJMVhQYnBUOHRMdlZJdURFbzhqbUE9PSIsInZhbHVlIjoia1dvTm4rN1pVQXRMd2tDZnNBbFY4VEpBc1FycTZETzFyaEhpYTBkTnBDY05wMjNuSENaRXZrOWhlUXRCbEd5SlRxekRlOXJGYkFqNldxSjRNVlRtOFl1MC8vczlpSkdjMkQzL0JROWI0ZkUrSmNtSzBFK0hSZDF6K09VUXl3dzMiLCJtYWMiOiI3MDFlYmRlYmJkYjFjMDc4MWIyOWFhZjgzM2FjY2NiN2FkMzdlODkwZDNiYWQwOTU1MTZkZWUyMzJhMzc1MGY4IiwidGFnIjoiIn0%3D; expires=Sun, 13 Apr 2025 00:33:47 GMT; Max-Age=7200; path=/; samesite=laxXSRF-TOKEN=eyJpdiI6ImhJMVhQYnBUOHRMdlZJdURFbzhqbUE9PSIsInZhbHVlIjoia1dvTm4rN1pVQXRMd2tDZnNBbFY4VEpBc1FycTZETzFyaEhpYTBkTnBDY05wMjNuSENaRXZrOWhlUXRCbEd5SlRxekRlO" 1 => "laravel_session=eyJpdiI6ImpEaXRmNVFIWEdhM09MMzNwelJWVEE9PSIsInZhbHVlIjoiSk50YUsvQ3k1N29GRzZqVkgxRTBuNnVVSThRRVFsQkdadEgwRXBvV0k1bUhQbGNENjlmZEp0dEJRMmNJaVVZVHppdkNZUVVPOE9hWGFBcGVXaGhyZFY1eHZpcExOd040Y0RMZ0VyK3NDR2w3SmpHMEVxdjI3QVR4YVBPd0s4RGEiLCJtYWMiOiIzOTQwYTY2NjkxNzBhODYyOThhYTM4YjI4Y2MxZmY1YmVlMzU4YjU2OWE5OWEyNmEyOWU4OTFlMWRlNzgwMTJjIiwidGFnIjoiIn0%3D; expires=Sun, 13 Apr 2025 00:33:47 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6ImpEaXRmNVFIWEdhM09MMzNwelJWVEE9PSIsInZhbHVlIjoiSk50YUsvQ3k1N29GRzZqVkgxRTBuNnVVSThRRVFsQkdadEgwRXBvV0k1bUhQbGNENjlmZEp0dEJRMmNJaVVZVHpp" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6ImhJMVhQYnBUOHRMdlZJdURFbzhqbUE9PSIsInZhbHVlIjoia1dvTm4rN1pVQXRMd2tDZnNBbFY4VEpBc1FycTZETzFyaEhpYTBkTnBDY05wMjNuSENaRXZrOWhlUXRCbEd5SlRxekRlOXJGYkFqNldxSjRNVlRtOFl1MC8vczlpSkdjMkQzL0JROWI0ZkUrSmNtSzBFK0hSZDF6K09VUXl3dzMiLCJtYWMiOiI3MDFlYmRlYmJkYjFjMDc4MWIyOWFhZjgzM2FjY2NiN2FkMzdlODkwZDNiYWQwOTU1MTZkZWUyMzJhMzc1MGY4IiwidGFnIjoiIn0%3D; expires=Sun, 13-Apr-2025 00:33:47 GMT; path=/XSRF-TOKEN=eyJpdiI6ImhJMVhQYnBUOHRMdlZJdURFbzhqbUE9PSIsInZhbHVlIjoia1dvTm4rN1pVQXRMd2tDZnNBbFY4VEpBc1FycTZETzFyaEhpYTBkTnBDY05wMjNuSENaRXZrOWhlUXRCbEd5SlRxekRlO" 1 => "laravel_session=eyJpdiI6ImpEaXRmNVFIWEdhM09MMzNwelJWVEE9PSIsInZhbHVlIjoiSk50YUsvQ3k1N29GRzZqVkgxRTBuNnVVSThRRVFsQkdadEgwRXBvV0k1bUhQbGNENjlmZEp0dEJRMmNJaVVZVHppdkNZUVVPOE9hWGFBcGVXaGhyZFY1eHZpcExOd040Y0RMZ0VyK3NDR2w3SmpHMEVxdjI3QVR4YVBPd0s4RGEiLCJtYWMiOiIzOTQwYTY2NjkxNzBhODYyOThhYTM4YjI4Y2MxZmY1YmVlMzU4YjU2OWE5OWEyNmEyOWU4OTFlMWRlNzgwMTJjIiwidGFnIjoiIn0%3D; expires=Sun, 13-Apr-2025 00:33:47 GMT; path=/; httponlylaravel_session=eyJpdiI6ImpEaXRmNVFIWEdhM09MMzNwelJWVEE9PSIsInZhbHVlIjoiSk50YUsvQ3k1N29GRzZqVkgxRTBuNnVVSThRRVFsQkdadEgwRXBvV0k1bUhQbGNENjlmZEp0dEJRMmNJaVVZVHpp" ] ]
        session_attributes
        0 of 0
        array:5 [ "_token" => "ePK960n1OdlUmb4qsOAjPjQRyBp60MvaNl43b9M7" "locale" => "ar" "_previous" => array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/optimize-ml-models-and-deploy-human-in-the-loop-pipelines" ] "_flash" => array:2 [ "old" => [] "new" => [] ] "PHPDEBUGBAR_STACK_DATA" => [] ]