Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

بواسطة: Coursera

Overview

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically.

By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow.

The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI.

Syllabus

  • Practical Aspects of Deep Learning
    • Discover and experiment with a variety of different initialization methods, apply L2 regularization and dropout to avoid model overfitting, then apply gradient checking to identify errors in a fraud detection model.
  • Optimization Algorithms
    • Develop your deep learning toolbox by adding more advanced optimizations, random minibatching, and learning rate decay scheduling to speed up your models.
  • Hyperparameter Tuning, Batch Normalization and Programming Frameworks
    • Explore TensorFlow, a deep learning framework that allows you to build neural networks quickly and easily, then train a neural network on a TensorFlow dataset.

Taught by

Andrew Ng

Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization
الذهاب الي الدورة

Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

بواسطة: Coursera

  • Coursera
  • مجانية
  • الإنجليزية
  • متاح شهادة
  • متاح في أي وقت
  • intermediate
  • Chinese, Arabic, French, Portuguese, Italian, German, Russian, Turkish, English, Spanish, Korean, Thai, Indonesian, Kazakh, Hindi, Swedish, Greek, Ukrainian, Japanese, Polish, Dutch, Hungarian
8.1.2PHP Version256msRequest Duration2MBMemory UsageGET ar/الدورات/{slug}Route
    • Booting (148ms)
    • Application (107ms)
    • 1 x Booting (57.95%)
      148.45ms
      1 x Application (41.81%)
      107.10ms
      14 templates were rendered
      • public.courses.show (resources/views/public/courses/show.blade.php)3bladefile
        Params
        0
        course
        1
        links
        2
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.courses.partials.details (resources/views/public/courses/partials/details.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.layouts.main (resources/views/public/layouts/main.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.meta (resources/views/public/layouts/partials/meta.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.navbar (resources/views/public/layouts/partials/navbar.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.links (resources/views/public/auth/profile/partials/links.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.layouts.partials.flash-session (resources/views/public/layouts/partials/flash-session.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      uri
      GET ar/الدورات/{slug}
      middleware
      web, localize:ar
      controller
      App\Http\Controllers\CourseController@show
      as
      ar.courses.show
      namespace
      prefix
      /ar
      where
      file
      app/Http/Controllers/CourseController.php:17-35
      7 statements were executed17.53ms
      • select * from `courses` where `slug_ar` = 'improving-deep-neural-networks:-hyperparameter-tuning,-regularization-and-optimization' limit 1
        16.38ms/app/Http/Controllers/CourseController.php:20corspedia
        Metadata
        Bindings
        • 0. improving-deep-neural-networks:-hyperparameter-tuning,-regularization-and-optimization
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:20
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • update `courses` set `visitors` = `visitors` + 1, `courses`.`updated_at` = '2025-04-12 23:59:22' where `id` = 6802
        360μs/app/Http/Controllers/CourseController.php:21corspedia
        Metadata
        Bindings
        • 0. 2025-04-12 23:59:22
        • 1. 6802
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:21
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `topic_id`, `slug_en`, `slug_ar` from `subjects` where `subjects`.`id` in (59)
        160μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `slug_en`, `slug_ar` from `topics` where `topics`.`id` in (1)
        140μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 25. /app/Http/Controllers/CourseController.php:23
        • 26. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 27. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 28. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 29. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `institutions` where `institutions`.`id` in (34) and `institutions`.`deleted_at` is null
        160μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `providers` where `providers`.`id` in (2) and `providers`.`deleted_at` is null
        150μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `html_files` where `html_files`.`id` = 6793 limit 1
        180μs/app/Models/Course.php:84corspedia
        Metadata
        Bindings
        • 0. 6793
        Backtrace
        • 21. /app/Models/Course.php:84
        • 28. view::public.courses.show:29
        • 30. /vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:125
        • 31. /vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • 32. /vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
      App\Models\HtmlFile
      1
      App\Models\Provider
      1
      App\Models\Institution
      1
      App\Models\Topic
      1
      App\Models\Subject
      1
      App\Models\Course
      1
        _token
        aW1Le1lavU7alVNJxPyOAcrFfOkG47fZ24b4xQxs
        locale
        ar
        _previous
        array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/improv...
        _flash
        array:2 [ "old" => [] "new" => [] ]
        PHPDEBUGBAR_STACK_DATA
        []
        path_info
        /ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/improving-deep-neural-networks:-hyperparameter-tuning,-regularization-and-optimization
        status_code
        200
        
        status_text
        OK
        format
        html
        content_type
        text/html; charset=UTF-8
        request_query
        []
        
        request_request
        []
        
        request_headers
        0 of 0
        array:24 [ "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "priority" => array:1 [ 0 => "u=0, i" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "cf-ipcountry" => array:1 [ 0 => "US" ] "cf-connecting-ip" => array:1 [ 0 => "18.191.37.140" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "x-forwarded-for" => array:1 [ 0 => "18.191.37.140" ] "cf-ray" => array:1 [ 0 => "92f6ad52ff9eeac0-ORD" ] "host" => array:1 [ 0 => "www.corspedia.com" ] "content-length" => array:1 [ 0 => "" ] "content-type" => array:1 [ 0 => "" ] ]
        request_server
        0 of 0
        array:50 [ "USER" => "www-data" "HOME" => "/var/www" "HTTP_SEC_CH_UA_MOBILE" => "?0" "HTTP_SEC_CH_UA" => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" "HTTP_CACHE_CONTROL" => "no-cache" "HTTP_PRAGMA" => "no-cache" "HTTP_UPGRADE_INSECURE_REQUESTS" => "1" "HTTP_PRIORITY" => "u=0, i" "HTTP_USER_AGENT" => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "HTTP_CF_IPCOUNTRY" => "US" "HTTP_CF_CONNECTING_IP" => "18.191.37.140" "HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "HTTP_SEC_FETCH_SITE" => "none" "HTTP_CF_VISITOR" => "{"scheme":"https"}" "HTTP_SEC_FETCH_MODE" => "navigate" "HTTP_SEC_FETCH_USER" => "?1" "HTTP_X_FORWARDED_PROTO" => "https" "HTTP_CDN_LOOP" => "cloudflare; loops=1" "HTTP_ACCEPT_ENCODING" => "gzip, br" "HTTP_SEC_FETCH_DEST" => "document" "HTTP_SEC_CH_UA_PLATFORM" => ""Windows"" "HTTP_X_FORWARDED_FOR" => "18.191.37.140" "HTTP_CF_RAY" => "92f6ad52ff9eeac0-ORD" "HTTP_HOST" => "www.corspedia.com" "REDIRECT_STATUS" => "200" "SERVER_NAME" => "corspedia.com" "SERVER_PORT" => "443" "SERVER_ADDR" => "141.95.147.152" "REMOTE_USER" => "" "REMOTE_PORT" => "47098" "REMOTE_ADDR" => "172.70.126.67" "SERVER_SOFTWARE" => "nginx/1.18.0" "GATEWAY_INTERFACE" => "CGI/1.1" "HTTPS" => "on" "REQUEST_SCHEME" => "https" "SERVER_PROTOCOL" => "HTTP/2.0" "DOCUMENT_ROOT" => "/var/www/corspedia/public" "DOCUMENT_URI" => "/index.php" "REQUEST_URI" => "/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/improving-deep-neural-networks:-hyperparameter-tuning,-regularization-and-optimization" "SCRIPT_NAME" => "/index.php" "CONTENT_LENGTH" => "" "CONTENT_TYPE" => "" "REQUEST_METHOD" => "GET" "QUERY_STRING" => "" "SCRIPT_FILENAME" => "/var/www/corspedia/public/index.php" "PATH_INFO" => "" "FCGI_ROLE" => "RESPONDER" "PHP_SELF" => "/index.php" "REQUEST_TIME_FLOAT" => 1744502362.3268 "REQUEST_TIME" => 1744502362 ]
        request_cookies
        []
        
        response_headers
        0 of 0
        array:5 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "no-cache, private" ] "date" => array:1 [ 0 => "Sat, 12 Apr 2025 23:59:22 GMT" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6ImxqZ01aa29FS3AzYlBHdGxQdWJvT1E9PSIsInZhbHVlIjoiN1VSQXBjMlY5dFBzRDNvandlVjlCd055VmxKNmtoQ05JcGZGKytmRUsyVnBZUEFEYVRab1d3K1VwM1V1elJnM0J0MW1wbEpoUE5zWlNMSFIwNDVLUlhOakgzTlBTeWNJMWorVXp0cU9KRFZoNEU0cm5OYzhNa2Z4VTdwcTdNYmUiLCJtYWMiOiIyYTk2ZWY5YjQ0NjA1ZDVhN2UzOGZiMGNiMDM3ZDViZTMzZTk1ZmVmOTc5YzE3NzBhZGE4ZDc5ZWNiNzRmZGZjIiwidGFnIjoiIn0%3D; expires=Sun, 13 Apr 2025 01:59:22 GMT; Max-Age=7200; path=/; samesite=laxXSRF-TOKEN=eyJpdiI6ImxqZ01aa29FS3AzYlBHdGxQdWJvT1E9PSIsInZhbHVlIjoiN1VSQXBjMlY5dFBzRDNvandlVjlCd055VmxKNmtoQ05JcGZGKytmRUsyVnBZUEFEYVRab1d3K1VwM1V1elJnM0J0MW1wb" 1 => "laravel_session=eyJpdiI6IlY1TmZFWnhMaW1RdzVKeWJHYWFNVHc9PSIsInZhbHVlIjoiNmNaMWdjbXRNWjZKZFAyaFQ2SkNONzVtbnh1YzVoTm43LzRxa05RNEFlK01laXA4Q1c5VXlUd2ozaFYrRGlpdDJMWDQ4dFRnTWFVeGFUT2xWUElaSHZFejJPK2RHVE14TWZ0WGZmWGVkNXdLTWpNOWs4RW9mNFpGbGxLWXY4M3ciLCJtYWMiOiI5OWE0M2Q3YjcxYWZkYzc1MmI1OWI4NTAwYWM4NDExOTZhMDg1YTQ4ODU2ZWFjNWE0N2I0YWM3ODE4ZDMxNjEzIiwidGFnIjoiIn0%3D; expires=Sun, 13 Apr 2025 01:59:22 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6IlY1TmZFWnhMaW1RdzVKeWJHYWFNVHc9PSIsInZhbHVlIjoiNmNaMWdjbXRNWjZKZFAyaFQ2SkNONzVtbnh1YzVoTm43LzRxa05RNEFlK01laXA4Q1c5VXlUd2ozaFYrRGlpdDJM" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6ImxqZ01aa29FS3AzYlBHdGxQdWJvT1E9PSIsInZhbHVlIjoiN1VSQXBjMlY5dFBzRDNvandlVjlCd055VmxKNmtoQ05JcGZGKytmRUsyVnBZUEFEYVRab1d3K1VwM1V1elJnM0J0MW1wbEpoUE5zWlNMSFIwNDVLUlhOakgzTlBTeWNJMWorVXp0cU9KRFZoNEU0cm5OYzhNa2Z4VTdwcTdNYmUiLCJtYWMiOiIyYTk2ZWY5YjQ0NjA1ZDVhN2UzOGZiMGNiMDM3ZDViZTMzZTk1ZmVmOTc5YzE3NzBhZGE4ZDc5ZWNiNzRmZGZjIiwidGFnIjoiIn0%3D; expires=Sun, 13-Apr-2025 01:59:22 GMT; path=/XSRF-TOKEN=eyJpdiI6ImxqZ01aa29FS3AzYlBHdGxQdWJvT1E9PSIsInZhbHVlIjoiN1VSQXBjMlY5dFBzRDNvandlVjlCd055VmxKNmtoQ05JcGZGKytmRUsyVnBZUEFEYVRab1d3K1VwM1V1elJnM0J0MW1wb" 1 => "laravel_session=eyJpdiI6IlY1TmZFWnhMaW1RdzVKeWJHYWFNVHc9PSIsInZhbHVlIjoiNmNaMWdjbXRNWjZKZFAyaFQ2SkNONzVtbnh1YzVoTm43LzRxa05RNEFlK01laXA4Q1c5VXlUd2ozaFYrRGlpdDJMWDQ4dFRnTWFVeGFUT2xWUElaSHZFejJPK2RHVE14TWZ0WGZmWGVkNXdLTWpNOWs4RW9mNFpGbGxLWXY4M3ciLCJtYWMiOiI5OWE0M2Q3YjcxYWZkYzc1MmI1OWI4NTAwYWM4NDExOTZhMDg1YTQ4ODU2ZWFjNWE0N2I0YWM3ODE4ZDMxNjEzIiwidGFnIjoiIn0%3D; expires=Sun, 13-Apr-2025 01:59:22 GMT; path=/; httponlylaravel_session=eyJpdiI6IlY1TmZFWnhMaW1RdzVKeWJHYWFNVHc9PSIsInZhbHVlIjoiNmNaMWdjbXRNWjZKZFAyaFQ2SkNONzVtbnh1YzVoTm43LzRxa05RNEFlK01laXA4Q1c5VXlUd2ozaFYrRGlpdDJM" ] ]
        session_attributes
        0 of 0
        array:5 [ "_token" => "aW1Le1lavU7alVNJxPyOAcrFfOkG47fZ24b4xQxs" "locale" => "ar" "_previous" => array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/improving-deep-neural-networks:-hyperparameter-tuning,-regularization-and-optimization" ] "_flash" => array:2 [ "old" => [] "new" => [] ] "PHPDEBUGBAR_STACK_DATA" => [] ]