Bandit Algorithm (Online Machine Learning)

بواسطة: Swayam

Overview

In many scenarios one faces uncertain environments where a-priori the best action to play is unknown. How to obtain best possible reward/utility in such scenarios. One natural way is to first explore the environment and to identify the `best’ actions and exploit them. However, this give raise to an exploration vs exploitation dilemma, where on hand hand we need to do sufficient explorations to identify the best action so that we are confident about its optimality, and on the other hand, best actions need to exploited more number of times to obtain higher reward. In this course we will study many bandit algorithms that balance exploration and exploitation well in various random environment to accumulate good rewards over the duration of play. Bandit algorithms find applications in online advertising, recommendation systems, auctions, routing, e-commerce or in any filed online scenarios where information can be gather in an increment fashion.
INTENDED AUDIENCE :
Computer Sceince, Electrical Engineering, Operations Research, Mathematics and Statistics
PREREQUISITES :Basics of Probability Theory and Optimization
INDUSTRIES SUPPORT :All companies related to Internet Technologies (ex. Google, Microsoft, Flipkart, Ola, Amazon, etc.)

Syllabus

COURSE LAYOUT

Week 1:Introduction to Bandit Algorithms. From Batch to Online SettingWeek 2:Adversarial Setting with Full information (Halving, WM Algorithm )Week 3:Adversarial Setting with Bandit InformationWeek 4:Regret lower bounds for adversarial Setting
Week 5:Introduction to Stochastic Setting and various regret notionsWeek 6:A primer on Concentration inequalitiesWeek 7:Stochastic Bandit Algorithms UCB, KL-UCBWeek 8:Lower bounds for stochastic Bandits
Week 9:Introductions to contextual banditsWeek 10:Overview of contextual bandit algorithmsWeek 11:Introduction to pure exploration setups (fixed confidence vs budget)Week 12:Algorithms for pure explorations (LUCB, KL-LUCB, lil’UCB).

Taught by

Prof. Manjesh hanawal

Bandit Algorithm (Online Machine Learning)
الذهاب الي الدورة

Bandit Algorithm (Online Machine Learning)

بواسطة: Swayam

  • Swayam
  • مجانية
  • الإنجليزية
  • متاح شهادة
  • أيام محددة
  • الجميع
  • N/A
8.1.2PHP Version334msRequest Duration2MBMemory UsageGET ar/الدورات/{slug}Route
    • Booting (218ms)
    • Application (116ms)
    • 1 x Booting (65.13%)
      217.59ms
      1 x Application (34.65%)
      115.77ms
      14 templates were rendered
      • public.courses.show (resources/views/public/courses/show.blade.php)3bladefile
        Params
        0
        course
        1
        links
        2
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.courses.partials.details (resources/views/public/courses/partials/details.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.layouts.main (resources/views/public/layouts/main.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.meta (resources/views/public/layouts/partials/meta.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.navbar (resources/views/public/layouts/partials/navbar.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.links (resources/views/public/auth/profile/partials/links.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.layouts.partials.flash-session (resources/views/public/layouts/partials/flash-session.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      uri
      GET ar/الدورات/{slug}
      middleware
      web, localize:ar
      controller
      App\Http\Controllers\CourseController@show
      as
      ar.courses.show
      namespace
      prefix
      /ar
      where
      file
      app/Http/Controllers/CourseController.php:17-35
      7 statements were executed6.59ms
      • select * from `courses` where `slug_ar` = 'bandit-algorithm-(online-machine-learning)' limit 1
        5.24ms/app/Http/Controllers/CourseController.php:20corspedia
        Metadata
        Bindings
        • 0. bandit-algorithm-(online-machine-learning)
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:20
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • update `courses` set `visitors` = `visitors` + 1, `courses`.`updated_at` = '2025-02-10 19:34:32' where `id` = 2685
        300μs/app/Http/Controllers/CourseController.php:21corspedia
        Metadata
        Bindings
        • 0. 2025-02-10 19:34:32
        • 1. 2685
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:21
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `topic_id`, `slug_en`, `slug_ar` from `subjects` where `subjects`.`id` in (4)
        220μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `slug_en`, `slug_ar` from `topics` where `topics`.`id` in (1)
        180μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 25. /app/Http/Controllers/CourseController.php:23
        • 26. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 27. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 28. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 29. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `institutions` where `institutions`.`id` in (67) and `institutions`.`deleted_at` is null
        210μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `providers` where `providers`.`id` in (14) and `providers`.`deleted_at` is null
        200μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `html_files` where `html_files`.`id` = 2676 limit 1
        240μs/app/Models/Course.php:84corspedia
        Metadata
        Bindings
        • 0. 2676
        Backtrace
        • 21. /app/Models/Course.php:84
        • 28. view::public.courses.show:29
        • 30. /vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:125
        • 31. /vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • 32. /vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
      App\Models\HtmlFile
      1
      App\Models\Provider
      1
      App\Models\Institution
      1
      App\Models\Topic
      1
      App\Models\Subject
      1
      App\Models\Course
      1
        _token
        sUPjegv068j6N71OAJuAG1wRCqCsrmNLsK2DJGbf
        locale
        ar
        _previous
        array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/bandit...
        _flash
        array:2 [ "old" => [] "new" => [] ]
        PHPDEBUGBAR_STACK_DATA
        []
        path_info
        /ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/bandit-algorithm-%28online-machine-learning%29
        status_code
        200
        
        status_text
        OK
        format
        html
        content_type
        text/html; charset=UTF-8
        request_query
        []
        
        request_request
        []
        
        request_headers
        0 of 0
        array:24 [ "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "priority" => array:1 [ 0 => "u=0, i" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "cf-connecting-ip" => array:1 [ 0 => "3.15.0.233" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "cf-ipcountry" => array:1 [ 0 => "US" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "x-forwarded-for" => array:1 [ 0 => "3.15.0.233" ] "cf-ray" => array:1 [ 0 => "90fe897feb9c8140-ORD" ] "host" => array:1 [ 0 => "www.corspedia.com" ] "content-length" => array:1 [ 0 => "" ] "content-type" => array:1 [ 0 => "" ] ]
        request_server
        0 of 0
        array:50 [ "USER" => "www-data" "HOME" => "/var/www" "HTTP_SEC_CH_UA_MOBILE" => "?0" "HTTP_SEC_CH_UA" => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" "HTTP_CACHE_CONTROL" => "no-cache" "HTTP_PRAGMA" => "no-cache" "HTTP_CDN_LOOP" => "cloudflare; loops=1" "HTTP_PRIORITY" => "u=0, i" "HTTP_UPGRADE_INSECURE_REQUESTS" => "1" "HTTP_USER_AGENT" => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "HTTP_CF_CONNECTING_IP" => "3.15.0.233" "HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "HTTP_SEC_FETCH_SITE" => "none" "HTTP_CF_VISITOR" => "{"scheme":"https"}" "HTTP_SEC_FETCH_MODE" => "navigate" "HTTP_SEC_FETCH_USER" => "?1" "HTTP_X_FORWARDED_PROTO" => "https" "HTTP_CF_IPCOUNTRY" => "US" "HTTP_ACCEPT_ENCODING" => "gzip, br" "HTTP_SEC_FETCH_DEST" => "document" "HTTP_SEC_CH_UA_PLATFORM" => ""Windows"" "HTTP_X_FORWARDED_FOR" => "3.15.0.233" "HTTP_CF_RAY" => "90fe897feb9c8140-ORD" "HTTP_HOST" => "www.corspedia.com" "REDIRECT_STATUS" => "200" "SERVER_NAME" => "corspedia.com" "SERVER_PORT" => "443" "SERVER_ADDR" => "141.95.147.152" "REMOTE_USER" => "" "REMOTE_PORT" => "37774" "REMOTE_ADDR" => "172.70.130.178" "SERVER_SOFTWARE" => "nginx/1.18.0" "GATEWAY_INTERFACE" => "CGI/1.1" "HTTPS" => "on" "REQUEST_SCHEME" => "https" "SERVER_PROTOCOL" => "HTTP/2.0" "DOCUMENT_ROOT" => "/var/www/corspedia/public" "DOCUMENT_URI" => "/index.php" "REQUEST_URI" => "/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/bandit-algorithm-%28online-machine-learning%29" "SCRIPT_NAME" => "/index.php" "CONTENT_LENGTH" => "" "CONTENT_TYPE" => "" "REQUEST_METHOD" => "GET" "QUERY_STRING" => "" "SCRIPT_FILENAME" => "/var/www/corspedia/public/index.php" "PATH_INFO" => "" "FCGI_ROLE" => "RESPONDER" "PHP_SELF" => "/index.php" "REQUEST_TIME_FLOAT" => 1739216071.9391 "REQUEST_TIME" => 1739216071 ]
        request_cookies
        []
        
        response_headers
        0 of 0
        array:5 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "no-cache, private" ] "date" => array:1 [ 0 => "Mon, 10 Feb 2025 19:34:32 GMT" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6InpVZDdOazJMY3FRYml6QW5XZFFHN0E9PSIsInZhbHVlIjoieFJDSzM0cEUvcnAvTTFsc3lxZnlsNzlBL3hxelJJeEkyck9lajBZWFdkSjZpQlV0OHUzR1M1WDBxNkEvcUxNSllMTFVwYnlrZ1pvTG5tUVhmNTlFU2R1SmxHS2hlNTZuUHp4L1FtV2M3OXk4dFhnT3kzWC82T1NQM2hPdUs2ajIiLCJtYWMiOiIzOTA1NzU0MmMzYjMyN2Y4NTRiM2ZhYTliZmYwZGQ5MGIxMmZkMjUyNmNjMTk4YjQ5YTRjMzRiMTNkYmE2ZGJhIiwidGFnIjoiIn0%3D; expires=Mon, 10 Feb 2025 21:34:32 GMT; Max-Age=7200; path=/; samesite=laxXSRF-TOKEN=eyJpdiI6InpVZDdOazJMY3FRYml6QW5XZFFHN0E9PSIsInZhbHVlIjoieFJDSzM0cEUvcnAvTTFsc3lxZnlsNzlBL3hxelJJeEkyck9lajBZWFdkSjZpQlV0OHUzR1M1WDBxNkEvcUxNSllMTFVwY" 1 => "laravel_session=eyJpdiI6InMzUlNLanlQZVFkSk82K0tMUlJkYVE9PSIsInZhbHVlIjoiMVJMQUNTYWRmdDNsOUFyS0FOcnBZaVFwL0NaRXlISnVseCsveUNLcHZmU1NrcXoxME4xNzlERUJkcmV5a1VWNlE4YVYxMVo2TDV1UGRZT1BwK2hzejJRZTliR3llam9Kci9KOHlmaG5kckw4ZzVDUVpnVHF2Rjh5Yk1BMHBsSVEiLCJtYWMiOiI3MDlhYTMzNzk5NTRjOWJiZmEzOGViMmJkZjBmMDhmZjY1YTIxZmI2Njc5Mzc2MzVmZDQzZWY1NjM3MmZmYmM1IiwidGFnIjoiIn0%3D; expires=Mon, 10 Feb 2025 21:34:32 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6InMzUlNLanlQZVFkSk82K0tMUlJkYVE9PSIsInZhbHVlIjoiMVJMQUNTYWRmdDNsOUFyS0FOcnBZaVFwL0NaRXlISnVseCsveUNLcHZmU1NrcXoxME4xNzlERUJkcmV5a1VWNlE4" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6InpVZDdOazJMY3FRYml6QW5XZFFHN0E9PSIsInZhbHVlIjoieFJDSzM0cEUvcnAvTTFsc3lxZnlsNzlBL3hxelJJeEkyck9lajBZWFdkSjZpQlV0OHUzR1M1WDBxNkEvcUxNSllMTFVwYnlrZ1pvTG5tUVhmNTlFU2R1SmxHS2hlNTZuUHp4L1FtV2M3OXk4dFhnT3kzWC82T1NQM2hPdUs2ajIiLCJtYWMiOiIzOTA1NzU0MmMzYjMyN2Y4NTRiM2ZhYTliZmYwZGQ5MGIxMmZkMjUyNmNjMTk4YjQ5YTRjMzRiMTNkYmE2ZGJhIiwidGFnIjoiIn0%3D; expires=Mon, 10-Feb-2025 21:34:32 GMT; path=/XSRF-TOKEN=eyJpdiI6InpVZDdOazJMY3FRYml6QW5XZFFHN0E9PSIsInZhbHVlIjoieFJDSzM0cEUvcnAvTTFsc3lxZnlsNzlBL3hxelJJeEkyck9lajBZWFdkSjZpQlV0OHUzR1M1WDBxNkEvcUxNSllMTFVwY" 1 => "laravel_session=eyJpdiI6InMzUlNLanlQZVFkSk82K0tMUlJkYVE9PSIsInZhbHVlIjoiMVJMQUNTYWRmdDNsOUFyS0FOcnBZaVFwL0NaRXlISnVseCsveUNLcHZmU1NrcXoxME4xNzlERUJkcmV5a1VWNlE4YVYxMVo2TDV1UGRZT1BwK2hzejJRZTliR3llam9Kci9KOHlmaG5kckw4ZzVDUVpnVHF2Rjh5Yk1BMHBsSVEiLCJtYWMiOiI3MDlhYTMzNzk5NTRjOWJiZmEzOGViMmJkZjBmMDhmZjY1YTIxZmI2Njc5Mzc2MzVmZDQzZWY1NjM3MmZmYmM1IiwidGFnIjoiIn0%3D; expires=Mon, 10-Feb-2025 21:34:32 GMT; path=/; httponlylaravel_session=eyJpdiI6InMzUlNLanlQZVFkSk82K0tMUlJkYVE9PSIsInZhbHVlIjoiMVJMQUNTYWRmdDNsOUFyS0FOcnBZaVFwL0NaRXlISnVseCsveUNLcHZmU1NrcXoxME4xNzlERUJkcmV5a1VWNlE4" ] ]
        session_attributes
        0 of 0
        array:5 [ "_token" => "sUPjegv068j6N71OAJuAG1wRCqCsrmNLsK2DJGbf" "locale" => "ar" "_previous" => array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/bandit-algorithm-%28online-machine-learning%29" ] "_flash" => array:2 [ "old" => [] "new" => [] ] "PHPDEBUGBAR_STACK_DATA" => [] ]