Generative Pre-trained Transformers (GPT)

بواسطة: Coursera

Overview

Large language models such as GPT-3.5, which powers ChatGPT, are changing how humans interact with computers and how computers can process text. This course will introduce the fundamental ideas of natural language processing and language modelling that underpin these large language models. We will explore the basics of how language models work, and the specifics of how newer neural-based approaches are built. We will examine the key innovations that have enabled Transformer-based large language models to become dominant in solving various language tasks. Finally, we will examine the challenges in applying these large language models to various problems including the ethical problems involved in their construction and use.

Through hands-on labs, we will learn about the building blocks of Transformers and apply them for generating new text. These Python exercises step you through the process of applying a smaller language model and understanding how it can be evaluated and applied to various problems. Regular practice quizzes will help reinforce the knowledge and prepare you for the graded assessments.

Syllabus

  • Language Modeling
    • This module introduces the concept of language modelling, which is the foundation of models like GPT.
  • Transformers and GPT
    • This module describes the technical background for neural language models and an overview of how they are used to generate text.
  • Applications and Implications
    • This module discusses considerations that are necessary when using GPT and similar models in real-world contexts, specifically discussing the risks of using these models and approaches to mitigating these risks.

Taught by

Mary Ellen Foster, Sean MacAvaney and Jake Lever

Generative Pre-trained Transformers (GPT)
الذهاب الي الدورة

Generative Pre-trained Transformers (GPT)

بواسطة: Coursera

  • Coursera
  • مجانية
  • الإنجليزية
  • متاح شهادة
  • متاح في أي وقت
  • intermediate
  • English
8.1.2PHP Version300msRequest Duration2MBMemory UsageGET ar/الدورات/{slug}Route
    • Booting (192ms)
    • Application (107ms)
    • 1 x Booting (64.05%)
      192.37ms
      1 x Application (35.71%)
      107.25ms
      14 templates were rendered
      • public.courses.show (resources/views/public/courses/show.blade.php)3bladefile
        Params
        0
        course
        1
        links
        2
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.courses.partials.details (resources/views/public/courses/partials/details.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.layouts.main (resources/views/public/layouts/main.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.meta (resources/views/public/layouts/partials/meta.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.navbar (resources/views/public/layouts/partials/navbar.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.links (resources/views/public/auth/profile/partials/links.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.layouts.partials.flash-session (resources/views/public/layouts/partials/flash-session.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      uri
      GET ar/الدورات/{slug}
      middleware
      web, localize:ar
      controller
      App\Http\Controllers\CourseController@show
      as
      ar.courses.show
      namespace
      prefix
      /ar
      where
      file
      app/Http/Controllers/CourseController.php:17-35
      7 statements were executed3.85ms
      • select * from `courses` where `slug_ar` = 'generative-pre-trained-transformers-(gpt)' limit 1
        2.57ms/app/Http/Controllers/CourseController.php:20corspedia
        Metadata
        Bindings
        • 0. generative-pre-trained-transformers-(gpt)
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:20
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • update `courses` set `visitors` = `visitors` + 1, `courses`.`updated_at` = '2025-02-10 19:53:08' where `id` = 312
        280μs/app/Http/Controllers/CourseController.php:21corspedia
        Metadata
        Bindings
        • 0. 2025-02-10 19:53:08
        • 1. 312
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:21
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `topic_id`, `slug_en`, `slug_ar` from `subjects` where `subjects`.`id` in (49)
        190μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `slug_en`, `slug_ar` from `topics` where `topics`.`id` in (1)
        170μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 25. /app/Http/Controllers/CourseController.php:23
        • 26. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 27. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 28. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 29. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `institutions` where `institutions`.`id` in (56) and `institutions`.`deleted_at` is null
        200μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `providers` where `providers`.`id` in (2) and `providers`.`deleted_at` is null
        190μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `html_files` where `html_files`.`id` = 306 limit 1
        250μs/app/Models/Course.php:84corspedia
        Metadata
        Bindings
        • 0. 306
        Backtrace
        • 21. /app/Models/Course.php:84
        • 28. view::public.courses.show:29
        • 30. /vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:125
        • 31. /vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • 32. /vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
      App\Models\HtmlFile
      1
      App\Models\Provider
      1
      App\Models\Institution
      1
      App\Models\Topic
      1
      App\Models\Subject
      1
      App\Models\Course
      1
        _token
        Vib8JK2tzst5D7pr4JTj8jj6ued79I8arCH2LL04
        locale
        ar
        _previous
        array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/genera...
        _flash
        array:2 [ "old" => [] "new" => [] ]
        PHPDEBUGBAR_STACK_DATA
        []
        path_info
        /ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/generative-pre-trained-transformers-%28gpt%29
        status_code
        200
        
        status_text
        OK
        format
        html
        content_type
        text/html; charset=UTF-8
        request_query
        []
        
        request_request
        []
        
        request_headers
        0 of 0
        array:24 [ "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "priority" => array:1 [ 0 => "u=0, i" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "cf-connecting-ip" => array:1 [ 0 => "18.221.190.79" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "cf-ipcountry" => array:1 [ 0 => "US" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "x-forwarded-for" => array:1 [ 0 => "18.221.190.79" ] "cf-ray" => array:1 [ 0 => "90fea4c168eb86e4-ORD" ] "host" => array:1 [ 0 => "www.corspedia.com" ] "content-length" => array:1 [ 0 => "" ] "content-type" => array:1 [ 0 => "" ] ]
        request_server
        0 of 0
        array:50 [ "USER" => "www-data" "HOME" => "/var/www" "HTTP_SEC_CH_UA_MOBILE" => "?0" "HTTP_SEC_CH_UA" => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" "HTTP_CACHE_CONTROL" => "no-cache" "HTTP_PRAGMA" => "no-cache" "HTTP_CDN_LOOP" => "cloudflare; loops=1" "HTTP_PRIORITY" => "u=0, i" "HTTP_UPGRADE_INSECURE_REQUESTS" => "1" "HTTP_USER_AGENT" => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "HTTP_CF_CONNECTING_IP" => "18.221.190.79" "HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "HTTP_SEC_FETCH_SITE" => "none" "HTTP_CF_VISITOR" => "{"scheme":"https"}" "HTTP_SEC_FETCH_MODE" => "navigate" "HTTP_SEC_FETCH_USER" => "?1" "HTTP_X_FORWARDED_PROTO" => "https" "HTTP_CF_IPCOUNTRY" => "US" "HTTP_ACCEPT_ENCODING" => "gzip, br" "HTTP_SEC_FETCH_DEST" => "document" "HTTP_SEC_CH_UA_PLATFORM" => ""Windows"" "HTTP_X_FORWARDED_FOR" => "18.221.190.79" "HTTP_CF_RAY" => "90fea4c168eb86e4-ORD" "HTTP_HOST" => "www.corspedia.com" "REDIRECT_STATUS" => "200" "SERVER_NAME" => "corspedia.com" "SERVER_PORT" => "443" "SERVER_ADDR" => "141.95.147.152" "REMOTE_USER" => "" "REMOTE_PORT" => "30524" "REMOTE_ADDR" => "172.70.131.88" "SERVER_SOFTWARE" => "nginx/1.18.0" "GATEWAY_INTERFACE" => "CGI/1.1" "HTTPS" => "on" "REQUEST_SCHEME" => "https" "SERVER_PROTOCOL" => "HTTP/2.0" "DOCUMENT_ROOT" => "/var/www/corspedia/public" "DOCUMENT_URI" => "/index.php" "REQUEST_URI" => "/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/generative-pre-trained-transformers-%28gpt%29" "SCRIPT_NAME" => "/index.php" "CONTENT_LENGTH" => "" "CONTENT_TYPE" => "" "REQUEST_METHOD" => "GET" "QUERY_STRING" => "" "SCRIPT_FILENAME" => "/var/www/corspedia/public/index.php" "PATH_INFO" => "" "FCGI_ROLE" => "RESPONDER" "PHP_SELF" => "/index.php" "REQUEST_TIME_FLOAT" => 1739217188.3175 "REQUEST_TIME" => 1739217188 ]
        request_cookies
        []
        
        response_headers
        0 of 0
        array:5 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "no-cache, private" ] "date" => array:1 [ 0 => "Mon, 10 Feb 2025 19:53:08 GMT" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IkVaZ05FVzhXdHVKbXJBUlhRM081OUE9PSIsInZhbHVlIjoiTlE0VkJEWnREck5VUWNhaWFoaFVFdnI1U01udEN2a2JBQ0tkUTJZM0wxd0dYSWdFVHJYb2lpRlNZNzkrOFduTkVFQ1BlK09vQ0QvSFFCRFA4SmZXbGxLOVNjbHBQR3RkL0R0by9aN0M2bzdmWjVBbWlmdEZIMVlPbnJXS3hlSXYiLCJtYWMiOiJhZTE2NjBjZDc0NDFkZDFiZjI5ZTQ3NDAxZDljMmJkNjdjMGRiNGY0MDE3YzVkNjE2MmMyNmY2NmIzMTIzNDhiIiwidGFnIjoiIn0%3D; expires=Mon, 10 Feb 2025 21:53:08 GMT; Max-Age=7200; path=/; samesite=laxXSRF-TOKEN=eyJpdiI6IkVaZ05FVzhXdHVKbXJBUlhRM081OUE9PSIsInZhbHVlIjoiTlE0VkJEWnREck5VUWNhaWFoaFVFdnI1U01udEN2a2JBQ0tkUTJZM0wxd0dYSWdFVHJYb2lpRlNZNzkrOFduTkVFQ1BlK" 1 => "laravel_session=eyJpdiI6IkdwWWJjR2NGdHk2ZDhJaDc1M1d4Q0E9PSIsInZhbHVlIjoiVTY1Ni9FNnF3Vk5WQVJ4cWVheU8yb1ErTzRYVGNxOUtlL3UxRlJ0YzBoNHp5Nkh3Vk0wc2htRkoxQUtNcDU0TmwwbWthaEpjRHl1aHh6bG1GaHpySFhrYXJmd0hSNzR6UDlXSnlMcm9HamdJMzdaLy9FWmhQNW43bWVRcTJubDIiLCJtYWMiOiJkZDExOTE3Njk5YTQzNWExZTY2ZDE5YjdkNzMzZDdiZjg4MWUzZGZiYmVmMDdiMzgzNDk3NTUzNWVkZjEyYTZlIiwidGFnIjoiIn0%3D; expires=Mon, 10 Feb 2025 21:53:08 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6IkdwWWJjR2NGdHk2ZDhJaDc1M1d4Q0E9PSIsInZhbHVlIjoiVTY1Ni9FNnF3Vk5WQVJ4cWVheU8yb1ErTzRYVGNxOUtlL3UxRlJ0YzBoNHp5Nkh3Vk0wc2htRkoxQUtNcDU0Tmww" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IkVaZ05FVzhXdHVKbXJBUlhRM081OUE9PSIsInZhbHVlIjoiTlE0VkJEWnREck5VUWNhaWFoaFVFdnI1U01udEN2a2JBQ0tkUTJZM0wxd0dYSWdFVHJYb2lpRlNZNzkrOFduTkVFQ1BlK09vQ0QvSFFCRFA4SmZXbGxLOVNjbHBQR3RkL0R0by9aN0M2bzdmWjVBbWlmdEZIMVlPbnJXS3hlSXYiLCJtYWMiOiJhZTE2NjBjZDc0NDFkZDFiZjI5ZTQ3NDAxZDljMmJkNjdjMGRiNGY0MDE3YzVkNjE2MmMyNmY2NmIzMTIzNDhiIiwidGFnIjoiIn0%3D; expires=Mon, 10-Feb-2025 21:53:08 GMT; path=/XSRF-TOKEN=eyJpdiI6IkVaZ05FVzhXdHVKbXJBUlhRM081OUE9PSIsInZhbHVlIjoiTlE0VkJEWnREck5VUWNhaWFoaFVFdnI1U01udEN2a2JBQ0tkUTJZM0wxd0dYSWdFVHJYb2lpRlNZNzkrOFduTkVFQ1BlK" 1 => "laravel_session=eyJpdiI6IkdwWWJjR2NGdHk2ZDhJaDc1M1d4Q0E9PSIsInZhbHVlIjoiVTY1Ni9FNnF3Vk5WQVJ4cWVheU8yb1ErTzRYVGNxOUtlL3UxRlJ0YzBoNHp5Nkh3Vk0wc2htRkoxQUtNcDU0TmwwbWthaEpjRHl1aHh6bG1GaHpySFhrYXJmd0hSNzR6UDlXSnlMcm9HamdJMzdaLy9FWmhQNW43bWVRcTJubDIiLCJtYWMiOiJkZDExOTE3Njk5YTQzNWExZTY2ZDE5YjdkNzMzZDdiZjg4MWUzZGZiYmVmMDdiMzgzNDk3NTUzNWVkZjEyYTZlIiwidGFnIjoiIn0%3D; expires=Mon, 10-Feb-2025 21:53:08 GMT; path=/; httponlylaravel_session=eyJpdiI6IkdwWWJjR2NGdHk2ZDhJaDc1M1d4Q0E9PSIsInZhbHVlIjoiVTY1Ni9FNnF3Vk5WQVJ4cWVheU8yb1ErTzRYVGNxOUtlL3UxRlJ0YzBoNHp5Nkh3Vk0wc2htRkoxQUtNcDU0Tmww" ] ]
        session_attributes
        0 of 0
        array:5 [ "_token" => "Vib8JK2tzst5D7pr4JTj8jj6ued79I8arCH2LL04" "locale" => "ar" "_previous" => array:1 [ "url" => "https://www.corspedia.com/ar/%D8%A7%D9%84%D8%AF%D9%88%D8%B1%D8%A7%D8%AA/generative-pre-trained-transformers-%28gpt%29" ] "_flash" => array:2 [ "old" => [] "new" => [] ] "PHPDEBUGBAR_STACK_DATA" => [] ]