{"id":29978,"date":"2025-02-04T10:08:47","date_gmt":"2025-02-04T15:08:47","guid":{"rendered":"https:\/\/www.bu.edu\/csmet\/?p=29978"},"modified":"2025-04-01T15:12:01","modified_gmt":"2025-04-01T19:12:01","slug":"understanding-llms-how-they-function-and-how-they-have-changed","status":"publish","type":"post","link":"https:\/\/www.bu.edu\/csmet\/2025\/02\/04\/understanding-llms-how-they-function-and-how-they-have-changed\/","title":{"rendered":"Understanding LLMs: How They Function and How They Have Changed"},"content":{"rendered":"<p><iframe loading=\"lazy\" width=\"608\" height=\"402\" id=\"kaltura_player\" src=\"https:\/\/cdnapisec.kaltura.com\/p\/2159741\/embedPlaykitJs\/uiconf_id\/54272102?iframeembed=true&amp;entry_id=1_hkt8v4x6\" allowfullscreen=\"allowfullscreen\" webkitallowfullscreen=\"webkitallowfullscreen\" mozallowfullscreen=\"mozallowfullscreen\" allow=\"autoplay *; fullscreen *; encrypted-media *\" sandbox=\"allow-downloads allow-forms allow-same-origin allow-scripts allow-top-navigation allow-pointer-lock allow-popups allow-modals allow-orientation-lock allow-popups-to-escape-sandbox allow-presentation allow-top-navigation-by-user-activation\" frameborder=\"0\" title=\"CS Research Seminar: Sebastian Raschka\"><span data-mce-type=\"bookmark\" style=\"display: inline-block; width: 0px; overflow: hidden; line-height: 0;\" class=\"mce_SELRES_start\">\ufeff<\/span><\/iframe><\/p>\n<p><strong>Guest Speaker:<span>\u00a0<\/span><\/strong><a href=\"https:\/\/sebastianraschka.com\/\">Dr. Sebastian Ratschka<\/a>, Staff Research Engineer at Lightning AI<\/p>\n<p><i>Moderated by<span>\u00a0<\/span><a href=\"https:\/\/www.bu.edu\/csmet\/profile\/reza-rawassizadeh\/\">Dr. Reza Rawassizadeh<\/a>, Associate Prof of CS<\/i><\/p>\n<p><strong>Monday, 3rd February, 2025 <\/strong><span><a href=\"https:\/\/bu-metcollege.zoom.us\/webinar\/register\/WN_eiDl0PZRTICFR1TOLdllEw#\/registration\"><\/a><\/span><\/p>\n<p><strong>Abstract:<\/strong><span>\u00a0<\/span>This talk will guide the audience through the key stages of developing large language models (LLMs). We\u2019ll start by explaining how these models are built, including the coding of their architectures. Next, we\u2019ll discuss the processes of pretraining and fine-tuning, detailing what these stages involve and why they are important. In this talk, attendees will also learn about recent developments, including how model architectures have evolved from early GPT models to Llama 3.1. Additionally, the talk will provide an overview of the most recent training recipes, which include multi-stage pre-training and post-training.<\/p>\n<p><strong>Bio:<span>\u00a0<\/span><\/strong>Sebastian Raschka, PhD has been working on machine learning and AI for more than a decade. Sebastian joined Lightning AI in 2022, where he is a Staff Research Engineer focusing on AI and LLM research and development. Prior to that, Sebastian worked at the University of Wisconsin-Madison as an assistant professor in the Department of Statistics, focusing on deep learning and machine learning research. He has a strong passion for education and is best known for his bestselling books on machine learning using open-source software. You can find out more about Sebastian\u2019s current projects at<span>\u00a0<\/span><a href=\"https:\/\/sebastianraschka.com\/\">https:\/\/sebastianraschka.com\/<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\ufeff Guest Speaker:\u00a0Dr. Sebastian Ratschka, Staff Research Engineer at Lightning AI Moderated by\u00a0Dr. Reza Rawassizadeh, Associate Prof of CS Monday, 3rd February, 2025 Abstract:\u00a0This talk will guide the audience through the key stages of developing large language models (LLMs). We\u2019ll start by explaining how these models are built, including the coding of their architectures. Next, [&hellip;]<\/p>\n","protected":false},"author":24214,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[10840],"tags":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/posts\/29978"}],"collection":[{"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/users\/24214"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/comments?post=29978"}],"version-history":[{"count":5,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/posts\/29978\/revisions"}],"predecessor-version":[{"id":30536,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/posts\/29978\/revisions\/30536"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/media?parent=29978"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/categories?post=29978"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/csmet\/wp-json\/wp\/v2\/tags?post=29978"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}