John Schulman and Barret Zoph | ChatGPT and the Art of Post Training

Abstract:

At this seminar, we’ll talk about the history and development process of ChatGPT, tracing its journey from initial prototypes to becoming a large-scale product and research effort. We’ll delve into the technical and operational challenges involved in post-training, including fine-tuning for conversational behavior, addressing alignment issues, and optimizing for large-scale deployment. Additionally, we’ll highlight some of the innovative solutions developed to tackle these challenges. To conclude, we’ll explore a set of open problems in the field, including ethical considerations, maintaining factual accuracy, and advancing the contextual understanding of conversational AI.