Lenny Rachitsky: Okay, I'm really glad you went there. This is essentially RLHF reinforcement learning with human feedback, which is exactly what I wanted to also talk about, right? Chip Huyen: Yeah. So I think it's general, it's a way of learning. It's training is [inaudible 00:18:33] learning and whether it learn from human feedback or AI feedback or verifiable rewards, I think I say it's just different way of collating signals. Lenny Rachitsky: Awesome. Yeah. We had the CEO of Anthropic on the podcast and he talked about their version of RLHF, which is AI driven reinforcement learning. I love the way you phrased it where basically you want to help the model, you want to reinforce correct behavior and correct answers, and this is the method to do it, whether it's say an engineer seeing an output from a model being like, "No, here's how I would code it differently." And it's training a different model that the original model works with to tell it, am I correct or not correct? Is that right, roughly?
It's training is [inaudible 00:18:33] learning and whether it learn from human feedback or AI feedback or verifiable rewards, I think I say it's just different way of collating signals.