Lenny's Podcast
How to scale intent, quality, and artistry with Al | Katie Dill (Stripe)
Ask the tutor about this video
Sign in with a verified email and ask the tutor anything about this video.
TL;DR
Research summary
Useful takeaways
Workflows
Define a product point of view
Set standards by clarifying the brand’s purpose, what users care about, and how the brand wants to relate to them.
Katie Dill · 5:58
- Define what the brand is for.
- Identify what users care about.
- Decide who the brand wants to be to those users.
- Use those answers as the basis for standards.
Build taste through observation
Notice users’ needs and study products and analogous areas to develop judgment about what feels good, great, or mediocre.
Katie Dill · 8:25
- Observe users’ needs and wants, including beyond what they say.
- Notice what feels good, great, or mediocre in products you use.
- Look to analogous situations, including art and science, for inspiration.
- Use that broader understanding to strengthen standards.
Edit work as a user
Review the whole journey from the user’s perspective, checking problem fit, coherence, and whether the work is fully formed; push it toward completion where needed.
Katie Dill · 13:41
- Experience the work as a user would.
- Check whether it solves the problem and fits how the user thinks.
- Review the end-to-end journey for coherence.
- Identify what is incomplete and push it toward completion.
Iterate and scrutinize AI-assisted work
Treat AI-generated work as a starting point and inspect the details through repeated iterations. Dill’s example took 56 iterations to reach the version she considered truly beautiful.
Katie Dill · 15:48
- Inspect the first output closely for details that feel wrong.
- Iterate to address the issues you notice.
- Continue scrutinizing until the result meets the intended standard.
Give AI specific inputs and challenge its outputs
Tie inputs to the brand and its distinct interests, add source material that establishes standards, ask for different ways of thinking, and push outputs further with critique from adversarial agents. The example wording is paraphrased; no complete target prompt is supplied.
Katie Dill · 18:43
- Make inputs specific to the brand and its unique interests.
- Add source material used to establish standards.
- Ask the AI to think in different ways.
- Push the output further and use adversarial agents to critique it.
- Continue improving it yourself.
Protect room for unusual ideas
When leading a team, make room to explore and preserve unusual ideas; Dill says AI’s lower creation cost can be spent making something special.
Katie Dill · 19:44
- Give the team room to explore with AI.
- Protect unusual ideas during exploration.
- Use some of the reduced creation cost to make something distinctive.
Setups
Encode standards into AI-enabled systems
Make standards available to builders and agents through a design system with documentation, templates, and flows. Dill describes Stripe’s CLI and harness as a way to consume documentation in the builders’ working context and guide product behavior.
Katie Dill · 11:29
- Put design standards into the system builders and agents use.
- Provide documentation at the time and place it is needed.
- Include templates and flows alongside components so the system can guide product behavior.
▶ From building boom to generic patterns
Dill describes how postwar builders reused modernist styles, whose original founders had deliberate principles, until repetition stripped away the intent and produced generic buildings ill-suited to their context. She compares these “zombie buildings” with the current AI building boom, where small teams can build what once required larger teams, and introduces three watch points for avoiding the same outcome.
▶ Three risks of AI-built products
Dill says language models are good at producing probable answers based on what has worked or is popular, but less good at creating work specific to a brand and user context. She warns that AI can make work feel finished prematurely and that fast, easy production can make work seem disposable, with little consideration for long-term maintenance. These risks can yield monotonous, vacant interfaces; she points to personality, thoughtful details, and anticipating users’ situations as signs that a product has been cared for.
▶ Define the point of view and quality bar
Dill’s first recommendation is to define a point of view: what the brand is for, what users care about, and who the brand wants to be to them. At Stripe, she says optimism informs details such as colors, writing, and products intended to support entrepreneurs. In a design critique, a discussion about what quality means for AI-made work led the team to align on the output standard and list 17 improvements; they use “Pepsi bubbling” to mean meticulous craft, aiming to go one level deeper than customers can see. She advises developing taste by noticing users’ needs and observing what feels good, great, or mediocre in products and analogous areas such as art and science.
▶ Encode intent into the system
Her second recommendation is to encode standards into the machine because interfaces, agent fixes, and generative UI may be built without a designer present. Dill argues that the design object is now the system, which must scale intent as well as consistency. She uses Gutenberg’s printing system as an example of machine production designed with enough variety and judgment to preserve the quality of the result. At Stripe, an initial MCP based on design documentation produced inconsistent results for the same prompt. The team then created a CLI built on its design system, with a harness that consumes documentation where builders work; it includes templates and flows as well as components, so it can guide product behavior. Dill says this creates a more coherent baseline, while cautioning that following rules alone can still produce something lifeless.
▶ Edit beyond done
Dill’s third recommendation is to refuse to confuse done with good. Because teams can now build many ideas quickly, filtering that used to happen throughout development may be deferred until after a build, when it is harder to reject. She says organizations need an editor who can assess the end-to-end product, experience it as a user would, test whether it solves the problem and fits the user’s thinking, and check that the journey is coherent and fully formed. Editing should help push work to completion, not only accept or reject it. She cites unexpected details, deeper meaning, and themes that make a whole as qualities AI may miss, and says editors take accountability for decisions down to individual pixels.
▶ Iteration can make AI-assisted work better
Dill describes an event-opening animation that began with a 3D scene and used AI to help create animations. The first version looked fun but felt off when scrutinized; Stefan iterated 56 times before arriving at a version Dill describes as more realistic, lively, subtle, and careful. She says AI opened up the possibility of a more complex scene and more viewpoints, while stressing that the result required scrutiny and was not one-and-done.
▶ Use AI to expand creative possibilities
Dill’s fourth recommendation is to unleash creativity and artistry. She argues that AI can produce monotony but can also act as a creative catalyst, helping teams invent new interfaces and aesthetics beyond chats, charts, and CLIs. She describes people bringing more interesting data visualizations to Stripe’s design team, restaurant websites with character, and people using AI to make art. At Stripe, she says a human created iterations for a Built to Grow cover and AI was then used to fine-tune colors and lines, scaling the humans’ judgment into the final work.
▶ Improve inputs, challenge outputs, protect originality
Dill recommends giving AI more specific inputs tied to the brand and its interests, adding source material that helps establish standards, and asking it to think in different ways. She says to push outputs further and use adversarial agents for critique, while keeping human responsibility for improvement. For teams, she recommends making room to explore and protecting unusual ideas, since using AI only to do existing work faster misses its creative potential. She closes by urging intentionality around point of view, system, quality bar, and ambition. Drawing on Ruskin’s discussion of Gothic architecture, she praises individually crafted details and says users value solved problems and clever touches that anticipate needs and express brand character.
Full transcript
[0:36] palettes, and no adornment. Now, say what you will about modernism, but the founders behind the movement had intentionality. There was a point of view behind the Villa Savoye. There were principles behind the Bauhaus. But after World War II, with urgency to build and new construction methods, the style was essentially copied again and again without intentionality. The thinking got thinner and thinner,
[1:07] and all that was left were generic patterns ill-suited to the context at hand. For example, we used to have banks that looked like this, signaling trust, reliability, and security. And then we got this. You're all too familiar with this reality because it carried on for decades until this day. Zombie buildings all over the place. Nothing differentiates them. Nothing says, "This is intentional. This is fit
[1:39] to purpose." They essentially show no care for the user and the habitats around it, no care for context or the brand. I think about this a lot. One, we're surrounded by them, and two, we're in another building boom now. The AI building boom. Everybody in their mother can build now. Teams of three can do what teams of 30 used to be needed for. But there's echoes of the post-war building boom, similarities that are actually watch
[2:10] points for us. I want to tell you about these watch points, and then we'll talk about how we can navigate them together. First, LLMs are really good at telling you the most probable answer, which essentially means that they're able to tell you what has been or is popular now, what has worked, what was in style. They're less good at telling you what's original or specific to you, your brand, and your user's context. I'll give you an example. Can you guess what this
[2:40] brand sells? Software? No, Korean barbecue. Now, this is, I'm sure, a very fine website, but it has no character, no context for the user or the context of the use, which brings me to my second point, the temptation of done. AI makes things feel finished really, really fast, all too often prematurely. I'll give you an example that I probably shouldn't admit. I have a microwave burrito for
[3:11] lunch all too often. I take the cold, hard rock out of the freezer, plop it on the plate, put it in the microwave, and 90 seconds I go from hungry to lunch. I'm willing to overlook some pretty serious flaws. It's nearly inedible, but I'm so enamored by the speed of execution. The same happens with AI. You type in a sentence, and boom, you have an interface. There's some nice little corners and maybe some drop shadows, and it's good to be done. But
[3:43] an apparently polished state can often be misleading. Does it really solve the problem? Does it really differentiate? All the questions Claire V brought up earlier apply here. And the third watch point, this work is so easy to do, so quick, that it often feels disposable and worst is treated that way. We sometimes let the responsibility of our decisions fall by the wayside and don't really think about the long term. Like who's maintaining this anyways? All
[4:14] three of these watch points can come together to a world that looks a little like this, an actual place in Turkey. If we're not careful, our building boom could end up a little too much like the post-war building boom. The proliferation of patterns ill-suited to the context at hand. Essentially, zombie UI. Zombie UI that's monotonous, vacant, or uncared for. We spend half our waking lives looking at screens. We want software that feels
[4:46] actually cared for. And that means has a little personality. It's something as simple as the Grockbot and the little animation and life that it brings. Or even these tiny details like actually having the date in the tab on the calendar. You probably click by this all day long, but that shows the builder cared about you. Or the understanding of context like the link agent wallet. It anticipates the issues that the agent could have while buying online and has the
[5:17] troubleshooting built in because they anticipate the context of the user even when it's an agent. These examples show builders that are care about their users and bring that care into the work. It shows the character of the brand and it has soul. We can do this with AI even though it can sometimes be the culprit. And I have four recommendations on how we can use AI to build products with care and soul. First one. You have to have a point of view.
[5:49] If you don't, AI will give it for you. And as we've already talked about, that's very likely to be generic and looking at the past. You need to define your brand. What is it for you? What is it for your users? Who do you want to be to them and what do they care about? This forms the basis of your standards. And I'd say this in any era, but it's all the more important right now when building is distributed, ownership is diffuse. We're all contributing to the products in more ways than we've been in the
[6:19] past. It's too easy to abdicate to AI. So, for example, at Stripe, we care a lot about optimism. So, we put that into all the details, big and small. The colors we choose, the way we write, what we write about, and the products that we ship to try to support entrepreneurs. Now, these details are so important to be aligned on because everybody is contributing and so much is changing. And I'll give you an example of where
[6:49] that friction can show up if there isn't alignment. We were recently in a design crit looking at an advertisement. So, this is an ad we were working on that shows our brand, the parallelogram, with our users' visuals. And as we were looking at this, we were like, all right, cool, you kind of get what's happening, but some things are off. And in the room was a cross-functional partner that was looking to move fast and get something shipped. Sure you can't relate. And we had a discussion because they brought up a really interesting
[7:19] question. What's the quality standard for something made with AI? What a curious question. Why should that matter how it was made? It doesn't matter to the users how it was made. It matters to them if it's good or not. And so, that is the basis of our standards. It's the output at the end of the day that matters. And they care if it's good, and so should we. So through that discussion we've got alignment on what really matters here and what we're striving for and then walked away with 17 bullet points of
[7:49] improvements to be made. The frost a little bit here and there, softer edges, bit smaller bubblers. And now we use this term as a verb to mean meticulous craft around the office called Pepsi bubbling. Now, we're not shooting for perfection, we're shooting to go one level deeper than what your customer could see. That meticulous craft will show up for them. And this point of view really drives the standards and drives the work, especially when there's agents involved and so much more is happening on a daily
[8:20] basis. So how do you develop it? Frankly, it's all about getting really good at noticing. Notice what your users need and what they want, not just what they say. Notice what about the world around you signals good and great and meh. Really take note of these things in the products that you use, but also in analogous situations so you can be inspired by art, science, and a broader point of view in the work that you create. Building your sense of taste and
[8:51] your understanding of the world around you and what's good and great can really improve your own standards that you will then need to scale. Which brings me to my second recommendation, which is encode your standards into the machine. It would be a wonderful world if every human had the shared understandings and could make the same decision, but the reality is is humans are not going to be a part of all of these decisions going forward. We are seeing interfaces built without a designer in the room.
[9:21] We're seeing agents finding problems and fixing them while we sleep. And we're certainly seeing generative UI built in real time for users. This is why design systems are having a moment again. And the system of yesterday is different than the system of today. In the past, you didn't have to write everything down because there was always a designer in the room to fill in the blanks. Now, we must enable distributed building and agentic construction and the
[9:52] decisions need to be easier to define. So, the object of design is no longer the screen. It is the system itself. Now, everybody knows Gutenberg created the first printing press. The lesser known part of the story is the system he built around it. He didn't create just 26 lowercase and 26 uppercase letters. He created 290 unique characters, different widths,
[10:23] abbreviations, ligatures. And the reason that he did this is because when he typeset it and wanted it to be fully justified, he wanted to make sure that there wouldn't be the weird rivers and lakes of white space that erode the beauty of the final product. Essentially, he was making it both extensible and opinionated enough that it could feel as good as handmade, although it was machine-made. This is what we should be aiming for as well. Now, the old system scaled consistency.
[10:54] But the new system needs to scale intent. We've been working on this at Stripe and you know, have hit a few snags along the way and so we're we're figuring it out. But one of the things we want to do is we want to empower builders to go from prompt to production nearly instantaneously. Like many, we started with an MCP that understood our design documentation. But the results were not great. It wasn't specific enough and it wasn't driving the right outcomes. Three different people could put in the same prompt and get three
[11:25] different results. So, since then we've evolved and now we have created a CLI built on our design system, it makes a harness that makes the AI far more obedient. Now, it's where the builders are building and it consumes the documentation at the right time and place to avoid context rot. Now, importantly, the big difference between this version and a previous design system is it's not just components and atomic parts, but it's actually full templates and flows. So,
[11:56] the system actually knows what our product is supposed to behave like and how it all comes together. Essentially, it's far more opinionated than it's been in the past. We embed these standards into the means of production, which enables a more coherent product. But, this is just the baseline. Christopher Alexander said, "A system can satisfy every rule and still be dead." This brings me to my third recommendation. Refuse to confuse done with good.
[12:30] The filter is gone. It used to be quality filter was essentially built in to every stage of the product development process, even before a project started. We've got 20 ideas and we can only staff one. And then we poked and we prodded and we pruned along the way and products grew somewhat methodically. Well, now we can build 20 ideas in a week. This is awesome in many ways and we can finally skip theoretical meetings
[13:01] talking about hypothetical products and react to the real thing in our hands. But, the filtering that used to be throughout the process now needs to happen post build when it is a lot harder to say no. This is where the role of an editor comes in and it is ultra important in this day and age. Who is doing this in your organization? Who's looking at the end-to-end and understanding whether or not we're actually building a whole. It's a new behavior. We have to unlearn that built means done
[13:34] and that done means good. The most important thing I can say to anybody working on their editing skills is you have to experience it like a user would. Does it actually solve the problem? Is it actually attuned to the way the user thinks about things? Is it actually coherent? A lot of things look good in isolation, but then when you pull it all together into the user journey, it feels a little disconnected. And it's not just about saying yes or
[14:04] no, this is good or it shouldn't go. It's actually about is it even fully formed? How can we push this to completion? We don't want done to be the enemy of good. A very insightful article by Nabeel Qureshi talks about what makes art great. And he says, it's about the unexpected details. Those little surprising things you're like, "Wow, I can't believe they thought of that." And then the deeper meaning that sits behind the surface or the continuous themes that tie it all together to a whole. These are the
[14:34] things that AI is not great at. But knowing what the gaps are in AI help us better navigate that and fill these gaps in. As Nabeel says, "One of the things that so offends us about AI slop is the sense that the details don't matter. The cup is green but may as well have been blue. An editor takes accountability for every decision, which is in many ways every pixel."
[15:05] I'll give you an example from this event. So, my team had the pleasure of designing the event. Stefan worked on this fabulous opening animation. He and the team started with a 3D model of the scene. And then they They it into AI to help with the different animations. Now, here's the first version. It's cool. It's, you know, fun to see the different little parts of it, and it's nice the way it moves. But, if you scrutinize scrutinize it pixel by pixel,
[15:36] something feels off, right? That's not quite the way it should move. It's a little jilted. You kind of want to get more of the scene, right? So, he did it again. And again. And again. 56 times. 56 iterations later, he got to something truly beautiful. Now, it's got more realism, it's got a little bit of life, and it's subtle differences, but you can sense the care. Now,
[16:06] one of the things that so impresses me about this is that if he didn't use AI, he may not have built such a complex scene, or he may not have tried so many different viewpoints. And while it wasn't one-and-done, and certainly took a lot of wherewithal and scrutiny, AI opens the possibility space. Now, this brings me to my last, and definitely my favorite, fourth recommendation. Unleash creativity and artistry. Yes, AI
[16:36] can help us manufacture monotony, but it is also the greatest creative catalyst we've ever had. This matters more now because when everybody and their mother is building something, it is going to be harder and more important to differentiate. And the interfaces of today, chats, charts, CLIs, no way is that the epitome of great interactions in the modern era. There is so much more we can do, and so
[17:06] much more we should do to make things feel generative, alive, responsive, dynamic. We can literally talk to the computers. So, let's branch out. It is time to invent new interfaces. It is time to invent new aesthetics. The best practices have not been written yet. We get to do that. And now we have the tools to do it. It's just like when multi-touch made it possible to do wholly new interactions. Or the
[17:36] synthesizer allowed us to create totally new sounds. AI is allowing all sorts of new creativity. And we're seeing so much of this online. We're even seeing folks showing up the Stripe design team with way more interesting data viz. And we're seeing websites for restaurants with character and personality. And people using AI to paint and create art themselves. It is a really interesting time. And at Stripe we're using AI to essentially
[18:06] amplify the abilities of the creative team. Our more most recent cover for Built to Grow, it was created by a human Marbler who worked on these stunning iterations. And then we used AI to fine-tune the details ever further to ensure we had the colors and the lines in all the right places. It was basically taking the good judgment of the humans and helping us scale it to make something truly stunning. Now, to make AI a stronger
[18:38] creative partner, there's a couple of things I recommend. One, improve your inputs. You want more specific going in the direction of your brand, your unique interests. So, put specific prompts. Don't just say, "Hey, I need a website for my Korean barbecue, but this is what I believe in. This is what good is. This is what we care about." Add your source material that you're using to build your own standards with. Help make it think in different ways. And then stress your outputs.
[19:10] Don't get tempted by the burrito dilemma. Always push a step further. And then, of course, use adversarial agents to help you critique it and bang it up a little bit. But, you yourself should always be pushing for better. Now, these are just tactics. The much harder thing is definitely cultural. It is easy to follow cookie-cutter patterns. It's safe and cozy. But, it is much more impressive and much harder to find something unique that improves the
[19:42] status quo. So, if you're leading a team, don't just tell them to use AI, but give them room to explore. Protect the strange. AI lowers the cost to create. Let's spend some of that savings on making something truly special. If we only use AI to make the things that we already make just faster, then we are definitely missing out on the most interesting part. AI can help us raise the ceiling, not
[20:13] just the floor. The most important thing is to be very intentional. It's your point of view. It's your system. It's your quality bar. And your ambition that you will want to bring to life with AI. In total contrast to the post-war building boom and modernism design is Gothic architecture. In 1850, John Ruskin wrote a lot about
[20:43] quality and craft and highlighted Gothic architecture as the epitome of great. He noticed that no Gothic building was alike. Frankly, not even one column was alike the other. Each detail was uniquely crafted and showed the unique hand and mind of the maker behind it. It felt truly cared for. This is what our users want. They're not impressed if we animated something with 3.js and Blender in 30 minutes.
[21:16] They are impressed by us solving their problems and clever touches in the details that show we anticipated their needs and that our brand has some character behind it. We have the choice in this building boom to not make the digital equivalent of zombie buildings. We can make this a creative renaissance. So, let's make some products that are more powerful and show the hand and care of the maker. Thanks, everybody. >> [applause]