Latest / Serious People Podcast / How to Build an AI Writing System That Sounds Like You | Sally Slater (Cofounder, InnovAItion Partners)

Transcript

How to Build an AI Writing System That Sounds Like You | Sally Slater (Cofounder, InnovAItion Partners) · Serious People Podcast

• Sally Slater (00:00): For anything writing related where there is a storytelling component or an emotive component where you need the writing to move people, you can't rely solely on AI for that. You can use AI in the equation, but you can't automate it end to end. It just fails. It's just not good. And you're seeing, I think, backlash now to AI slopaganda where there's actually a growing premium on human content. Noah Levin (00:23): Welcome to Serious People. I'm Noah Levin. Today, we're talking about one of the most annoying things about AI, why so much of the writing sounds exactly the same and what to do about slopaganda. My guest is Sally Slater, cofounder of InnovAItion Partners. She helps agencies and professional services firms build AI workflows, and she built SlopSentry to catch all the stuff you shouldn't publish. We talk through a writing system for my own podcast and newsletter and what that system actually needs to make it successful. An idea from a human, a clear way to describe your voice, and an editing pass that catches sentences that sound good but don't say anything. We also get into how to build a company brain from information you already have and why automating everything should not be the goal. Here's my conversation with Sally. Sally, thank you for joining Serious People. (01:15): Good to have you. Sally Slater (01:16): Thank you for having me. Noah Levin (01:17): So today you're going to teach me a lot about how agencies use AI. You're going to teach me how to be a good writer, not a bad writer with AI. And we agreed in advance, I'm going to put myself on the line a little bit and let you do what you do, which is coach me and give me your structured advice, put me through your playbook for making my agency a better user of AI and getting my writing in shape because God knows there is some slop that I'm putting out there and I need to rein it in. So you consult for agencies as your main job and you see agencies of all different types come to you and ask you what? What's the question that you are normally trying to help them solve? Sally Slater (01:57): Usually they're coming to me in a little bit of a state of panic and it's, I feel like everyone else is further ahead with AI than I am. What do I need to do right now to catch up? Or how do I stay relevant? My clients are putting pressure on me. Noah Levin (02:13): What's the right question for most agencies to ask or does it vary by agency? Sally Slater (02:17): It really varies by agency and I usually like to start with goal. And I do think that there is a common misconception that the sole purpose of AI adoption is efficiency. And I usually try to lead people away from solely focusing on how can I cut the amount of time that I spend on something because I think that is a pathway towards total commoditization. And if you can replace everything you do with automating it end to end, at a certain point you cease to be relevant to anybody. And so I try to help them understand what are the things that they do that are truly very unique, what is their special sauce and how can we use AI to accelerate that or to create something totally net new in addition to realizing the efficiencies? The efficiencies don't go away, we definitely need them. But if that is all you're striving to do with AI, I think you've got a pretty short runway before nobody will need you anymore. Noah Levin (03:10): What's an example of an agency? Give us a specific type of agency or instance that you worked with and what's the thing that you think is irreducibly human versus something that's a good candidate for being structured as part of an AI workflow or tool? Sally Slater (03:25): Yeah, it's interesting because you tend to think of agencies. The agencies I work with are PR, marketing, digital marketing, advertising. You tend to think of them as writers, designers, et cetera, but they're also very much consultants in terms of what is the strategy you should take basically to persuade people to behave differently, whatever that group of stakeholders may be. It might be customers, it might be internal employees, could be investors. The goal is to convince them to do something differently, whatever goal that may be. And that requires a deep understanding of human psychology. It requires creativity. It requires experience having done something like this before. AI is not great at that. You can give some suggestions, you can use synthetic personas as an aid, but I don't think you can remove humans from that strategy portion. And I don't know if this is forever going to be the case. (04:19): I do think for anything writing related where there is a storytelling component or an emotive component where you need the writing to move people, you can't rely solely on AI for that. You can use AI in the equation, but you can't automate it end to end. It just fails. It's just not good. And you're seeing, I think, backlash now to AI slopaganda where there's actually a growing premium on human content. So you have to strike that balance between not contributing to the AI slop problem while also harnessing the efficiencies that your clients, your investors now expect. And Noah Levin (04:54): I know we're going to get into some creative workflows in a little bit. Let's offer me up as a client for you and I run Serious People. I wore the hat today. Sally Slater (05:04): I Noah Levin (05:04): Like Sally Slater (05:04): It. Noah Levin (05:04): You can tell it's an LLC, so hopefully I never have to convert to S Corp. And so I do AI consulting for operators. I help businesses figure out where to apply AI, a little bit meta because I know that's what we're doing here. I come to you as an agency. What's the first step in working with you? What's the right workflow that we're going to work on together? Sally Slater (05:22): Well, I first usually want to understand what is it that you're doing with AI right now and where is it working well for you and where isn't it? Or what's on your wishlist? What do you wish you could transform that you haven't yet? Noah Levin (05:34): Well, I'll tell you what's on my mind today. My entire workflow end-to-end is AI inserted and everywhere. But for this podcast, I would very much like for most of it to be automated. I love having this conversation with you. I'm obviously not going to be replacing you with an avatar or a Genmoji or whatever. We posted the episode from last week and I haven't posted it on LinkedIn yet. I would like that to be happening for me in the background. And that's sort of a recurring theme of promotion and just getting the word out there about the podcast, editing the podcast. There's lots that I feel like could be more automated than it is. Sally Slater (06:09): So that is a fabulous example of where AI I think can be helpful without much human in the loop, especially because you are starting from a human source. And that to me is key for any content related workflow is that somewhere in the equation has to be a kernel of human insight. It can't just be AI end-to-end. And this is the portion for you that is the kernel of human insight. This is all real conversation. This is real me, I think. This is real you, I think. So you have actual words that are not coming from a bot that you can leverage across any channel that you can think of. I do actually have to come at it somewhat like a marketer and think about who your target audience is, where are the channels that they're active? Have you already identified all the parts or all the channels and formats that you want to repurpose the podcast into? (06:59): Does that still need to be figured out in terms of what is the distribution strategy? Noah Levin (07:03): I mean, I certainly have ideas, but I'll put it this way. If I had spare change lying around, I would love to hire a podcast producer and not think about any of this. I know that it was being done perfectly. Sally Slater (07:14): You could just ask AI to come up with the strategy and identify all the channels, but you may not have the experience of, well, yes, this has worked well, this hasn't. This is a realistic cadence in how often you should publish on LinkedIn versus Twitter and X, et cetera. I do think that there is sort of no replacement for expertise, but I can tell you how I would do research if this were something that I were coming to totally fresh. Noah Levin (07:41): So is this workflow is fair game you think for something that should have AI applied to it? Sally Slater (07:46): Yes, absolutely. Because you have the transcript and you can repurpose that transcript in 10 million different ways. Noah Levin (07:52): So when you're evaluating use cases for AI, it sounds like what you're looking for is, is there something built into that workflow that is sort of the core IP or the human seed of something interesting or useful or new that AI can operate on? Is that right? Sally Slater (08:09): I would say that's true. 90% of the time there is a 10% window where something just is so rote that I think it's fine to totally automate. So for example, this may be controversial. I think press releases, if you've ever seen those come out from a company, I think those are the most rote formulaic things in the history of time. One of the challenges with AI is probably at this point most people know is that there's what they call either claw-ish or claudish, pick your preference for how you talk about the way that AI speaks where it talks in these really kind of same, same patterns that becomes the more you look at it or the more you can recognize it and you stop sounding like yourself and transcripts help the AI break out of it. So that's why I like them. It's probably one of my top sources for that kernel of human insight. (08:57): I'm a big fan of AI interviewing you to get thoughts outside of your head. I think it's another great way, especially if you're... I think a lot of people struggle with blank page syndrome or just not knowing where to start or even how to come up with an idea. An AI interviewer is great for doing that. The one that might be more controversial is I think original data can also be a source for automated content if that data is truly original. (09:28): If the concept for the data and the survey or the study or whatever you did is human led, having that original data enables more original content so that you're not just reproducing something that's been said by somebody else. Noah Levin (09:43): So Sally, one of the things I know you spend a lot of time thinking about is creative workflows and how to take creative output of an agency. For me, it might be this podcast or I send an email every week with the podcast and a few other nuggets and how you can take that and apply AI, keeping the human in the loop in the right way. So tell me how you think about that and maybe we can go into my podcast or how about this? We'll do the Substack email that I want to send with this podcast going out at the same time. We can hit that as an example. Sally Slater (10:12): Sure. I do always start by asking, do you already have a Substack format in mind or do you need to come up with what that looks like? Noah Levin (10:21): I do. I kept it really easy. There's a guy named James Clear, the Atomic Habits guy, and he figured out that if he gave himself a really easy to accomplish format for his newsletter, he would actually send it every week. So mine is something to listen to, which shockingly is almost always the podcast, something to read with a little bit of color commentary on it and something to try. Sally Slater (10:42): And do you ideally want it fully automated where it's finding all of those things for you and you're not even signing off? Or is there a point where you want to have, let's say, a list of possible selections for something you're going to read and then you say yes, that one? I Noah Levin (10:56): Think it has to be the second one because I don't think I'm fully comfortable putting my name on something that is just coming out of the machine. I feel accountable for my own work in this sort of meta way where if I'm contributing to the problem, then what am I actually teaching people? But I really won't do it if I have to write it from a blank page every week. So I do need help. Sally Slater (11:17): Okay, that's helpful. This actually reminds me of something I do for myself where I send an AI digest to my clients every week and it's actually personalized by a client and I have it working in three layers. The first layer is it does a search and it goes out and finds the possibly relevant stories, which for you would fall into those three categories. Before it does anything else, including figuring out what the heck is interesting about it, I confirm, yes, these are the right stories, which is what I would have you do too. Here's the categories of things. Is this the right fit to cover in the newsletter? Yes or no? And it's simple yes or no. Obviously, you want to build in guardrails, past examples, teach it what specifically does make sense to look for. The more past examples you can give it, the more you can tell it what you don't ever want to see, the better it's going to perform. (12:06): And over time, it should get pretty good at actually predicting what you are going to say yes to. So that's step one is just even getting to the right stories for your Substack. The second piece is figuring out what the heck you're going to say about it. Now, I haven't seen your exact format, so I don't know how much text you have or how much additional your own insight you put into it. Well, let me pause and just ask you, how much additional insight are you adding on top of the links that you're sharing? Noah Levin (12:34): Why don't we look at an example? Sally Slater (12:35): That's Noah Levin (12:36): A good idea. Let's go to the board. Sally Slater (12:37): Amazing. Noah Levin (12:38): So here's the last email I sent from, I think this was actually two weeks ago. So something to watch where we're mostly looking at the show notes, something to read. So what I do is actually sounds like a variant on what you're describing, which is I bookmark things on X all week, and so I have my agent go and recommend a couple of the things I bookmarked as possible things to surface here, and that's sort of the seed for this section. And then something to try is usually something that comes out of a lesson from one of my client conversations. Sally Slater (13:11): Got it. One idea on this last piece, and you may already be doing it, do you keep all of your meeting transcripts somewhere centralized? Noah Levin (13:19): I do. I use Granola. Sally Slater (13:21): Well, the challenge with Granola is that you can sort of search within the Granola tool, but the only way to get your AI to access the Granola transcripts is say, "Hey, Granola, go find and look up transcripts," which you can do it every day, but it's not storing them anywhere. I like to actually store them somewhere and tag them thematically, automatically obviously. But the reason is that if I want to look back from the past week and say, "Find any lessons from my transcripts," it's much easier for AI to go and find them. And I don't know if that's similar to the approach that you're taking or you just know off the top of your Noah Levin (13:55): Head. I've had a lot of success with the Granola MCP server. My agent knows to go look in the archives in Granola and find what it needs typically. I think that's most of the time what it's doing. I have a separate process running in the background where every hour or so it scans, it being ChatGPT, scans all my inboxes, including Granola, and starts sending little context notes to all of my persistent threads. So the session on Sally Slater is going to get a great record of this conversation shortly after we're done. Sally Slater (14:23): I do something similar. I do think that it can be helpful to just store them somewhere and again, tag them. If you wanted things automatically tagged, a lesson was said during this meeting, you could do that, but it sounds like you've got a good flow going with the MCP server, but are you prompting it when you're writing this or is this already something that is automated to some degree in terms of when you're coming up with that lesson? Noah Levin (14:46): Well, I don't want to lead the witness, but I'll tell you what, I got some advice a while ago that sounds a lot like what you're nudging me towards, which is that you want me in the loop somewhere on this. And so every Wednesday I take the dog for a walk in the morning and I've gotten in the habit of having a little rant to my phone while I'm on the walk about all of my thoughts on whatever article it is we're going to post into something to read topic. And the instruction that my AI has is try and take as much verbatim out of that conversation as you can, but structure it as something that's coherent and not me trying to catch my breath while walking up a hill with two dogs. Sally Slater (15:23): I think that's a great approach. I will say if your podcast is being linked, I'm personally fine with pulling insights from the podcast without you having to provide additional context. But if you're already doing that anyway, there's so much rich material that you can give to AI and it can draw on in future sessions and really understand your voice and pull in relevant anecdotes. What some people like to do is they'll have an automation where if you typically wrap your day at 6:00 PM, that you just have an automated question or series of questions and it's like three questions. What's one interesting thing you learned for the day today? What's one thing you wish you did differently? It's very different when you're an agency and you're dealing with clients. And I will say there's this notion in agency land of thought leadership and that you're working with SMEs who are thought leaders. (16:11): Not all thought leaders have thoughts. And so that is a challenge that every agency has to deal with where they are not getting good material from the actual supposed experts. Noah Levin (16:26): That sounds like a deeper problem than an AI problem. Sally Slater (16:29): It is, but you remain responsible for producing thought leadership without the thoughts. And so that is something where AI can actually be helpful. Noah Levin (16:36): You mentioned that having proprietary data is another source of content generation. Say more about that. What does it look like to have your own proprietary data and to use it in a creative workflow? Sally Slater (16:46): This can take lots of different forms. If I'm talking about an agency setting, let's say you are doing some form of advertising or social media across multiple clients, you have that performance data, and if you aggregate it, especially if you're focusing in on a specific industry vertical or maybe you're focusing in on a specific channel or format, you have insights that literally nobody else has access to because only you have access to all of your different clients' data. Now, I don't want to get into the legalese of what your clients say you have permission to do with that data. Obviously to do any form of analysis, you would need to anonymize the data, but that's a rich source that you could then analyze and use to fuel insights about best practices about what's working and what's not that AI doesn't have because AI is not able to read this private information. (17:37): It's just one example. It could be you have access to all of these conversations that you've been having across all your clients for the last year. You can even apply some form of thematic data analysis to the conversations you're having, what's coming up the most often across all of your clients. I think it's just coming up with interesting ways to spend data that is private or proprietary to you. It could be that you're doing a formal survey, just sort of depends what you have access Noah Levin (18:04): To. For writing workflow, like the one that we're talking about for my newsletter, where are you interacting with AI? Is this happening just in a ChatGPT or Claude session? Sally Slater (18:14): If I were personally writing Noah Levin (18:16): Your - If you were implementing this for an agency and telling them how to build their weekly newsletter process, is this all a skill that lives inside of my normal agentic harness or is there a separate tool I should be using? Sally Slater (18:27): I would say that this is probably primarily living inside whatever platform you're in. Noah Levin (18:32): So you pick the right model, you're doing it in the place where you normally do your AI work, either in Claude or ChatGPT, depending on which model you're using, but you said you have to use it a certain way. So we've talked about putting in your own thoughts and sometimes having it interview you. What else are you telling your agent so that it produces usable writing at the end of the day? Sally Slater (18:54): So I think of writing as kind of a sandwich or a hamburger, pick your analogy. Let's go with a hamburger, that's more fun. Where the top bun is how you want to sound. So this is the proactive positive, I want to sound warm, funny, whatever adjectives you want to throw out there. The meat, that's what you want to say. And that's to me the part that's most important has that human kernel of insight in it. And we've talked a little bit about the formats I can take. And then the bottom bun is how you don't want to sound. All of the AI-isms, if you work in a corporate setting, you might have a bunch of legal words that you can't say. That's the bottom piece. And people try to do all three things, like trying to eat the entire hamburger in one go, and AI doesn't work that way. (19:43): You need to separate those pieces. My personal preference is, and what I've seen work best, is that you need a brand voice defined per channel. And I say this because my guess is the way that you're talking on this podcast may not be a hundred percent of the same way that you're talking on, let's say, your LinkedIn company page. Your personal LinkedIn may sound pretty similar, but maybe you're also posting on Twitter or you're writing a formal article for something. All of those have a little bit of a unique spin on the way that you write and the way that you want to sound, and so you just can't have one giant brand voice to roll them all. You need to really get down into the weeds of here's how I want to sound per channel. If that sounds like it's really intimidating, this is where I think AI can be really helpful in analyzing writing samples by channel. (20:35): The thing that I see a lot of people do wrong is they're solely focusing on the soft squishy stuff, the stuff that you might see in a style guide, which is again the warmth, how funny they are. Is it casual? Is it formal? AI responds really well to hard targets, to math and to numbers. And so I'm a big fan of this concept of stylometry, which is the math of writing and essentially measuring how often do you use adjectives for a hundred words? How often do you stack those adjectives together? What about adverbs? What's the coefficient of variance between your sentences? People often say, write long sentences and write short sentences. That's not helpful. AI is going to do that on its own. It's more often how frequently are these bundled together? You can get pretty specific as long as you have five samples of writing and get that math and that range and give AI those targets. (21:30): And you're going to find that that's going to do far more than the squishy soft stuff in making AI sound like you. So I do think it is worth the investment of time to create a really good brand voice, again, by channel for the places where you're active. And obviously I use a skill to do this, to be very clear. I have a voice reference building skill and it does all this stylometry math for me. The meat is the part that comes from you in some shape or form. Could be this podcast transcript, could be original data, could be AI interviewing you, could be research that you then react to. Lots of different ways to get to the meat, keep the human insight. The bottom bun is extracting out all of the horrible crap that AI does, the slop sounding words. I track, I think at this point, 600 plus different words and phrases that AI uses ad nauseum. (22:23): That's just one layer of it. That's the obvious AI tells. One thing that I hate as a writer that AI does is they write sentences that sound beautiful or sound profound, but if you look closer at it, you're like, I literally have no idea what the hell it is that you are saying. This sentence sounds really beautiful. I could see it on a motivational poster, but it doesn't actually mean anything. And so you need to help AI write in a more straightforward plain way. It works a lot better if you have AI focus on how to sound in the positive and then perform a self-check against all the things that you don't want to do. And you don't want the same agent doing the positive affirmative writing to also do the check. It's kind of the same thing as real writers and editors. There's a reason why every author has an editor, and that's because you're not particularly good at editing your own work. (23:19): So that's why I like to view it in that layer of you separate how you want to sound from how you don't want to sound. And that last piece is a follow-on check to the first piece. Noah Levin (23:28): You're a bit of a slop expert, I think, and you've gotten passionate enough on this topic to the point where you actually built a product to play the role of that editor agent to pull out slop. Tell people about SlopSentry, but also how does it work? Sally Slater (23:44): There are a lot of tools out there that they'll call humanizer tools. And my general issue with them is that I think they only focus on one component of writing slop, but I view writing slop in two main categories. One is sort of the lexical slop, the actual words and how they sound. That is where most people focus. That's the obsession with em dashes, which Lord knows I love an em dash, so I have not totally bought into that. But here's the thing, it's not about X, it's about Y. And some of that is really hard to get out. And so it absolutely takes care of that. I will say that I think we have maintained the largest list of tells and remove the largest list of tells compared to any other tool you'll find out there. The second piece that is less talked about is what we call substance slop. (24:35): And that's what I was saying before. It's the lipstick on a pig. I don't care how beautiful the words are. If the words don't say anything, it does not matter. And so the way that this will actually work is our agents will go through the writing and they will ask themselves, "Can I follow this?" And if the answer is no, they will either say, "This sentence goes," because it literally says nothing, or they will rewrite it in such a way that it is straightforward and you can follow it. It also flags things like contradictions because AI does, especially if you're using AI and editing over and over again, up here you might say two thirds of humans do blah, and down here you say three-fourths do it. So it'll catch things like that. But to me, the substance slop is far more insidious than the language slop because again, if you're just not making any sense, who cares if the writing is beautiful? Noah Levin (25:24): Yeah. I think that AI slop sometimes has a lorem ipsum quality to it where it compiles as a sentence, but there's nothing really there. And the way I have learned to look for AI slop on LinkedIn is if you actually take your glasses off and relax your eyes a little bit and just look at the blurred out shape of a post, you can almost always spot something that's kind of got the right line breaks and the right sentence structure for it to be an AI written LinkedIn post. Sally Slater (25:53): That is a hundred percent true. We actually built a separate mode in just for LinkedIn because LinkedIn has these unique Noah Levin (26:01): Patterns. SlopSentry has met its match in LinkedIn. Yes. Sally Slater (26:04): We did have to make a separate mode for it. I like to call it LinkedIn broetry. I don't know if I coined that or if somebody else did, but it's those sort of single sentence lines that just sound kind of profound-ish, but not really. That drives me crazy. Noah Levin (26:21): So we've talked through the hamburger of how to write creatively, how to take your style, your brand style, how to bring that to bear on something you actually think and believe as a person, some of your own personal IP, how to remove the slop, and now you've got your creative output. So this whole thing end-to-end constitutes one of the workflows that you would be building for an agency? Sally Slater (26:43): Yeah. So I tend to think of the voice and the compliance check as part of a broader system because they can be applied across the board. There might be multiple different ways that you mix and mash or you get to that meat, but the brand voice is always the same per channel or per individual, and the compliance is by and large always the same. So those are static in your system. It's then a question of how you want to compile. There are different ways to go about approaching the actual automated content workflow. So going way back to your podcast here and your news, your Substack coming out of it, there might be an automated research workflow that's leading into the creation of content. There is a template that you are creating that's used over and over again. There may be additional checks that you build in, for example, crosschecking against past issues of your Substack so that it's not repeating stories or lessons that you've already told. (27:36): All of that can be built in and automated, and so that's where you add in additional layers. Noah Levin (27:40): What do you do to structure a workflow like this that's happening every week, every day, multiple times a day, if it's happening across a team and there's a right way to do it at the agency? Sally Slater (27:51): So this is where you also have to know how to create the right context down to what does your client care about? Who is your client? Who is this being written for your client? Is it for the brand? Is it for the individuals? Yet even complex within your client because you might have 10 different spokespeople, more thought leaders that you're working with, and so you need to be able to create and switch between those different thought leaders without having to rebuild an entirely new work. It would be really annoying. Let's say you're an agency, you've got 50 clients and you do the same type of work for all 50 of them. You're realistically not going to create 50 different workflows that would be a lot of time and effort. You can of course just duplicate them, but if you're thinking about each component in terms of variables, like brand voice is a good example where you can just swap in and out the brand voice for the individual, for the client, et cetera, but you also need to be able to switch out the context in terms of what are the themes and the topics that they care about. (28:47): So what you have to try to do is figure out the connective tissue between whatever it is that multiple people are doing and to some degree standardize it so that the process always works the same way and figure out how many variables you can swap in and out. Noah Levin (29:03): These are pieces of what you've called a company brain or agency brain? Sally Slater (29:06): They can be. So most agencies that I work with and that are working in, let's say Claude or ChatGPT, one of the biggest issues is sharing and collaborating and creating consistent content or context for clients across teams. So you might've created a folder over here for Apple with a bunch of information on who they are, their messaging, what they care about. And this other person over here, maybe they have a folder that they've created, but the two are not synced. You are working off of different definitions of who that client is and what they care about, and that is a really inefficient and way to run and also leads to a lot of inconsistencies in quality. I do think that at some point, ChatGPT is going to figure out that they need to add a better collaboration layer for knowledge sharing than they have already. (29:57): Claude just came out with documents, which at least allows collaboration in the same document, but it's still not great. So with the notion of creating an agency brain, you are figuring out how do I build the right knowledge that is shared across my team that is also easily read by AI? And that sounds like pretty basic, except for the fact that you do have to understand the nuances of how do you get AI to access the right information at the right time so that you're not constantly having to say, okay, now I'm working with this client, let me go find this information. You want it to be able to just pull that in, otherwise it's creating a lot of extra work for yourself. Noah Levin (30:34): I run an agency, I work with clients, I do work for these clients. Talk me through what I would need to do to build a company brain for the type of business I run. How do I find the context? How do I structure it? What system does it actually live in? Sally Slater (30:47): Yeah. So I would say it depends on how techie you are. For you, you're pretty techie. So I would probably be building this in GitHub. GitHub can be used and it's meant originally for coding, but it's a great tool for enabling version control and collaboration. So for a techie AI native company, I think GitHub can be the place that facilitates here's how I'm going to control my contacts and make sure that I understand the versions, et cetera. And AI can obviously access GitHub content really well. And then the files live locally, you're managing the versions through GitHub. That doesn't make sense for 90% of agencies. You actually can use Google Drive or SharePoint if you structure your permissions correctly because that's often the file system that people are already living in and that's a good place to start. Where that will fall short is if version control becomes an issue. (31:42): The argument I often make there though is that version control is already an issue. It's no different. You are already overlapping over each other on these documents. So that's sometimes the first place to start is honestly just restructuring your files in either SharePoint or Google Drive, I suppose Dropbox too, I don't have any clients that work in Dropbox, into a format that AI can understand and also makes sense to your humans. I'm a big fan of something that Google came out with called the Open Knowledge Format, and it's basically coming up with a file and folder system that is made for AI to read and find things really quickly. So I'm often building folder systems in that structure, which sounds really boring. Maybe it is a little bit boring, but where it has a little bit of AI behind it is every company I've ever worked at, the SharePoint or the Google Drive has been a complete hot mess. (32:34): It just sprawls and gets worse and terrible. Nobody follows the structure that they're supposed to follow. You can use AI to maintain the system for you. So every time you're adding a new file, AI is figuring out this is where the file goes, this is how I'm going to tag the file so that AI can read it. I'm going to update the index that says this is where that file lives. And I have some kind of cleanup mechanism that's running on a recurring basis that's flagging when, hey, you just added this new document and this document completely contradicts what this document says over here. Which one is the version of truth? It's not going to decide that for you, but it's at least going to flag those inconsistencies so you can make that decision and make sure that you're keeping your context clean. So that's often the simplest place to start for building your first brain. (33:19): There's reasons to get more complex. There's reasons to get into a full-on RAG system, but often for many agencies, it's simple enough to just restructure your SharePoint system to work with how AI likes to work. Noah Levin (33:33): I think the thing that breaks my brain sometimes about these second brains or agency brains or company brains is it's not about how the context is structured. It's ultimately just a bunch of files that are there and are organized in some way, maybe they're indexed, like you said with RAG, but how do they get updated? How do I pull the context from the world where the work is happening into my brain? Sally Slater (33:57): In terms of automatically updating, I am a big fan of scheduled tasks using skills. That's something that I do with my own personal company brain that I build for clients' brains. You decide what information is relevant to continually update the client. That might be, I usually will create a meetings transcripts folder on the client level that will pull in from Zoom, whatever, name your transcription tool, we'll pull in that context, index it, find themes, surface best practices, surface pain points. One of my favorite things to do is actually have a meetings dashboard where once a week it will automatically update what those pain points and opportunities are across all of your meetings transcripts for your client. And you can also do it agency-wide if you want and have a little LLM summary of what's changed in the last week. I think that's a really great use case, but those meetings are being auto filed and auto tagged. (34:54): Same thing if you want, you can actually do that for emails as well, is obviously you can use an MCP to pull it whenever you want and update who the client is, et cetera. But you may also find it valuable to actually keep all your threads in your folder system and apply analysis on top. It's just a matter of what types of insights are useful. I do also like to layer on external signals. So what's going on in the competitor environment and AI is fabulous at doing competitor research across a wide variety of different channels. So usually that's a component too where you start off in however you're structuring your brain. Here are the client's competitors. Maybe you have something like a SWOT, maybe you have profiles of the competitors. What are they doing that is relevant to the work that your client does? And that can just auto update on its own. (35:42): If you use skills or agents or workflows in the right way, that can all kind of be auto updated. If you are relying on SharePoint or Google Drive, the same rule applies where the folders you have access to are the folders you have access to. If you don't have access to a folder, it doesn't matter that somebody else is given access to Claude, you're not going to see it and vice versa. So you can rely on those mechanisms to work on those folders. I sometimes will create a private operating system that's designed to work with the broader brain when it's truly meant to be down to the individual layer and I'll create what I call a promote to public skill. And basically you proactively decide, okay, I've been working on this privately, help me think through which components of this I can actually promote to the public operating system because some components of it are and it'll help you walk through that without totally screwing up and putting the wrong information in there. Noah Levin (36:42): One sort of cheat code for this is if you're doing most of your work through ChatGPT or Claude on your computer, then by default, the tools that you're connected to are all tools that you have access to and every user at the company is inheriting like, I can only see my email, I can only see Google Drive documents that I can see. And so if you change the sharing permissions on those things, then you're kind of updating what the company brain effectively is. It gets way harder when you start to build systems that kind of live independently because then you start to have to make these judgment calls about in what context are they operating and what should they be allowed to see? That's a lot more of a custom development type of scenario. The thing that I think is not always intuitive about why bother doing a company brain is if you have access to email and you have access to Google Drive documents directly, why do you need to bother indexing them? (37:34): And sometimes the index is enough and sometimes it tells it where to go to find the primary source, but you're kind of giving it a cheat code on this open book test where it can much more efficiently find and navigate to the information that it needs. Sally Slater (37:47): 100% agree. It does actually end up saving you a lot of tokens because rather than having to search through every single folder ever to find whatever it is that you said you were looking for, it's given basically a map to find the right information at the right time. Noah Levin (38:01): We talked about ChatGPT and Claude as being kind of two load bearing pillars of a lot of people's stack. Are there any other tools that you think agencies should be bringing to the table for typically use? Sally Slater (38:15): One of my favorite platforms is one called Suits, suits.ai, and it's built basically entirely for agencies. And there's a couple reasons I like it, one of which is that you don't have to choose between ChatGPT or Claude. You get access to Claude, OpenAI, and Gemini models right now. If you're locked into one of these models or one of these platforms, it is harder to do that. It's not to say it's not doable. There's workarounds, especially if you're operating in either Codex or Claude Code, but unless you're in coding, not that many agencies are comfortable with that. So I do love a platform where you don't have to choose, but the other thing I really like about it is all of this notion of building an agency brain. If you're working in either Claude or ChatGPT, you have to kind of build them from scratch. They're not giving you an out of the box model where you can say, "Okay, I'll just put my files over here. (39:05): Here's how it's structured." You kind of have to make it up on your own. Suits takes care of that for you. They have a structure where every single client gets its own knowledge base and its own brand voices and its own personas. So a lot of the thinking, a lot of the context engineering that you would have to stand up from scratch in other platforms is just done. All Noah Levin (39:26): Right. Enough about you. Let's go back to me. Perfect. So I have this podcast. We're going to have this great long form interview that is going to have to get edited into a thing, and then it's going to have to be turned into a newsletter and a bunch of pieces of content. If I'm your client, how are you helping me set up my creative workflow for what happens after we hit stop recording on this interview? Sally Slater (39:48): I would say let's build a workflow that accounts for all of the channels and formats where you want to distribute this content, and it should automatically take this transcript and restructure it into those different formats hyper-specific to the best practices for those channels. So that's part of what the build process would be is fortunately, I guess fortunately or unfortunately, I happen to know a lot of those best practices having worked in digital marketing for a very long time, but if you happen to have an opinion on it, I would want your opinion Noah Levin (40:20): On - Do you want to share some alpha right now with the people on this call? What are the dos and don'ts of LinkedIn and Twitter, which are the two that I'm active on? Sally Slater (40:28): Yeah. So people way over index on company pages on LinkedIn, but the reality is that the mileage that you get from individuals is far, far further and there's all sorts of data to back this up. So I would much rather see more time and energy be spent on whatever comes out of your LinkedIn than your company page. The other thing that's kind of interesting is that everyone assumes video is going to outperform literally everything. That is true on virtually every other channel except for LinkedIn. For Twitter, if you can't, and I'm sorry, I'm going to call it Twitter. Millennial, it's Twitter. I know it's X, it's Twitter. If you can juice the algorithm a little bit on Twitter by finding tweets first that are relevant to your expertise that you can respond to before you do your own post, that you're going to get better results. (41:17): This is actually true on LinkedIn too. Increasingly, you're commenting ways into how the algorithm chooses to distribute your posts. So if you were really trying to game the system, what you would do is use AI to find, let's say, three relevant posts from people in your niche and in the 15 to 30 minutes before your post goes live, comment on their stuff. Now I have very strong opinions about automating the actual writing of comments. I would generally advise against it. I don't think it's wise, but it goes a really long way in terms of boosting the visibility of your own posts, which you can automate that from the transcript. But that's the true secret sauce on both those platforms is that it's all about engagement. Noah Levin (42:03): So when you build a workflow for your clients and you structure these steps of taking the raw material and turning it into content per channel, how do you surface it up to the clients? Is it a schedule that runs and then they check in ChatGPT to see what it's produced and respond to it in line in the thread? Sally Slater (42:20): I would say that it depends on the client because these are their workflows, right? It's like however they need to serve the client and whatever format their client expects it in. So sometimes to your point, a workflow might be something like it's surfacing relevant ideas or every time there's a trigger that happens where the video isn't uploaded to some site somewhere, that becomes the automated trigger to then run the workflow that takes the transcript, transforms it into all of these different formats, and then they get their choice, email, Slack, Teams notification, "Hey, this is ready for you to look at." 99% of the time, it's just either in the chat or sometimes people prefer a Word or Google Doc because I don't have any agency clients where that's where it stops, they edit it. There's always that layer of human editing because this is what they're getting paid for. (43:07): It's not on their own behalf. They have to make sure that they're giving tough, top quality content to their clients. So they're going to edit it in a Word doc. They often have to send that Word doc to their client. And so we've actually thought about should we create some kind of cool interface that would enable agents to interact with agents when reviewing this content or just enable new ways of collaboration? The reality is you're lucky if you can get your client to actually respond in a digital format. Sometimes they'll take out a red pen and be like, "Here's a PDF." So you are somewhat beholden format-wise, so whatever your client wants to do, which is a niche of if you're working in agency land. Obviously, if you're a founder or it's for yourself, you have total control over what format is most helpful to you. (43:52): I'm a big fan of HTML and I find it a little bit easier to visualize. Sometimes I like to see it in situ and what it would look like as an actual LinkedIn post, but it's a matter of preference. Noah Levin (44:04): So a client has edited the artifact, they've sent it off to the client, they get to go home and sleep at night. What's the learning loop? How do you get better at this over time after you're kind of correcting and nudging and editing what the agent puts out every week? Sally Slater (44:20): There's a few different approaches that I take or sometimes advise clients to take. I like to build in a learnings.md file, and I'll usually do that at a few different layers. I'll do it at the project layer where, especially if they're actively working on it and they keep yelling at AI because it's doing something bad, that it captures those lessons there. And then in an ideal world, which not everybody does, they've completed the project. Yay. I have a skill called complete project, and what it will do is it will look at that lessons.md file and it's going to look at lessons.md files across other similar projects. And if there is, let's say, three or more of similar lessons, it will elevate that project, those individual project lessons into something that lives across all of those project types going forward. That's just one way that I find that's helpful so that it benefits the whole team and lives beyond just that single individual project. (45:16): I do also think that's the beauty of skills and training people to take an iterative approach to them. In my opinion, a good skill is never done. You should constantly be tweaking it based on things that are not going well. And if you don't have time in that moment, at least leaving behind a log of future things to fix. But to the extent that I can automate it through that lessons.md file, I do find that works really well because the only thing you have to remember to say is this project is done. Noah Levin (45:47): Sally, this has been fantastic. If folks want to get ahold of you, where do they go and how do they support you? Sally Slater (45:52): The best way to reach me is going to be my email or on LinkedIn. I'm pretty active there as we've kind of alluded to. My email, I'll just say it out loud if that works. It's my first name, Sally. And then here's the weird way to spell it. Spell the word innovation. Everyone knows how to spell the word innovation, and then add an I after the A in innovation, partners.com. So that's I-N-N-O-V-A-I-T-I-O-N partners.com. And if you're interested in trying out SlopSentry, which is still in beta, there is a free version. You can just go to slopsentry.ai and sign up for it there. Noah Levin (46:27): I am now using SlopSentry and I will be using it for the newsletter for this week. Thanks for building that. That's very, very helpful. Sally Slater (46:33): Awesome. I look forward to your feedback on it. Noah Levin (46:35): Sally, thank you for coming on the pod. Thanks for sharing so much wisdom, and we'll see you soon. Sally Slater (46:39): Great. Thanks for having me. Noah Levin (46:40): Thanks for listening to Serious People. If you want more conversations like this in your inbox every week, subscribe to Serious People on Substack at seriouspeopleai.substack.com. You'll get the episodes and practical notes in your inbox along with something to read and something to try. And if you're trying to make AI useful inside your own business, Serious People can help you find the right work, build the system, and put it into production with your team. Find us at seriouspeople.ai.