Latest / The Search Signal / A Short History of How AI Search Was Developed

Transcript

A Short History of How AI Search Was Developed · The Search Signal

• Michael Transon: Hey, welcome to The Search Signal. It's good to have you here today. My name is Michael Transon. I'm going to be your host, and I am also the CEO and founder of a search marketing agency called Victorious. So today's episode of The Search Signal is going to be a little bit different than the last few we have run together. Today, we are going to do a deep dive and we're going to jump into the history of large language models. We're going to talk about how tools like ChatGPT and Google search are built to understand how the world works and everything inside of it. So as marketers, this is something that we sometimes take for granted. We just assume how these tools work and we understand we know how they work. And ⁓ important we actually do know how these things work because without understanding how they function, you can't really build a plan to perform better inside of them. So I think it's a great practice to go back to first principles. It's a good way to get our arms and heads wrapped around how these things tend to work, how they were invented, and then also importantly, how they actually operate today, ⁓ and to able to look at that on the most fundamental level. So if you've ever asked yourself, how does ChatGPT work or how does it even understand the questions I ask and how does it know how to give these answers, this is a great podcast episode for you. So let's get into it. So go right now, and if you were to ask ChatGPT or whatever your favorite LLM is, what your company does, it's probably going to take a second and it will give you an answer about you pretty quickly. And if your company is really known at all, you're probably going to get a reasonably accurate answer the first time that you ask. Now, if you were to take that same LLM and ask a more obscure question about either your company or something else. the LLM will probably spend a similar amount of time thinking and then will come back and give you an answer and probably will do so very confidently. And there is a good chance that if the question is obscure enough, while the answer is going to be coming back as confident, it might be completely wrong. And interesting thing about this, which are called hallucinations, is they operate on top of the same system that the LLM uses to give you accurate information. So the question is, why does that happen? And the reason goes back actually a couple of decades, it goes back to two very distinct and different ideas about how humans tackled the challenge of how to get a machine to know anything in the very first place at all. So ⁓ idea that was taken forward is that you build knowledge for computers or machines by hand, which means you write the facts down. And you do it very carefully you write them as And it be like, you know, this thing is a company and it was founded ⁓ on this date. It's got this many employees. It was founded by this person. It competes against these other companies, right? ⁓ This type ⁓ knowledge is structured and it can fact-checked, and it would be way that any scientist or even librarian would And then there's the opposite or other idea that we took to understand or help machines understand how the world works, which is you don't write any facts down at all. And instead, what you do is you feed a machine an enormous amount of text and data, and then let it work out ⁓ statistically what tends to go with what, what correlates with what. And the machine never gets told a fact ⁓ once and is never told a fact directly. It just reads that the pattern starts to look like Knowledge. So these are the two main ideas on how to get machines to understand how the world works. And they grew up separately for about 40 years. ⁓ One of them became something called Google's Knowledge Graph. And then other ⁓ became the large language models behind ChatGPT and all the rest. ⁓ as these have been growing separately for the last 40 years, over the last 12 to 18 months, is these new parts are starting to get welded together. ⁓ the fusion is what is actually answering the questions when you are going to an LLM and asking it about anything that exists in the world. So I want to spend today diving into this with you and walking through both halves of it. And I want to talk about where they came from, how they work, and then what happens when you take these two very different approaches to help machines understand how the world works and put them together. So with that being said, let's jump into it. Okay, so let's start with the hand built side because first off, it's the older of the two options. So I want to take us all the way back to the 1980s. And during that there computer scientists who were looking this problem and basically had if we want a machine to be smart, we have to teach it the world and we have to do it one fact at a time. So there was famous project called Cyc started in 1984 or so. a team literally sat down to hand encode common sense. So this is stuff water is wet, ⁓ and if you've got a kid, older than them as their parent. ⁓ and all sorts of millions of little truths. And these were each individually typed in by people, and it was ⁓ an enormous and painstaking project. And unfortunately, it didn't really quite work the way that they had originally hoped when they started the project, but the framework and paradigm around it stuck around for decades. ⁓ So the paradigm is this knowledge facts, and facts have a shape. There's something, some relationship, and then some other thing, right? Charles Dickens wrote Great Expectations. Paris is the capital of France. Like if you write enough of these just facts down in a consistent format, ⁓ machine can follow the connections between them. The people who do this for a living call this ⁓ a unit. they call it a triple. And it's subject, it's a relationship, and it is an object. And then you can chain ⁓ millions them into this big web. A guy named Tim Berners-Lee, it's the guy who invented the web. ⁓ It, the the inventor of the web, ⁓ was pushing a really early version of this in the early 2000s, and he called it the semantic web. and then the idea being that the whole internet. could be linked together by meaning and not just by links that are pointing at one website to another website. Now the company that took this original idea and actually turned it into something that you've seen was a company called MetaWeb. ⁓ And might not have heard of them, or maybe you have, but around 2005 or so, there was a group, including a computer scientist named Danny Hillis, who started. building exactly this type of thing, right? A giant open database of what I would call entities, right? These are real world stuff, right? And real world stuff with their facts and their relationships, all laid out for machines to read. ⁓ they called that Freebase and it went live in 2007. So anybody could contribute. It was a lot like Wikipedia, but it was structured and it was built for machines and not for people. And then in 2010, Google bought them. And they bought them pretty quietly. And there was not a lot of fanfare. And then about two years after Google bought the company in 2012, you now have the thing that you've seen with your very own eyes, ⁓ if you never knew it by name. Google announced something called the Knowledge Graph. So while there wasn't a lot of fanfare, there was a blog post that Google published announcing this. And when they launched it, it had a line that I would say kind of became famous in our corner of the world. And it said, and I quote, things not strings. Things not strings. Which you know, for up to Google's entire life as a business to this point, ⁓ it said, and I quote, things not strings. Things not strings. Which you know, for up to Google's entire life as a business to this point, ⁓ Search was actually just string matching. You typed in some letters and then Google found pages with those matching letters. The Knowledge Graph, ⁓ when Google bought the technology was Google starting keep a separate model of things themselves and Search was actually just string matching. You typed in some letters and then Google found pages with those matching letters. It had absolutely no idea that the letters, you know, ⁓ L B R O N referred to a person named LeBron or a thing that's out in this world, right? The Knowledge Graph, ⁓ when Google bought the technology was Google starting to keep a separate model of things themselves and it was sitting apart from whatever strings of ⁓ that people happened to type into the it was sitting apart from whatever strings of ⁓ that people happened to type into the search bar. So now if you think about it today, and we'll get to where it's at today, but like if you like were to go to Google search right now and you search company or you search a person, you might see a panel show up on the right side and that panel might include like if it was a company when it was founded and you know who's the CEO of the company. search bar. how big it is, how much money they make. That is Graph. that is Knowledge Graph interacting with you in the search function. So at launch, when Google launched it, they said that there was something like 500 million ⁓ of these individual that the Graph had in its system. And then by 2020, which is eight years later, they were saying there were 500 billion facts and about five billion facts, things or entities. for us as humans, the panel that we see in search is like the visible part, but the more important things actually underneath it, right? Because in the Knowledge Graph system, every entity in there gets a stable, fixed identity. everything has like a permanent ID. And importantly, that's a permanent ID and entity that ⁓ all of other products can point at and use and look at. It's you know, I would say it's the difference between Google knowing that there is a word like apple, right? knowing that it is just a word or that it is not just a word, but it's also a specific company. ⁓ that specific company named Apple ⁓ distinct from the fruit apple, right? The company has an ID. dedicated ID in the Knowledge Graph and the fruit has a dedicated ID in the Knowledge Graph. Even though they have the same word representing both entities, Knowledge Graph can associate them as different things. So it's how the machine that ⁓ keeps your separate from, ⁓ for every other thing out there that shares a name with you, right? For for our own company, Victorious, we've got a dedicated ID in Knowledge Graph that distinctly separates us as Victorious from a mid 2000s Disney show, also named Victorious. It's how it does not conflate us and TV show. So back to the history of how we got to where we're at today, Freebase itself, which is the original open database that was being used, Google eventually wound that down. And they announced that in like 2014 or so, that they were closing it and then they were going to move the data into something called Wikidata, which is, if you're not familiar with it, Wikipedia's structured data sibling site. ⁓ And that thing is very much alive and feeds actually a lot of the systems and search systems that are being used today. But by the time that Freebase had shut its doors in around 2016, it had something like 48 million different topics. And that data didn't disappear or get lost. It moved into an open ecosystem in Wikipedia, which is actually how the AI systems are gathering and reading data and establishing their own understanding of what exists in the world and the relationship between them. Okay, so that is the first way in which humans have taught machines what exists in the world and how it operates. And that first approach was used to create Google's Knowledge Graph and is a main driver for how businesses are found online today using search. Now let's talk about the other way in which humans have taught machines how to work. And this is the one that was used to create LLMs like ChatGPT and other major LLMs that you've used. And it came from a very opposite approach than the one that we talked about with Knowledge Graph. So instead of writing facts down, which was done to help create the Knowledge Graph, what if you never hand a machine a single fact at all? ⁓ instead you just show it a huge amount, an enormous amount of text and data, and then let computer or machine work it out the patterns and work out what it all means on its own. the seed of this idea is actually a very old idea from ⁓ study of linguistics. So there was a guy ⁓ 1955/ 57, named Firth who put it this way. He said, You shall know a word by the company that it keeps, meaning that you can just work out what a word means just by noticing which other words show up around it ⁓ over over and over. And you don't need, in his perspective, a definition. So if a word keeps, for example, turning up near the word purr or a litter box, a machine could theoretically figure out it's something like a cat without anyone ever having to define what a cat is. So that idea, the company that a word keeps, sat around for a very long time, for decades, until computers got big enough ⁓ got performant enough to be able to take idea and run with it. And in 2013, ⁓ there a team at Google that released something called Word2Vec that turned this into actual math. And what they did was they they turned every word into a long string of numbers, which was basically like like a set of coordinates. that words that were used in similar ways landed near each other in a kind of ⁓ ⁓ esoteric mathematical space. So you don't need to know how the coordinates work because meaning actually became geometry. And you could do arithmetic on words that were represented in numbers, which is kind of wild. You could take the, like for example, you take the coordinates for king, you could subtract man, and you can add woman, and then you could land just about where queen is in that mathematical space. So nobody told the computer what a king is, and and it didn't tell them what a what a what a queen is, right? The computer was able to work that out from the company that these words keep, right? So that's how these systems and machines used words and how it they take them one at a time. And for a this was just the ceiling on how machines understood the world because Word2Vec which is that system, gave every word just one fixed set of coordinates. But the result of that is it also gave each word a frozen or specific meaning, right? But like bank account ⁓ and the river bank. Both use the word bank, but they are very different meanings of the word bank. And in this system, the computer would not be able to tell these two applications bank apart. And the reason was because it was missing something and it was missing context, right? To read a word right, you actually have to understand and read against the other words that are also around it. It's context. ⁓ And that was the next advancement for large language models 2017. And it came from another Google team, actually, in a paper that they titled Attention is All That You Need. ⁓ And introduced concept of something called the Transformer. Now, the Transformer is the design every model that you pretty much ever heard of. It is you know, the T in GPT, which is in chatGPT, that T stands for Transformer. ⁓ let me explain what this means. So when a model takes in a sentence, it does not do what a human does, and it does not look at each word ⁓ to right and eat and read each word at a time. What it does instead. is it takes the entire sentence and it ingests it all at once. And then for each of the words that are in the sentence, it asks question, which is what are the other words in here? ⁓ how do they change what ⁓ that individual word, mean? And then it scores that. And each word looks at all the other words ⁓ that in that sentence or paragraph or piece of content and assigns them a weight. And it goes high for the ones that matter or are correlated to it, and then low for the ones that don't. So that scoring, which is each and every word, sizing up each and every other word, is what the word attention in the title of that content is pointing at. It's nothing mystical. It's essentially the model figuring out for each word what other words it needs to pay attention to. So let's use an example. Okay, let's take the sentence. The trophy didn't fit in the suitcase because it was too big. The word in sentence is the problem because does it mean the trophy? Or does it mean the suitcase? ⁓ Right? and I settle on that and understand that instantly because. too big tells you that it has to be the trophy, right? That logic that we use in our minds, that is attention in the model, which is doing the same thing, which is the word it, looking around the sentence and landing most of its weight on the word trophy and then resolving what it refers to. you change the, like, for example, the last two words to too small, right? And we would probably flip the answer what it means to the suitcase, right? The suitcase was too small for the trophy, right? It was too small for the trophy. And a model with attention will flip it too for that, you know, same reason because it's reading it in the context of the words that are pinning it down. this idea of the Transformer just blew out every other competition or approach that came before it on helping machines understand what things are and how they work together and how they are connected. So ⁓ reason is because the older systems they read ⁓ sentence in order, like humans do. And it they read them one word at a time. ⁓ that is actually a slow process. And You can lose the thread over long stretches of text, right? Attention, which is this different approach, lets a language look at all of the words all at the same time and then work out all of the relationships in parallel. And that made it way faster. And in fact, it it fast enough to train on an enormous amount of text. ⁓ And the combination reading everything in context and being able to do it at that scale is what allowed these large language models to be built and everything that we are currently using to be built on. So Google put this to work and its own search program actually pretty quickly. So we look back to 2019. They rolled out a model called BERT, B-E-R-T, into search. ⁓ And the thing that BERT fixed might sound small, but it was huge. Let's go back to that original thing that we have been talking about related to the word bank, right? A riverbank and a bank account. These are completely different things. And the only way to tell which ones somebody means is to have the context of the words around it. ⁓ And then as BERT was launched into Google search in 2019, at the same time, OpenAI was also actively training its language model, its latest language model, in a very similar manner. So by 2020, the dominant or sorry, not dominant, but the upcoming language model that was being used was GPT-3. And that was being trained on a huge slice of the open web and a big, you know, pile of books. But the training itself. was actually a lot more simpler than that sounds. So the way that it worked was the OpenAI would show a model a stretch of text, and then it would take the next ⁓ word and would hide it, and then it would make the model guess what that word would be. ⁓ And then show the model whether it got the word right or whether it got the model wrong, and then it adjusts itself a little bit, a little bit by little. And then you do that, but you do it billions and billions of times over and over and over. I think it was something like 175 billion times for GPT-3. Each time that you do that, it gets a little bit better at guessing what ultimately comes next. And when you think about these large language models, a step back, the big takeaway is that's the whole job to predict the next word. ⁓ Like everything these models do, every answer you have ever gotten out of ChatGPT or Claude or Gemini. It's just this one system working at an enormous scale. It has read so much text ⁓ guessed the next word ⁓ many times that to get good at guessing, it had to pick up other things like grammar and facts and to structure an argument or how people structure recipes, like all of this stuff, right? And it's not because anyone taught the model any of these things directly, but it's because you cannot reliably as a model ⁓ the next word without them, right? So That's how LLMs or large language models work. They function on very, very good guessing about what happens next, right? But how this matters to us, A model that operates like this is never looking anything up at all. So when you're asking it a question, it is not opening up its brain and pulling out a factual answer. It's just predicting word by word, the most likely thing that should come next after your question. And when it has seen, for example, a topic like discussed all over the internet, the most likely next words happen to be true. And it looks like it knows the answer. But when it has not, it still produces the most likely sounding next words. And those can be oftentimes. confidently and completely fluently wrong. Just dead wrong. And that is what people call hallucination. It is not ⁓ to what you might read, it is not the model breaking. It is the model actually doing the exact same thing it has always done, which is predicting the next word and predicting it on a topic where the likely sounding answer and the true answer might not be the same. So that is the second way in which human beings have taught computers and machines how the world works and what is inside of it. It is the way of sending large amounts of data to help machines tweak and guess what word comes next. And it is the predominant way in which large language models function that you use today. It ultimately learns from patterns and does not learn from facts. So its strength is that it can read more than any other model or approach could ever read. And it's very, I would say shockingly fluent across almost anything that you throw at it or any question you ask. But its weakness is it doesn't hold any specific fact you could point to. And you can't go in and correct it because there's nothing in there to correct. That's just the pattern on how they work. Okay, so now you got two systems that we've talked about, and they are basically the opposites, right? One of them knows a lot of specific facts, but only the facts that has been have been given and handed to it. And then the other one has read pretty much everything that exists in the world, but doesn't have a single fact that it has in its system that we can rely on. And when you set like one of them next to the other like that, you realize one is very strong in one area and weak in the other. But the other model is strong in one area and weak in the other. So when you think about strengths and weaknesses, you might want to think about bolting them together. And that has started to happen. So the cleanest version of this actually started in 2020 and it came out of Facebook's AI lab. And they gave it a name that is all over the place. And you might have heard of it before if you have spent any time researching how AI and LLMs work. And that is something called retrieval augmented generation. Or R-A-G or RAG. The idea is a lot simpler than it sounds. So the way that it works is you take the fluent language model, that one that has read everything but makes things up. And right when someone asks it a question, you go and fetch relevant documents and hand them to the model along with the question. So instead of answering from what might be some like fuzzy word and sentence patterns in the model's head, it answers from the text that you just put in front of it. So the researchers that were doing this had two for the two pieces. So the stuff that was baked into the language model's weights, they called that parametric memory, which is the patterns. The documents you go fetch on the fly, they call that memory, which is the lookup. Okay. there's like one brain that is going to reason and write, and then the other brain or side of the brain goes and checks. And the goal is to bring those two things cleanly together. So ⁓ that, I just described, this connection point, this RAG augmented generation, right? More or less, this is what is answering questions in language models ⁓ things like your company today. Right. When you use AI Search now, there is a language model that is doing the talking ⁓ and communicating, but behind it in the systems, there a retrieval step that is doing things like pulling live pages and also structuring facts to help keep the answer accurate and honest. Okay, so let's go back to the thing that we opened with in today's episode, which was, you know, you ask ChatGPT a question about your company, and it might answer in a, you know, a couple seconds, a confident, mostly right answer. And then you could also ask it maybe a more obscure question about your company or even just a random obscure question, and it might make something up completely and couple that with the same confidence that it gave a correct about something else. So going back to that, now you know why, right? Left on its own, a language model is like only ever predicting what is happening in the next word in the sentence. It ⁓ really checks anything. So when an LLM has seen your company like a thousand times and a lot about it from seeing the correlation of words to other words, the responses that it might give to questions about your company are more likely to be true. But when it hasn't seen your company frequently and hasn't seen it correlated to other words, it's going to make some educated guesses, right? Good sounding guesses, and they're going to deliver it in the same exact voice and confidence that it would give any other answer, right? It's the same machine at the end of the day, right? And it's not a glitch. It's just like that's the way that these machines and LLMs are designed to function. So the fix to this, right, and the one that all of these LLMs in large language models landed on to help avoid situations where they are making up information is to stop letting it run on prediction alone, right? So when you're asking a question now, some of these better LLM tools go out and they do a retrieval first, right? They will go and pull a web search, they will grab real pages, ⁓ will pull out structured facts out of a Knowledge Graph, and then they hand all of that to the model ⁓ with your question to give an answer. So the model isn't just you know fishing through a bunch of memory of everything it's ever read on the internet, right? It's answering off of specific documents that are sitting right in front of it that were given to it, right? The predictor that we've talked about and that was invented is still writing the sentence itself, right? It's just no longer allowed to just make facts up. And that's the whole reason why LLMs like Google's AI Overviews ⁓ pages and then you know, Perplexity or chatGPT show you the sources from other sites that it's used, the citations, right? And Gemini leans on Google search and on Google's Knowledge Graph underneath when it pulls its answers. So they're bolting these like fact-keeping brains onto the fluent predicting brain so that the fluent one stops lying, right? And you know, you go to chatGPT and you ask it to describe a company that it knows, right? It's probably gonna give a little, we've talked about this before, a little identifying line that comes back word for word identical every time you ask it. A pure predictor can't do that. It would, you know, word things a little bit differently on each of the runs, kind of like if you were to tell the same story or if I were to record this podcast, I wouldn't say the same things twice, right? Word for word sentences doesn't mean that like, you know, the it's not the part at all. It means it's it's it's being built off of something fixed. Underneath, right? And that's like the hand built layer that we've talked about, holding up the factual parts that the predictor can't be trusted to invent. So that's the answer to the whole puzzle of LLMs. There's, you know, two ideas that spent a long time apart, 40 plus years, as rival ways to be helping machines understand how the world works. ⁓ there were people who wanted to write how the world worked down by hand. And there are people who wanted to skip the facts and learn from patterns, right? And those two have now combined and are stacked on top of each other inside of. single systems like ChatGPT and Gemini and they cover for each other's weaknesses and give us the answers that we use when we operate and function with LLMs on the daily basis. So I think that's the thing that I would say we should walk away with from today's episode more any of the fun facts or dates or names and the history of how this got here. Is that every time an AI is telling you something, either it's about you or your company or a competitor, anything at all. There are two different machines that are working together. Right? Not one. There's one that talks then there is one that I would say knows. And they are very different in their own unique ways. You know, the talker is very fluent and will confidently give you answers and sometimes make things up. And then there's a knower, which is accurate, but only about the things that people have made itself legible to it. So, you know, once you're watching for both, ⁓ the AI, you know, search stops a mysterious. black box that, you know, either likes you or doesn't like you. And you can actually reason and see how it's come to decisions about how it talks about you. So while today's episode has been mostly about the history behind LLMs and how they were built and how they function, I do want to just wrap up with a little bit of a takeaway, which is an to the question you ⁓ might come in with, which is again, how do you show up ⁓ in systems at all in the first place, right? The the answer is that it was never really just one question. Your company in both sides of the machine and you reach it in very different ways, right? talked about this, but the knower, you can edit, right? Your entity in a place like Wikidata, your facts about your company consistent everywhere that a machine can check, right? That is stuff that we can be in control of and we can fix. But there are a lot of companies that don't bother to do it. So it mostly just sits there blank or it might sit there wrong, right? Then you have the talker, and you can't edit the talker at all, right? It only picks up who you are ⁓ and what you do from how often and how clearly the open web puts your name next to what you do, right? ⁓ One of these you can correct and the other of these you earn. Right? These are two different machines with different jobs. And the goal is from today's episode, you can see exactly which is which. So that's today's episode. brains, now you know what they are, how they function, and they're doing when someone asks about your company on an LLM. If you got something out of this podcast episode, go ahead and hit follow wherever you are listening or watching so that the next one can find you. And as always, the sources for all of the information we talked about in today's episode, the papers and the studies are all going to be available in the show notes if you would like to dig in yourself. That is it for me. Thanks for listening to the Search Signal this week, and we will see you at the next one.