Latest / The Practical AI Digest / AI for Code: How Models Write Software

Transcript

AI for Code: How Models Write Software · The Practical AI Digest

• 0:00
  Welcome back to the Deep Dive. Usually we take a wide angle
• 0:04
  lens to a topic, you know, exploring the ecosystem, the
• 0:06
  market dynamics, or even the philosophical implications.
• 0:10
  But today we're doing something a little different.
• 0:12
  This is our Practical AI Digest edition.
• 0:14
  We are zooming in. Weigh in.
• 0:16
  We're going past the headlines and straight into the
• 0:20
  engineering. Exactly.
• 0:21
  We aren't just going to talk about cogeneration in the
• 0:24
  abstract. We are going to unpack
• 0:27
  specifically how to build and I think maybe more importantly,
• 0:32
  how to evaluate the models that build software for us.
• 0:34
  And we have a perfect subject for that today, the.
• 0:36
  Topic on the table is the engine that really launched this whole
• 0:40
  era of AI assisted programming, Open AI's codecs.
• 0:44
  This is the model that powers GitHub Copilot.
• 0:47
  And you know, while it's really easy to just install the
• 0:49
  extension and let it auto complete your loops, the actual
• 0:52
  mechanism under the hood is, well, it's a fascinating case
• 0:56
  study and machine learning adaptation.
• 0:58
  So what are we basing this on? What's the source measure?
• 1:00
  We're basing this deep dive on the primary research paper from
• 1:03
  the Open AI team. It's called Evaluating Large
• 1:06
  Language Models Trained on Code. We'll also pull in some context
• 1:10
  on the launch from a great piece in Singularity Hub.
• 1:13
  OK, so the mission for us today is to bridge that gap between it
• 1:17
  works like magic and it works like math.
• 1:20
  I like that. We really want you to understand
• 1:23
  how a model translates natural language, you know, plain
• 1:26
  English, into functional Python code.
• 1:29
  And to do that, we have to look at 3 very specific areas.
• 1:33
  Right, first the architecture. Specifically how it even reads
• 1:37
  text. Then the metrics.
• 1:38
  Why do standard language scores totally fail for code?
• 1:41
  And finally, the tuning. How do you actually control the,
• 1:45
  let's call it, creativity of the machine?
• 1:48
  And I think for the developers listening or maybe anyone
• 1:50
  managing a technical team, there's a key concept here we're
• 1:53
  going to hit. Hard Oh yeah, the Passat collie
• 1:55
  main metric it is absolutely crucial.
• 1:58
  It really changes how you should think about the reliability of
• 2:00
  these tools Well, it's not just about getting the right answer,
• 2:03
  it's about the probability of generating the right answer
• 2:07
  within a specific budget of demps.
• 2:10
  It's a whole new way of thinking about correctness.
• 2:13
  Right, let's start at the beginning, then the origin story
• 2:15
  with rewind a bit back to GPT 3. The original text Titan.
• 2:20
  When GPT 3 launched, it was the king of text.
• 2:24
  It could write poetry, it could summarize emails.
• 2:27
  It could generate marketing copy.
• 2:29
  But was it a coder? Not really.
• 2:32
  I mean not in any practical sense.
• 2:33
  It was a generalist. So what was it trained on?
• 2:36
  It was trained on the Common Crawl, which is basically a
• 2:38
  scrape of the entire public Internet.
• 2:40
  And the Internet has code on it. Oh, it has a lot of code.
• 2:43
  If you script the web, you're going to get Stack Overflow.
• 2:45
  You get technical blogs, you get random snippets and forums.
• 2:49
  O GT3 definitely saw code, but it just saw it as another form
• 2:53
  of text. O it didn't understand the like
• 2:56
  the physics of code, the rules. Exactly, that's a perfect way to
• 3:00
  put it. It could mimic the structure O.
• 3:01
  If you asked it for a for loop, it would give you something that
• 3:05
  looked like a for loop. But it wouldn't work.
• 3:06
  Often not. The variables wouldn't quite
• 3:08
  match, or the syntax would be just slightly off.
• 3:11
  It was like a student who memorized the textbook glossary
• 3:15
  but didn't actually understand the grammar or the logic flow.
• 3:18
  It was producing text that looked like code rather than
• 3:21
  functional instructions. Let's just say it was a mimic.
• 3:23
  So the pivot to codex then wasn't just let's make GPT 3
• 3:27
  bigger, it was something more fundamental.
• 3:30
  Right, it was. Let's specialize.
• 3:32
  They took the foundational GPT 3 model and they fine-tuned it.
• 3:37
  They basically force fed it a massive diet of pure code.
• 3:40
  Where'd they get all that code? Specifically, they gathered
• 3:43
  publicly available code from GitHub.
• 3:46
  We are talking about a data set that even after filtering came
• 3:49
  out 259 gigabytes of unique Python files.
• 3:53
  I want to pause on that word filtering because it sounds so
• 3:55
  trivial, but in data science that's everything, isn't it?
• 3:57
  It's the whole game, you can't just feed it every single repo
• 4:01
  on GitHub, that would be a disaster.
• 4:03
  Why? What's wrong with the data on
• 4:04
  GitHub? Well, a lot of it is auto
• 4:06
  generated. Think of those massive compiled
• 4:09
  JavaScript files or configuration dumps.
• 4:13
  It's repetitive and it's very low quality information.
• 4:16
  It's noise. OK, so they stripped that out.
• 4:19
  They stripped that out. They also removed files that had
• 4:22
  super long lines, like over 1000 characters, because that's
• 4:26
  usually just data storage, not actual logic.
• 4:29
  Right, like a big Jason BLOB or something.
• 4:31
  Exactly. And most importantly, they
• 4:33
  filtered for quality to some extent, which we'll get into
• 4:35
  later when we talk about the Codex S version.
• 4:38
  But just the sheer volume here is the first big differentiator
• 4:42
  from GPD 3. But here's where it gets really
• 4:44
  interesting for the engineers listening.
• 4:46
  You can't just feed code into a text model without changing how
• 4:50
  the model reads, can you? No, you can't.
• 4:52
  We have to talk about the tokenizer.
• 4:53
  This is such a crucial piece of engineering that I think most
• 4:56
  people just totally overlook. So what is a tokenizer,
• 4:59
  honestly? Large language models.
• 5:01
  They don't read words like cat or print, they read tokens,
• 5:05
  which are just chunks of characters and the standard text
• 5:09
  tokenizers like the one in Jeeple T3.
• 5:12
  They're optimized for English sentences.
• 5:14
  So they're really good at encoding common words like the
• 5:18
  and, or, you know, suffixes like ING.
• 5:20
  Exactly. They're very efficient at that,
• 5:23
  but code isn't English. The whole distribution of
• 5:26
  characters is completely different.
• 5:28
  And in Python specifically, there's a very unique problem.
• 5:31
  White space. Because in Python, the
• 5:32
  indentation isn't just for looks, it's grammatical.
• 5:36
  It actually defines the scope of the code.
• 5:38
  Right, If you're using Java or C++ L, you have curly braces.
• 5:42
  You can mess up the spacing all you want and the compiler
• 5:44
  doesn't care. The code still runs.
• 5:46
  But in Python, 4 spaces versus 2 spaces completely changes the
• 5:49
  logic of the program. It's a different program now.
• 5:52
  A standard GPT 3 tokenizer is just terrible at this.
• 5:55
  It treats multiple spaces really inefficiently.
• 5:57
  If you have a heavily indented block of code, a standard
• 6:01
  tokenizer might burn through 5 or 6 different tokens just to
• 6:04
  represent the blank blank space. Before you even get to the
• 6:07
  actual code. Before the line even starts.
• 6:09
  And remember, these models have a limited context window, a
• 6:13
  limit on how much text they can remember at once.
• 6:16
  So wasting tokens on blank space is a huge performance hit.
• 6:20
  It's a massive hit. It's pushing the actual logic,
• 6:22
  the important stuff, out of the models memory.
• 6:25
  It's a waste of memory and compute.
• 6:27
  So what did they do for codecs? They modified the tokenizer.
• 6:31
  They added a specific set of new tokens that represent runs of
• 6:34
  white space of different lengths.
• 6:36
  So 2 spaces is its own token, 4 spaces is another token.
• 6:39
  Precisely 8 spaces is a token and so on.
• 6:42
  It's kind of like you're zip compressing the silence.
• 6:44
  That's a great analogy, and the paper notes that this single
• 6:47
  optimization allows the model to represent code using
• 6:50
  approximately 30% fewer. Tokens.
• 6:54
  That is a massive number. That means for the same amount
• 6:57
  of computing power, the model can see 30% more of your code
• 7:01
  base. Correct.
• 7:02
  It can. Look further back in your file
• 7:04
  to understand the context of what you're trying to build.
• 7:06
  It's not just some efficiency hack, it is a capability unlock.
• 7:11
  Because if the model can see the function definition from 50
• 7:15
  lines ago because it saves space on indentation.
• 7:18
  It's going to write better code, it's just that simple.
• 7:20
  OK, so we have a modeled trained on billions of lines of code
• 7:23
  with a tokenizer that's optimized to read it
• 7:25
  efficiently. Now comes the hard part.
• 7:29
  The evaluation problem How? Do you know if it's actually any
• 7:31
  good? This is where the paper really
• 7:34
  shifts the paradigm for the entire industry.
• 7:36
  So in traditional natural language processing, we'd rely
• 7:39
  on metrics like the Bleu score, right?
• 7:42
  Bleu. Yes, and VLEU is essentially a
• 7:46
  text matching score. It's pretty simple if I ask a
• 7:49
  translation model to translate the cat sat on the mat into
• 7:52
  French and I have a reference translation from a human.
• 7:56
  You just see how much they overlap.
• 7:57
  You just check how much the machine's output overlaps with
• 8:00
  the human's output. The more matching sequences of
• 8:02
  words and grams, the higher the score.
• 8:04
  Which makes perfect sense for language.
• 8:06
  The cat sat on the mat and the cat sat on the rug are very
• 8:09
  close in meaning and in structure.
• 8:11
  But code is fragile. It's brittle.
• 8:14
  Let me give you the example from the paper.
• 8:16
  Suppose the correct code to solve a problem is X = y + Z OK.
• 8:22
  Simle addition. Now imagine the model outputs X
• 8:25
  = Z + y. Textually, that's different.
• 8:28
  The Z and the Y have swapped places.
• 8:31
  A text matching metric like Bleu would see that the order is
• 8:34
  wrong and penalize it. Exactly.
• 8:36
  The Bleu score would drop, but functionally.
• 8:39
  It's identical. It's perfect addition is
• 8:41
  commutative. The code works 100%.
• 8:44
  The computer doesn't care about the order.
• 8:46
  OK, so now imagine the reverse. Let's say the model outputs
• 8:49
  XAYZ. Well that looks almost identical
• 8:52
  to the correct answer. The text match is incredibly
• 8:55
  high. It's just one character off.
• 8:57
  The Bleu score would be near perfect, probably a 99% match.
• 9:01
  But the logic is completely wrong.
• 9:02
  It subtracts instead of ads. It's a totally different
• 9:05
  program, or even worse, imagine it just misses a single colon at
• 9:09
  the end of a function definition.
• 9:10
  The program crashes. It won't even compile.
• 9:12
  It won't even run. It's a syntax error and yet to a
• 9:15
  text matching algorithm it looks like a master piece.
• 9:18
  It's a 99.9% match. So the researchers found that
• 9:22
  high Bleu scores just didn't correlate with working code.
• 9:26
  They explicitly found that the correlation was weak.
• 9:28
  You could optimize a model to get a really high Bleu score and
• 9:32
  it would produce code that looked like code.
• 9:35
  It had the right variable names and the right structure, but it
• 9:38
  just didn't function. It was what they called cargo
• 9:40
  cult programming. Exactly.
• 9:42
  It's going through the motions without understanding the
• 9:44
  substance. So Open AI had to just abandoned
• 9:47
  the standard NLP metrics. They needed a metric that
• 9:49
  actually measured software quality.
• 9:51
  And they moved to functional correctness.
• 9:54
  They effectively applied the principles of Test Driven
• 9:56
  Development to AI evaluation. And to do that, they built a new
• 10:00
  benchmark. Yes, a benchmark called Human
• 10:03
  Evil. Tell us about human evil.
• 10:05
  Why did they have to build a new one?
• 10:06
  I mean why not just use problems from say leet code or hacker
• 10:10
  rank? Data contamination.
• 10:12
  It's a huge, huge issue in machine learning.
• 10:15
  Remember, the model was strained on all of GitHub.
• 10:18
  It is almost certainly seen every solution to every popular
• 10:21
  leak code problem ever posted. So if you tested on those
• 10:24
  problems. You aren't testing its ability
• 10:26
  to code, you're just testing its ability to memorize.
• 10:30
  It's like giving a student the exam questions a month
• 10:32
  beforehand. Exactly.
• 10:34
  So what the Open AI researchers did was they handwrote 164 brand
• 10:38
  new programming problems. These are problems the model has
• 10:41
  never ever seen before. And what's in each problem?
• 10:45
  Each one has a function signature, a dock string that
• 10:47
  explains what needs to happen, and this is the absolute key, a
• 10:51
  suite of unit tests. And the rule is simple, it
• 10:54
  either passes all the tests or it fails.
• 10:56
  Zero or one, black and white. There's no partial credit for
• 11:00
  looking nice or being close. And this binary success fail
• 11:03
  criteria leads us to the most important metric in the entire
• 11:07
  paper, the metric that really defines the current state of AI
• 11:10
  coding. We are talking about pass at K.
• 11:13
  Pass at Ki think for a lot of people just seeing K in a metric
• 11:17
  name brings back some bad memories of Alger class.
• 11:20
  Yeah, probably. So let's break this down
• 11:22
  conceptually. Why do we even need a variable
• 11:24
  there? Why not just pass or fail?
• 11:26
  Because these models are probabilistic, they are non
• 11:29
  deterministic. If you ask codecs to write a
• 11:32
  function to sort a list of names 5 different times, you might get
• 11:37
  5 slightly different pieces of code.
• 11:39
  So we can't just ask does it work?
• 11:41
  No, we have to ask how often does it work?
• 11:43
  That's the right question, so pass at Tay answers this.
• 11:46
  If I let the model generate a color number of solutions, let's
• 11:50
  say 100 different solutions, what is the probability that at
• 11:54
  least one of them is correct? At least one.
• 11:57
  OK, that sounds like a very generous metric, doesn't it?
• 12:00
  If I'm a developer, I don't want to have to dig through 99 broken
• 12:04
  snippets just to find the one that works.
• 12:06
  You would think so, but let's think about how these tools are
• 12:08
  actually deployed in the real world.
• 12:10
  When you use an auto complete in your IDE it often gives you a
• 12:13
  drop down list of suggestions, right?
• 12:15
  You get a few options. Or if you're using a chat
• 12:17
  interface, you can just hit the regenerate button.
• 12:19
  True, if the first one looks wrong, I just hit the button
• 12:22
  again. Right, so from a user experience
• 12:25
  perspective, if the model generates the correct code
• 12:28
  within the top few tries, the user is happy, they can
• 12:31
  recognize the right solution, they accept it, and they move
• 12:33
  on. So pass it K is really measuring
• 12:36
  the hit rate of the models brainstorming process.
• 12:39
  That's a great way to put it. It treats the AI as a generator
• 12:43
  of candidates, not as a sole source of truth.
• 12:46
  Now, there's a specific engineer's note in the paper
• 12:49
  about how they calculate this. It's not as simple as just
• 12:52
  running it 100 times and dividing dividing by 100.
• 12:54
  No, that would be incredibly computationally expensive and
• 12:57
  also statistically noisy. They use what's called an
• 13:00
  unbiased estimator. It's a bit of math gymnastics,
• 13:03
  but it's really important for getting an accurate number.
• 13:05
  Can you give us the simplified version?
• 13:07
  Sure. Essentially what they do is they
• 13:09
  generate a large pool of samples upfront.
• 13:12
  Let's say they generate 200 samples, so an equals $200.
• 13:16
  Then they run the tests and check how many of those are
• 13:17
  correct. Let's call that number 2.
• 13:19
  OK, so you have your big pool of answers.
• 13:22
  Then instead of running a whole new experiment for every single
• 13:25
  value of Calor, they just use a combinant torques formula to
• 13:28
  calculate the probability. Right, the one in the paper is
• 13:31
  $1.00 Frack Banal Deitman. Exactly, and without getting out
• 13:34
  the chalkboard, the intuition there is that it's calculating
• 13:37
  the probability of picking purely wrong answers.
• 13:41
  So it's asking, if I reached into this bag of 200 samples and
• 13:44
  pulled out, say, 10 samples, what are the odds that every
• 13:48
  single one of them is a failure? Precisely.
• 13:50
  And then you just subtract that number from 1:00 to get the
• 13:53
  probability of success. And if the probability of
• 13:56
  getting all failures is low, then the probability of finding
• 14:00
  at least one success must be high.
• 14:02
  Correct. It lets them report very
• 14:04
  accurate statistics for any value of tellers without having
• 14:07
  to run an infinite number of tests.
• 14:09
  It's a standard statistical method for sampling without
• 14:13
  replacement, just applied here to software generation.
• 14:15
  So let's get to the scoreboard using this very rigorous pass at
• 14:19
  Co metric. How did codecs actually perform
• 14:22
  on the Human Evil benchmark? OK, first the baseline.
• 14:25
  Let's talk about the original GPT 3 model.
• 14:28
  How did it do? It solved 0% of the problems.
• 14:31
  Zero as in none. Effectively 0.
• 14:34
  It just could not handle the functional logic constraints.
• 14:38
  Like we said, it might write a nice poem about the code, but it
• 14:40
  couldn't actually write the code.
• 14:41
  OK, So what about codecs? Now looking at codecs,
• 14:44
  specifically the 12 billion parameter version, on the very
• 14:48
  first try which is pass at one, it solved 28.8% of the problems.
• 14:53
  28.8% that is. Well, it's not taking anyone's
• 14:57
  job just yet. That's a failure rate of over
• 14:59
  70% on the first guess. It's AC minus student at best.
• 15:03
  But, and this is a huge but, this is where that pass at K
• 15:06
  logic kicks in. What happens if we give it a
• 15:08
  hundred tries? What's the number for tasks at
• 15:10
  100? If you let it generate 100
• 15:12
  different samples, and specifically if you use the more
• 15:14
  fine-tuned Codex S model which we'll talk about, the success
• 15:17
  rate shoots up to 77.5%. Wow.
• 15:21
  OK, that is a staggering difference.
• 15:23
  We go from barely functional to solving more than three out of
• 15:26
  every 4 problems. It tells us something really
• 15:28
  profound about the nature of these models.
• 15:30
  They aren't oracles of truth, they're idea generators.
• 15:33
  They contain the knowledge to solve the problem.
• 15:35
  Yes, but they have trouble focusing on the exact right
• 15:38
  syntax in a single shot. But if you let them just shotgun
• 15:42
  the problem with a bunch of attempts, they almost certainly
• 15:44
  find the answer somewhere in that spray.
• 15:47
  Which brings us to a really practical dilemma for the user,
• 15:51
  for the developer. If I need to generate 100
• 15:53
  samples to find the right one, how do I configure the model?
• 15:57
  I mean, if I just ask the same question 100 times, won't it
• 16:00
  just give me the same wrong answer 100?
• 16:02
  Times it would, yes, unless you tune the temperature.
• 16:05
  This is section four of our digest the Concept of
• 16:08
  Temperature in AI. Temperature is a hyper parameter
• 16:11
  that controls the randomness of the model's next token
• 16:13
  prediction. It's basically a knob that you
• 16:16
  can turn to adjust how adventurous or creative the
• 16:19
  model is allowed to be. Let's define the extremes.
• 16:22
  What does a low temperature look like?
• 16:23
  What does that do? A low temperature, let's say .0
• 16:27
  or 0.2, makes the model very conservative.
• 16:31
  When it predicts the next word or the next token, it looks at
• 16:34
  the probability distribution and it almost always just picks the
• 16:36
  top winner. It's greedy.
• 16:38
  It plays it safe, it doesn't take any risks.
• 16:40
  Very safe. It's almost deterministic.
• 16:42
  If you run the same prompt 10 times that temperature 0, you're
• 16:46
  going to get the same exact code 10 times.
• 16:48
  It just locks in on what it thinks is the most likely path
• 16:51
  and sticks to it. OK, So what about high
• 16:53
  temperature? A high temperature like .8 or
• 16:56
  even 1.0 flattens that probability curve.
• 16:59
  It makes the model much more willing to take risks.
• 17:02
  It might pick the second most likely token, or the third, or
• 17:05
  even the 10th. This introduces variation.
• 17:09
  It gets creative. Creative or sometimes it
• 17:11
  hallucinates depending on how you look at it.
• 17:14
  Now the paper presents a crucial strategy guide for this.
• 17:17
  The optimal temperature to use depends entirely on your taller
• 17:20
  strategy. So if I only have one shot if my
• 17:22
  Telugu was one, what should I do?
• 17:24
  You use low temperature? Absolutely.
• 17:25
  If you only get one guess, you want the models absolute best,
• 17:28
  most confident guess. You don't want it experimenting
• 17:31
  on your time, you want the highest probability.
• 17:33
  Path. But if I have 100 shots if Kalu
• 17:36
  equals 100. Then you absolutely must use
• 17:39
  high temperature. Think about it, if you use a low
• 17:42
  temperature for 100 samples, you just get 100 copies of the same
• 17:46
  potentially wrong piece of code that's totally useless.
• 17:49
  You need diversity. You need diversity.
• 17:51
  By cranking the temperature up to .8, you force the model to
• 17:55
  try different approaches. Sure, maybe fifty of those
• 17:58
  samples will be complete gibberish or syntax errors
• 18:01
  because it took a weird risk, but five of them might be
• 18:05
  brilliant, novel solutions that solve the problem in a way you
• 18:08
  didn't expect. It's the difference between
• 18:10
  asking one expert for their single best piece of advice
• 18:14
  versus asking a whole room full of people to just brainstorm
• 18:16
  wild ideas. Precisely.
• 18:18
  And for code generation, the paper proves that the
• 18:21
  brainstorming approach yields significantly better results.
• 18:24
  That's how you get to that 77.5% success rate we mentioned.
• 18:27
  This creates a new problem, a very practical 1.
• 18:29
  The selection problem. Right, in the research lab they
• 18:32
  had the unit tests. So they can generate 100 crazy
• 18:36
  ideas, run them all against the tests and just see which ones
• 18:38
  pass. But if I'm a developer sitting
• 18:40
  in VS Code, I don't have a unit test for the function I haven't
• 18:43
  even written yet. Exactly.
• 18:45
  This is what's called the Oracle problem.
• 18:47
  The model generates 100 options in the background.
• 18:50
  It knows one of them is probably right, but how does it know
• 18:54
  which one to actually show you? It can't run the code itself.
• 18:57
  So they need a heuristic, a way to guess which code is the
• 19:01
  winner without actually running it.
• 19:03
  And the paper explores a few of these.
• 19:05
  The most robust one they found is mean log probability.
• 19:10
  Can you put that in English for? It its confidence, the model
• 19:15
  assigns A probability score to every single token it generates.
• 19:18
  If you average all those scores across the whole snippet of
• 19:21
  code, you get a kind of confidence score.
• 19:23
  So it's basically the model saying.
• 19:25
  I felt really sure about this code when I was writing it,
• 19:28
  yeah. That's it.
• 19:29
  So the heuristic is you show the user the one the AI felt best
• 19:33
  about, and it's a pretty good proxy for correctness.
• 19:36
  It helps filter out that high temperature nonsense where the
• 19:39
  model took a wild risk and just got confused.
• 19:41
  If the model is kind of mumbling its way through the code with
• 19:44
  low confidence on every token, you probably shouldn't show that
• 19:47
  one to the user. Probably not.
• 19:49
  They also mentioned a really clever technique that involves
• 19:52
  dock strings. This is what they called Codex
• 19:54
  D, right? The D is for dock string.
• 19:57
  Yes, this part is fascinating. So far we've only talked about
• 20:00
  text to code. Codex D is a model that's
• 20:03
  fine-tuned for the reverse code to text.
• 20:05
  It reads a piece of code and then explains in plain English
• 20:09
  what it does. Right, it generates the doc
• 20:11
  string. So here's a way you could verify
• 20:14
  your code without needing any unit tests back translation.
• 20:17
  OK, how does that work? It's a round trip verification.
• 20:19
  So step one, take your prompt, write a Python function to
• 20:23
  calculate the area of a circle. Step 2, the model generates a
• 20:27
  code snippet. Step three, you take that code
• 20:30
  snippet that it just generated and you feed it back into Codex
• 20:34
  D and ask, hey, what does this code do?
• 20:36
  And Step 4, Codex D writes a new doc string a new explanation.
• 20:41
  And then you compare you compare the new explanation to your
• 20:44
  original request. If the model looks at its own
• 20:47
  code and says this function calculates the perimeter of a
• 20:50
  square, then you know the code is wrong.
• 20:53
  It drifted from the original intent.
• 20:56
  But if it says this function takes a radius and calculates
• 20:59
  the area of a circle, then the intent was preserved.
• 21:02
  Exactly, It's a semantic check. The paper does note that this
• 21:05
  successfully filters out a lot of bad code, though.
• 21:08
  Surprisingly, it wasn't quite as effective as the simple mean log
• 21:11
  probability heuristic. But it shows the direction
• 21:13
  researchers are going. Totally.
• 21:15
  It's using AI to grade AI. It's mimicking how a human might
• 21:19
  double check their own work. Wait, does this code actually do
• 21:22
  what I wanted it to do? I want to circle back to
• 21:24
  something you mentioned earlier. You said codecs.
• 21:26
  You mentioned filtering the training data, right?
• 21:28
  This is the supervised fine tuning version.
• 21:31
  And we know GitHub is a a messy place.
• 21:34
  It is. It's full of half finished
• 21:36
  projects, homework assignments that probably got AC minus and
• 21:40
  just, you know, bad coding habits that have been copied and
• 21:43
  pasted for a decade. Lot of spaghetti code.
• 21:45
  A ton. So if you train the model on
• 21:47
  everything, it learns the bad habits too.
• 21:49
  It learns to write inefficient loops, or to use terrible
• 21:53
  variable names like temp one and temp two.
• 21:56
  It learns to write code that doesn't even compile.
• 21:58
  So for cut excess, they changed the diet, they got pickier.
• 22:01
  Much pickier. They identified high quality
• 22:04
  code. They didn't just grab random
• 22:06
  files from GitHub. They specifically looked for
• 22:08
  standalone functions that could be compiled and run.
• 22:11
  Where do they find this good code?
• 22:13
  They prioritize code from 2 main sources, competitive programming
• 22:17
  websites where solutions are almost always correct and highly
• 22:19
  optimized, and from repositories that had heavy continuous
• 22:22
  integration or CI testing. So code that is proven to work.
• 22:26
  Proven and clean and the impact was significant.
• 22:29
  Standard codec solved 28.8% of problems on the first try.
• 22:32
  Pass at 1:00. Kodak staff solved 37.7%.
• 22:36
  Well, that's nearly A10 percentage point jump just by
• 22:38
  cleaning up the data. It really reinforces the golden
• 22:41
  rule of AI. Data quality almost always beats
• 22:45
  sheer data quantity. You want the model to learn from
• 22:48
  the senior engineers, not from the interns.
• 22:51
  OK, we've covered the tech. The model is powerful, it's
• 22:54
  efficient. But is it safe?
• 22:58
  Section 6. The paper covers limitations and
• 23:00
  risks, and frankly, there are some serious engineering red
• 23:04
  flags here. There really are, and for anyone
• 23:07
  thinking about deploying this in an enterprise environment, you
• 23:09
  absolutely need to listen to this part.
• 23:11
  Let's start with what they call alignment failure, or as I was
• 23:15
  thinking of it, the bad influence problem.
• 23:17
  This is from the Settle bug experiment.
• 23:19
  The researchers wanted to know, does the model always try to
• 23:22
  write the best possible code? You would assume it would You
• 23:25
  assume it's trying to be helpful and correct.
• 23:27
  But it's not. It's not trying to be helpful,
• 23:29
  it's trying to predict the next token based on the context it's
• 23:32
  given. So they ran this experiment
• 23:34
  where they gave the model a prompt that already included a
• 23:36
  bug. They wrote some sloppy,
• 23:38
  incorrect code in the setup part of the prompt.
• 23:40
  They set a bad example for it. And codecs followed it.
• 23:44
  Instead of fixing the bug or writing clean code, it produced
• 23:47
  more buggy code. It aligned with the sloppy style
• 23:51
  of the user. That is wild.
• 23:52
  It's like a chameleon. If you act incompetent in the
• 23:55
  prompt, it mimics incompetence back at you.
• 23:58
  The paper calls this misalignment.
• 24:00
  The user's actual content is help me write correct code, but
• 24:05
  the models objective is just match the distribution of the
• 24:08
  text provided. If that text looks like garbage,
• 24:11
  it generates more garbage to match.
• 24:12
  That feels like a massive trap, especially for junior
• 24:15
  developers. If you don't already know what
• 24:17
  you're doing, the AI will just validate your mistakes.
• 24:19
  It will amplify them. It gives you this false sense of
• 24:22
  confidence because the code it generates looks just like your
• 24:25
  code. OK, what about complexity?
• 24:26
  Can I ask codecs to, you know, build me a clone of Twitter?
• 24:29
  Not a chance. The performance drops off a
• 24:32
  Cliff as complexity increases. The paper shows this really
• 24:35
  clear graph of success rates for what they call chained
• 24:38
  components. Meaning what like do task A,
• 24:40
  then do task B, then do task C? Exactly.
• 24:44
  A prompt like read the data from this file, parse it as a CSV,
• 24:48
  sum the values in the third column and then print the
• 24:51
  average. A multi step process.
• 24:53
  As soon as you chain more than just a few operations like that,
• 24:56
  the model gets confused. It struggles with binding, with
• 25:00
  remembering which variable holds the data from step one while
• 25:03
  it's working on Step 3. So it has a short attention
• 25:05
  span. It's a sprinter, not a marathon
• 25:08
  runner. It is excellent at writing a
• 25:10
  single complex function. It is terrible at designing a
• 25:14
  multi part system architecture. Security is another big one.
• 25:18
  We hear about supply chain attacks constantly.
• 25:20
  Is Codex writing secure code? The findings in the paper are
• 25:25
  pretty concerning on this front. The researchers asked codecs to
• 25:28
  generate cryptographic configurations.
• 25:30
  You mean like encryption keys and settings?
• 25:32
  And it frequently generated insecure configurations.
• 25:34
  For example, it would often use RSA encryption keys with fewer
• 25:37
  than 2048 bits. Which is considered cracked or
• 25:41
  at least very weak by modern standards.
• 25:43
  Why on earth would it do that? Because it's trained on the
• 25:45
  Internet, and the Internet is absolutely full of tutorials
• 25:48
  from 2012 that use small keys. Because they are faster for
• 25:51
  demos and examples. So codecs doesn't know what year
• 25:54
  it is. It has no idea that NIST
• 25:56
  standards have changed. It just knows that
• 25:58
  statistically, after these words I usually see this number.
• 26:01
  It propagates obsolete security practices.
• 26:04
  So if you just copy paste security related code from an AI
• 26:08
  without having an expert audit it, you are very likely
• 26:11
  introducing vulnerabilities. Yes, you're just repeating the
• 26:14
  statistical average of the Internet, and the average code
• 26:16
  on the Internet is definitely not secure code.
• 26:19
  And finally, let's talk about bias.
• 26:21
  We often talk about this in terms of text and images, but
• 26:24
  does it manifest in code? It does, and in a really
• 26:27
  interesting way. Specifically, it manifests in
• 26:30
  the data structures the model creates.
• 26:33
  If you ask codecs to create a class for a person or a user,
• 26:36
  what happened? It often defaults to things like
• 26:38
  binary gender fields or very limited outdated racial
• 26:41
  categories. Because that's how databases
• 26:44
  were designed in the 90s and 2000s.
• 26:46
  Exactly. It's mirroring the structural
• 26:49
  biases of the legacy code it was trained on.
• 26:51
  It preserves the rigid categorization of the past.
• 26:54
  If your modern application needs more or inclusive data
• 26:58
  structures, codecs might actually fight you on it because
• 27:01
  it just hasn't seen enough examples of that new way of
• 27:03
  thinking yet. It's an artifact of the training
• 27:06
  datas age and distribution. A direct reflection 1.
• 27:09
  Last thing that people always ask about this stuff copyright.
• 27:12
  Is this thing just a big plagiarism machine?
• 27:15
  Is it just stealing code? The paper addresses this
• 27:18
  directly. They did a specific check for
• 27:20
  what they called literal copying.
• 27:22
  Verbatim copy paste from the training data.
• 27:25
  And they found that literal copying occurred in less than
• 27:28
  .1% of cases. That's.
• 27:30
  Extremely low. It's almost nothing, and when it
• 27:32
  did happen, it was almost always what you'd call boilerplate
• 27:36
  standard license headers or very common simple algorithms like
• 27:39
  quicksort that pretty much everyone writes the exact same
• 27:42
  way. So it acts.
• 27:43
  It's more like a synthesizer than a photocopier.
• 27:46
  That's a great way to describe it.
• 27:47
  It learns the patterns, but it rarely regurgitates the exact
• 27:50
  source file. So pulling all this together,
• 27:52
  we've looked at the specialized tokenizer, the big shift from
• 27:55
  Bleu to pass AK, the importance of temperature tuning, and all
• 28:00
  these risks of alignment and security.
• 28:02
  What is the final verdict here? The verdict is that codecs and
• 28:07
  tools like it are incredible productivity multipliers, but
• 28:11
  they fundamentally change the job description of a programmer.
• 28:14
  How so? The job is moving away from
• 28:17
  writing syntax and towards verifying intent.
• 28:20
  You're not the writer anymore, you're the editor.
• 28:22
  You're the architect and you are the quality assurance department
• 28:25
  all rolled into one. The pass at Hick metric proves
• 28:29
  that the AI is a candidate generator.
• 28:31
  It just gives you options. You, the human, must have the
• 28:35
  skill to look at those options and distinguish the diamond from
• 28:37
  the glass. That's a really powerful
• 28:39
  thought. It doesn't lower the bar for
• 28:40
  knowledge. In some ways, it actually raises
• 28:43
  the bar for critical thinking. It does.
• 28:45
  If you can't read code well, you can't safely use the tool that
• 28:48
  writes it for you. Exactly.
• 28:49
  You can't delegate the responsibility of correctness to
• 28:52
  the machine. Not yet.
• 28:54
  Before we wrap up, I want to leave our listeners with a
• 28:57
  thought that really struck me while I was reading the paper.
• 29:00
  They mentioned that codec struggles with dock strings
• 29:03
  describing long chains of operations.
• 29:06
  It can't handle infinite complexity yet.
• 29:08
  Right, the sprinter, not the marathon runner.
• 29:11
  But let's just assume Moore's Law holds.
• 29:13
  Assume the models get better and better.
• 29:15
  Assume they can eventually handle massive chained
• 29:18
  complexity perfectly. OK, where are you going with
• 29:21
  this? Well, today the doc string, the
• 29:23
  English explanation is just a comment, it's ignored by the
• 29:27
  machine. The code itself is the source of
• 29:29
  truth. But if the model gets perfect at
• 29:32
  translating English to code, does the code itself become
• 29:36
  invisible? That is the ultimate abstraction
• 29:38
  layer. If I can just write the dock
• 29:40
  string and the machine perfectly and reliably executes the logic,
• 29:43
  will we eventually lose the ability to even read the code
• 29:45
  entirely? Will Python just become the new
• 29:48
  assembly language, something only the machines look at?
• 29:50
  Well, we humans just write prompts.
• 29:52
  We might. The dock string would become the
• 29:54
  source code. The English language becomes the
• 29:56
  programming language. A world where we program in
• 29:59
  plain English. It's democratizing, it's
• 30:01
  exciting, and for those of us who kind of love the syntax,
• 30:06
  it's a little terrifying it. Makes verifying that intent the
• 30:10
  most valuable skill on the planet.
• 30:12
  Absolutely. Well, that is all for our
• 30:14
  Practical AI Digest on Open AI's codecs.
• 30:18
  We hope you have a much better understanding of the engine
• 30:20
  under the hood of your autocomplete now.
• 30:21
  And if you are a developer, go into your settings, try tuning
• 30:25
  that temperature parameter. See what happens when you crank
• 30:27
  it up to .8 on a really hard problem and.
• 30:30
  Let us know if you get any brilliant or just hilarious
• 30:32
  results. Thanks for listening to the deep
• 30:34
  dive. We'll see you next time.