Latest / DevQuestions with Tim Corey / 263. GitHub, Copilot, and More With Martin Woodward of GitHub

Transcript

263. GitHub, Copilot, and More With Martin Woodward of GitHub · DevQuestions with Tim Corey

• 0:00
  SPEAKER_01: If you go to a chat window and ask it a question, if you get an answer back and it's trash, you're never gonna ask a question again.
• 0:06
  SPEAKER_01: You need devs more than you ever have need devs.
• 0:17
  SPEAKER_03: Welcome to a special episode of Dev Questions.
• 0:20
  SPEAKER_03: I'm here with Martin Woodward.
• 0:22
  SPEAKER_03: Martin, welcome.
• 0:22
  SPEAKER_03: Thank you for having me.
• 0:24
  SPEAKER_03: Yeah.
• 0:24
  SPEAKER_03: So you're the vice president of developer relations at GitHub, yeah, that's right.
• 0:27
  SPEAKER_01: Uh-huh.
• 0:28
  SPEAKER_01: Excellent.
• 0:28
  SPEAKER_03: Yeah.
• 0:29
  SPEAKER_03: So tell me a bit about what that role entails.
• 0:31
  SPEAKER_01: Yeah, mostly deleting a lot of email, I think, when she gets a VP.
• 0:35
  SPEAKER_01: But um, so I um help developers try and build delightful things.
• 0:39
  SPEAKER_01: So try and help people um understand what GitHub can do for them and make sure that they, you know, understand the latest and greatest.
• 0:47
  SPEAKER_01: That's part of the job.
• 0:48
  SPEAKER_01: That's kind of the advocacy side.
• 0:50
  SPEAKER_01: Um I also look after it sounds silly for GitHub, but I look after the open source community.
• 0:55
  SPEAKER_01: So making sure, like some of my metrics around open source community health and things like this, so making sure the open source community are happy on GitHub and they're able to be successful, um, and then advocate for their needs across the product and engineering teams in GitHub, you know, if there's particular areas of interest, if it's hot topics, things like that that we need to dig into.
• 1:14
  SPEAKER_01: Um, I also look after um education, and I know this is a topic you want to touch on uh uh today, but um we have um you know offerings that we give to kind of students and teachers and you know the whole education community, like the future of developers.
• 1:29
  SPEAKER_01: Um so I look after that space, and then things like GitHub sponsors and open source funding, making sure again it's related to the health of the open source community, making sure that there's um a sustainable way to build open source and try and um increase the different ways that people can have uh pay, you know, um feed their families and do open source and help make sure there's a diverse array of options there.
• 1:53
  SPEAKER_01: So yeah, there's one of me.
• 1:59
  SPEAKER_01: Yeah, yeah.
• 2:00
  SPEAKER_01: So it's um well we need people need different funding models for being able to build open source.
• 2:06
  SPEAKER_01: Um, you know, some people I got my job um from doing open source, like that's how I got into this space um uh directly, but I built a company around the open source thing I was providing.
• 2:21
  SPEAKER_01: Um and then that company got acquired, and then you know, I worked at Microsoft and then came over to GitHub.
• 2:26
  SPEAKER_01: But the um so that's one option, you know, and then another option is um uh you know different funding models, different ways of doing like support and services kind of thing.
• 2:37
  SPEAKER_01: And then we also make available a thing called GitHub Sponsors, which allows people to kind of um do different models?
• 2:44
  SPEAKER_01: Like is it it are you gonna be almost like a sort of you know um funding voluntary where you kind of just give money to somebody because you want to support them, or can we also give those sponsor people the way of being able to tier some benefits for different levels of sponsorship and things like that?
• 3:01
  SPEAKER_01: And then the last area that we've recently done is around um you know we're trying to sort of it sound we're making it so that companies can invest in a fund which then helps secu helps give maintainers the time to help secure their projects and come with education and training as well as the funding to be able to take time to do the education and training.
• 3:23
  SPEAKER_03: Okay.
• 3:24
  SPEAKER_01: Um and that's the secure open source fund, and that was again a new thing we did because we um because A, it's very important, but also um and lots of open source maintainers, including some of the biggest projects, like really wanted that training and education um to understand what they can do.
• 3:42
  SPEAKER_01: But then we also wanted to make sure that um there's different pools of money around in companies, and uh like if you're going to the board and say, I need to secure my dependencies, that's a different conversation to I want to make sure I have good PR, you know?
• 3:59
  SPEAKER_01: And the secure your open source dependencies is much easier to kind of justify that level of investment that you're making in your dependencies, some of which happen to be open source.
• 4:08
  SPEAKER_01: Sure.
• 4:08
  SPEAKER_01: So yeah, we're just trying to tackle different ways again, help provide different funding options and just try and you know make sure there's options for people.
• 4:16
  SPEAKER_01: But people have to find their own way to to do things, I think, as well.
• 4:19
  SPEAKER_01: Excellent.
• 4:20
  SPEAKER_03: Um, you touched on it briefly, but can you kind of give us a background of like how'd you get here?
• 4:24
  SPEAKER_01: Oh yeah.
• 4:26
  SPEAKER_01: So I um I had like a real job for a long time.
• 4:30
  SPEAKER_01: So I was, you know, um I didn't do Compsci.
• 4:33
  SPEAKER_01: Um I did a degree in physics, I'm a a failed physics teacher.
• 4:36
  SPEAKER_01: Okay.
• 4:36
  SPEAKER_01: And then um, but I was doing lots of programming.
• 4:39
  SPEAKER_01: I've been programming since I was six, sort of thing, and then I decided, you know, do I want to go be a physics teacher or do I sell out to the man and go earn um you know, go into this IT thing and as a graduate programmer earn more than probably I'll ever earn as a teacher kind of thing, and opt opted for that option because I enjoyed programming.
• 4:59
  SPEAKER_01: I enjoy I still enjoy teaching people, I enjoy that side, but I enjoyed the coding, you know.
• 5:04
  SPEAKER_01: Um so went and did that purely because it was the more lucrative route, if I'm honest.
• 5:08
  SPEAKER_01: And because I love programming, but yeah.
• 5:10
  SPEAKER_01: And then um did that, had a real job, you know, insurance companies, banks, airlines, that sort of thing, and then um did a startup with a few friends around um some cross-platform tooling based around Eclipse and Java.
• 5:24
  SPEAKER_01: Okay.
• 5:24
  SPEAKER_01: Um so that was I'd already been involved in open source at that point, but that was kind of where I was really contributing to a major open source.
• 5:31
  SPEAKER_01: Uh well, I've done some Apache work, but that's where I contributed to a major open source project.
• 5:35
  SPEAKER_01: Um did that, and then uh that got acquired by Microsoft in 2009, that company, um, and then was involved in kind of um building uh like source control.
• 5:45
  SPEAKER_01: I did a thing called Co um when we're blanking on the name.
• 5:49
  SPEAKER_01: The uh the Microsoft's old open source hosting area before GitHub came along.
• 5:54
  SPEAKER_01: Um co I'm blinking on the name.
• 5:58
  SPEAKER_01: Anyway, I did that because it was like it was hugely impactful.
• 6:03
  SPEAKER_01: Um I shut that down as well.
• 6:05
  SPEAKER_01: Um so that was um when GitHub came along.
• 6:08
  SPEAKER_01: I did the original like partnership with GitHub and kind of was like, okay, this is where the community's coming, rather than us being in our like little the Microsoft kind of um ghetto of uh of open source, let's make sure we're part of the major community.
• 6:22
  SPEAKER_01: So then was part of the sort of created the Microsoft GitHub account, um helped like us to change our open source standards and things like this, and then yeah.
• 6:29
  SPEAKER_01: And then when we acquired uh when we did GitHub, I was part of the acquisition team and then uh came over after that.
• 6:35
  SPEAKER_03: Well, I really appreciated the the way you acquired GitHub.
• 6:39
  SPEAKER_03: Okay.
• 6:40
  SPEAKER_03: Um because the fact that you know there's a lot of panic, maybe is the right word, in the community about you know Microsoft and rolling steam rolling over the open source community.
• 6:49
  SPEAKER_03: Yeah, yeah, yeah.
• 6:49
  SPEAKER_03: Um and instead the you know, the first thing you did was some really big wins, you know, like the the free private repos and let's take care of the community and let's move it forward, let's move some people from Azure DevOps over into you know do GitHub actions and really sort of elevate what GitHub can do rather than what feels like has been the past of Microsoft, which is the let's let's make it our only for us.
• 7:12
  SPEAKER_03: Yeah, you know, and that that's been really great to see that that GitHub has kind of continued expanding and growing in the community, not just from Microsoft.
• 7:20
  SPEAKER_01: And much more so.
• 7:21
  SPEAKER_01: Um so I originally did the I did the original paper on should we buy GitHub or not.
• 7:26
  SPEAKER_01: Um and that was uh you know when we looked at it the first time, it was no, we would we would break it if we bought it.
• 7:34
  SPEAKER_01: The good things about GitHub, like the community and all of that, um we would risk breaking it by bringing it in.
• 7:42
  SPEAKER_01: And so how do we then change the business to be a better home for an asset like GitHub for you know for a community like GitHub?
• 7:50
  SPEAKER_01: So we actually in 2014 started changing how we worked with open source as a company and rolled out a bunch of open source training, changed how we approached open source, contributing significantly more and becoming one of the the major contributors to open source in the business.
• 8:04
  SPEAKER_01: And then by 2018, when Nat Friedman came along, um, we were able to take a look again and go, yeah, now's the right time, we can do this, but we did it very much keeping GitHub GitHub and deliberately keeping GitHub separate from the rest of Microsoft and kind of try and use that to infect Microsoft with GitHub rather than you know what I mean to try and do it that way and to to protect like there's no point spending eight billion dollars on something if you're gonna then ruin it, you know.
• 8:34
  SPEAKER_01: Yeah, so it was how do we protect the community that's behind it?
• 8:36
  SPEAKER_01: It's a huge like asset to the world.
• 8:40
  SPEAKER_01: Like we cannot um if we were messed that up, we'd have lost the trust of developers forever.
• 8:46
  SPEAKER_01: Like trust is built on your actions and proof of actions, and we needed to make sure that was a success, because otherwise we wouldn't be here at build, like developers would not be interested in GitHub.
• 8:57
  SPEAKER_01: And by doing things like um investing in GitHub, making sure that it you know actions came along, um, making sure we built a bunch of new features and functionality, investing in VS Code and building up that side as well, um, then yeah, we wouldn't be where we are today.
• 9:11
  SPEAKER_01: And then on the GitHub side, like community growth-wise, I love looking at the curve of like where the community was at 2018 when we did the acquisition.
• 9:18
  SPEAKER_01: Um and then all of a sudden it like got up to 65 million and like 100 million, and now we're at like a 150 million people using Git, you know, and that's oh okay.
• 9:28
  SPEAKER_01: So it's it's not it's not it's still growing exponentially, it's not that's not stopped.
• 9:33
  SPEAKER_01: Um and open source is clearly one as a way to build software.
• 9:38
  SPEAKER_01: Um, and these are just yeah, yeah, hugely uh impactful things that you kind of sit back and step back and go, wow, I actually got to witness that.
• 9:46
  SPEAKER_01: You know, I was barely like I was a cog in a wheel, you know, as part of the process, but um got to witness at that point in history and what a what a privilege that is, you know, to be there.
• 9:56
  SPEAKER_01: It was great.
• 9:56
  SPEAKER_01: I'm sure.
• 9:57
  SPEAKER_01: Yeah.
• 9:57
  SPEAKER_01: So let's talk about GitHub Copilot.
• 9:59
  SPEAKER_03: Yeah.
• 10:00
  SPEAKER_03: I'm assuming you're a fan.
• 10:01
  SPEAKER_03: Yeah.
• 10:03
  SPEAKER_03: So tell him a bit more about Copilot, how it came about, yeah, um, and where it is today.
• 10:08
  SPEAKER_01: Well you say I'm a fan originally, so I was like, it was it was sort of internal research project originally.
• 10:14
  SPEAKER_01: Um and we were digging into like um with you know chatting with this company called OpenAI and we're seeing a bunch of like stuff that they were doing, and then we were looking at a model, and then we um did like a fine-tuned model.
• 10:27
  SPEAKER_01: I think it was based on GPT-3 if I remember rightly, but it's kind of slightly fine-tuned to be more coding friendly, and started looking at it.
• 10:35
  SPEAKER_01: And um the the research team looking at it, they were came up with a genius approach.
• 10:40
  SPEAKER_01: So they were looking at how good the answers were and um what was the like most inexpensive way in terms of user interaction of dealing with it.
• 10:49
  SPEAKER_01: Right.
• 10:50
  SPEAKER_01: And we went with auto-complete first, because the level of if you go to a chat window and ask it a question, if you get an answer back and it's trash, you're never going to ask a question again.
• 11:01
  SPEAKER_01: Right.
• 11:02
  SPEAKER_01: But with autocompletes, you're typing, and then if it gives you something and it's not it's not what you need, you just keep on typing.
• 11:09
  SPEAKER_01: But then if it gives you an answer back that's good, you're like, oh, that's interesting, and you hit tab, and then you hit tab again, and you're then before you know it, you're tab, tab, tapping through the file.
• 11:17
  SPEAKER_01: I was in I was like, there was five folks on the team, they'd been working on it.
• 11:22
  SPEAKER_01: Um, and then um they sort of brought me into hey, we're thinking about like launching this, like we should talk.
• 11:28
  SPEAKER_01: I was like, okay, so get involved, and I was super skeptical.
• 11:32
  SPEAKER_01: I've been uh it's one of those, it's like there's two places I've been in, I've been completely wrong on things.
• 11:38
  SPEAKER_01: I'll tell you the other one later, because that's related to open source.
• 11:40
  SPEAKER_01: But they the this was one of them.
• 11:41
  SPEAKER_01: I was like, yeah, this is you know AI, whatever, this isn't gonna work.
• 11:45
  SPEAKER_01: And then switched on, and that like early version, which now looking back, you know, like it's so much better than it is now.
• 11:51
  SPEAKER_01: It's so much better now than it was then.
• 11:52
  SPEAKER_01: But that early version, I was like, oh, this is useful.
• 11:56
  SPEAKER_01: Like I can code again.
• 11:58
  SPEAKER_01: I I joked about deleting emails, like I love to code.
• 12:01
  SPEAKER_01: Coding I love doing, but like I don't get didn't get time to and was putting projects off because I knew I I know enough about coding to know how much work something was going to be.
• 12:11
  SPEAKER_01: Um, I was coding again at the weekends and uh uh in my spare time, and I was able to, especially the kind of work I do, where I'm oftentimes doing proof of concept applications and building things out.
• 12:20
  SPEAKER_01: I'm code, I was like, this is this is actually pretty good.
• 12:23
  SPEAKER_01: And then it's just got better and better, you know, in terms of um once the answers have got up to a sufficient level where with additional thought we for autocomplete, you have like a 200 millisecond time budget before you need to be able to get an answer back, because otherwise it's pointless, you know, you've you've carried on typing.
• 12:41
  SPEAKER_01: Um with chat, um, that was uh you know you get longer to think about it, um, but then you can create higher quality answers of that because the models have got better.
• 12:51
  SPEAKER_01: And then with um as we go through chat to edit now agent mode inside of VS Code, you have the ability to um have a prompt and then it takes a lot longer to give back answers, but do a lot more because it's got a lot better.
• 13:05
  SPEAKER_01: And then here at Build, we've kind of taken that to the next level where um you know if you as you're using that and you're doing vibe coding or whatever you want to call it, I hate that term, it makes me cringe, but I don't understand it because I feel that joy when I'm doing it.
• 13:18
  SPEAKER_01: You're doing agent mode, you'd kick it off, and quite often you'll kick it off and you'll do something, and it's gonna go do like quite a lot of stuff.
• 13:24
  SPEAKER_01: So you go have a cup of tea, you'll go check another issue, you do whatever.
• 13:27
  SPEAKER_01: Um so what we've kind of done is made that asynchronous.
• 13:31
  SPEAKER_01: So we've made it so you can now do the agentic stuff in a pull request, go have that interaction, have that work in the background for you, and then when it's ready, come back in, look at it, see where we're at, maybe give the agent some more feedback, have it do some more things either interactively or back in the pull request again.
• 13:47
  SPEAKER_01: And then when you've got to a place where you're happy um and it passes tests and it does all the good stuff, merge that into your repo and off you go and work on the next one.
• 13:55
  SPEAKER_01: And that actually that flow of doing it within a pull request and using GitHub issues and things, that actually works really well with how I work because I can like I can assign it a few, I can look through my backlog, go, you know, you can once you use the tool, you know what it's good at because it's not a silver bullet, you know.
• 14:13
  SPEAKER_01: But you um you know what it's good at.
• 14:15
  SPEAKER_01: You can kind of go, well, this one here, this is pretty, I think it actually quite do quite a good job of this.
• 14:19
  SPEAKER_01: So tick, tick, tick, like these three, I think he's gonna do good on.
• 14:23
  SPEAKER_01: Let's assign those to copilot.
• 14:24
  SPEAKER_01: Like this thing on my backlog, I think I need to dig in a bit more before I understand the problem enough to be able to give good direction.
• 14:31
  SPEAKER_01: So I'm gonna go work on that one while these three are you know the the agents doing some stuff, copilot's doing some stuff, and then you go do your thing, you come back and then you look at it and go, oh no, that that's all done.
• 14:41
  SPEAKER_01: And so you end up not putting stuff off.
• 14:43
  SPEAKER_01: So, like all of that refactoring or like um you know, the a lot of the tech debt stuff that you can kind of put off because you're trying to deliver customer value.
• 14:52
  SPEAKER_01: You end up like put hunting it to copilot, letting it do see how it does, sure, and then go, oh okay, this doesn't suck.
• 14:59
  SPEAKER_01: You know what I mean?
• 15:00
  SPEAKER_01: I'm gonna do that.
• 15:01
  SPEAKER_01: Sorry, that's not very good.
• 15:02
  SPEAKER_01: I you can tell I'm not in marketing, but this is amazing, this is awesome, super awesome.
• 15:07
  SPEAKER_01: Um, and then to take that, use it and uh and move on from there, or you look at it and go, okay, well, that's not quite what I wanted, I'll go do it myself, you know.
• 15:15
  SPEAKER_01: And so um the trick is to do these interactions in a way that help you iterate quicker and don't block your flow, they enhance your flow.
• 15:25
  SPEAKER_01: And that's kind of been the overall uh philosophy behind Copilot as we've built it out over the past few years.
• 15:31
  SPEAKER_03: Yeah, I like that the idea that you're uh essentially assigning work to somebody else.
• 15:36
  SPEAKER_03: Yeah.
• 15:37
  SPEAKER_03: Right?
• 15:37
  SPEAKER_03: You know, you're just saying, hey, intern, go do this, yeah, and then doing a review in the pull request and saying you suck.
• 15:44
  SPEAKER_03: Yeah.
• 15:45
  SPEAKER_03: Or, you know, hey, that that's great, let's let's make that work and implement it.
• 15:48
  SPEAKER_03: Um I was talking to somebody the other day that um uh their code base was not up to a standard.
• 15:55
  SPEAKER_03: Like they weren't using Pascal cases in the right places, they were you know didn't have the right code comments, they didn't have all these things.
• 16:00
  SPEAKER_03: They said, well, what if I just used the agent?
• 16:02
  SPEAKER_03: Just go do that.
• 16:04
  SPEAKER_03: And you know, what would be a laborious task that doesn't really add much value?
• 16:09
  SPEAKER_03: It doesn't add any customer value, yeah, yeah.
• 16:11
  SPEAKER_03: Um but at the same time, it needs to be done.
• 16:12
  SPEAKER_03: Yeah.
• 16:13
  SPEAKER_03: So it's it's more readable and more uh usable.
• 16:16
  SPEAKER_03: Um yeah, it just it just did it for them, and that was excellent.
• 16:19
  SPEAKER_01: And the way you do that is there's a file called copilot instructions, which you can check into your repo.
• 16:24
  SPEAKER_01: You can have those files at the org level as well, um, and you can provide your coding standards.
• 16:30
  SPEAKER_01: So you do things like you know, use camel case and blah blah blah, or um uh you know w how you want your tabs and um and spaces to be.
• 16:38
  SPEAKER_01: Or you can also do things like, hey, when I'm implementing this type of feature in React, like let's wrap it in, maybe maybe use something like launch darkly or something like that for feature flags, let's wrap it in a feature flag condition.
• 16:49
  SPEAKER_01: Um, and then you just build out these coding standards over time.
• 16:52
  SPEAKER_01: And then the more context you give the model, the more context you give co-pilot, the more accurate it can code um based on where you want to go, not on what it's seeing in your code base, which explains where you are.
• 17:06
  SPEAKER_01: And quite often you want to you want to point co-pilot at where you want your code to be, not where your code currently is.
• 17:12
  SPEAKER_01: You know what I mean?
• 17:13
  SPEAKER_01: You're saying that's a difference sometimes, occasionally, maybe it's maybe it's just my code.
• 17:18
  SPEAKER_01: Sure.
• 17:18
  SPEAKER_01: Yeah.
• 17:20
  SPEAKER_03: Excellent.
• 17:21
  SPEAKER_03: So like you're thinking about AI, and you know, Microsoft is of course, you know, build was AI.
• 17:27
  SPEAKER_03: Yeah.
• 17:28
  SPEAKER_03: Like that was the whole conversation.
• 17:29
  SPEAKER_03: Um, but we're thinking about AI and how it's moving forward.
• 17:32
  SPEAKER_03: What do you see as kind of the next steps for Copilot as far as usage?
• 17:37
  SPEAKER_03: Like an everyday person that that's thinking about Copilot, should they be thinking of it as you know, their interns that are working for them?
• 17:44
  SPEAKER_03: Should you think thinking about it as their management and they're letting copilot do all the work?
• 17:49
  SPEAKER_03: Oh right.
• 17:50
  SPEAKER_03: Um, where do you see that fit in the process?
• 17:53
  SPEAKER_01: Um it's much more regardless of what level of developer you are, so um, like where you're a senior dev or a junior dev or whatever, or like somebody who's a hobbyist, treating copilot like it's your team.
• 18:07
  SPEAKER_01: So treating um so because there are different while we call it co-pilot, there are different agents within that, like it's a gentic, it has it has it looks at the conditions, learns, and then you know, give g uh automated uh responses to some stuff.
• 18:22
  SPEAKER_01: So um yeah, so treating it more like a team that you're uh you're the tech lead of or you're the team lead of and you're kind of controlling direction.
• 18:30
  SPEAKER_01: AI isn't creative.
• 18:32
  SPEAKER_01: AI um if you can find a customer that can tell you what your problem is and how to s best solve it with a computer, then sure, like AI could probably do that.
• 18:44
  SPEAKER_01: Like I've never met that customer, they don't know what their problem is, and they certainly don't know what's the most cost-effective, quickest way of solving this with automation because they don't know what the computers are capable of, what's easy and what's hard.
• 18:57
  SPEAKER_01: And as a a good tech lead knows how to solve a problem in an easy way, and so you need to give that direction to Copilot, and then it can do the stuff for you, and then you can kind of um yeah, focus on solving customer problems, like keeping in the the flow of where you're at and trying to background task some stuff and then come back to it, kind of thing, and just generally um I see it improving that speed of iteration, um, making sure you're iterating more quickly and getting you know three out of the system more quickly, but being able to stay in the flow and not have to jump out to things.
• 19:33
  SPEAKER_01: So that's kind of where we're going.
• 19:35
  SPEAKER_01: You talk about like um what's next?
• 19:38
  SPEAKER_01: What's next is actually getting more people um learning how to use these tools and things.
• 19:44
  SPEAKER_01: Um like uh you know, we we've got decent adoption of Copilot, like you know, we um but it's um it and it's since we did copilot copilot's now available for free, certain levels of copilot available for free, and obviously, you know, adoption's gone through the roof there, which is great.
• 20:00
  SPEAKER_01: But typically, like when a company rolls out copilot in an organization, they'll maybe buy you know 3,000 seats, say, um, for their entire company or whatever, um, or 50 or 10 or whatever.
• 20:12
  SPEAKER_01: But that they um the 25 is it the adoption of Copilot follows like a Moore's Law adoption curve where you have like 25-30% of people will just grab it and use it because they like the cool hotness, they'll use it, they'll see how it works, they'll do the work to do that because they're early adopters.
• 20:32
  SPEAKER_01: You then have this kind of um the innovation gap, they call it, but you have to and you have to cross the chasm.
• 20:37
  SPEAKER_01: Um, but it's that kind of rollout, and then you've got a you know, late majority and things, and there's ways to get across that chasm inside organizations and help the overall company learn how to use Copilot and work with it well.
• 20:49
  SPEAKER_01: But developers are busy, they don't want to like you've got a working environment that works, I can solve problems.
• 20:55
  SPEAKER_01: You're telling me to install a thing that's gonna change how I work, like I'm busy, I'm not gonna do that.
• 21:00
  SPEAKER_01: And so different people can slot it in into different times, and like learning how to do that, I think, is the next big push.
• 21:06
  SPEAKER_01: Um, and then we've all obviously building out like more identity capabilities, we've added, you know, so now we can do it in the background.
• 21:13
  SPEAKER_01: How can we now talk to more tools?
• 21:16
  SPEAKER_01: And that you've heard lots of stuff at BuildRound MCP support.
• 21:18
  SPEAKER_01: That's that's the fundamental building block for enabling more tool calling.
• 21:23
  SPEAKER_01: Um how can we add additional agents?
• 21:25
  SPEAKER_01: So we've we have a code review agent now as part of Copilot, so it can do your code reviews for you as well as assign it work, which is that's awesome because I live in the UK and when I'm doing stuff, people are often asleep.
• 21:39
  SPEAKER_01: And so having being able to give Copilot a first pass before any humans look at my code is fabulous.
• 21:44
  SPEAKER_01: Um, we also showed some stuff this week around like Java migrations and.NET migrations.
• 21:50
  SPEAKER_01: If you think about what's holding you back on.NET framework sort of thing and going towards a newer version of.NET, oftentimes it's that all of that stuff.
• 21:59
  SPEAKER_01: The migration we That nobody nobody wants to do and no customer wants to pay for.
• 22:04
  SPEAKER_01: If we like if we can't build an amazing.NET migration wizard, who can?
• 22:08
  SPEAKER_01: You know?
• 22:09
  SPEAKER_01: Sure.
• 22:09
  SPEAKER_01: So we should be able to do that.
• 22:10
  SPEAKER_01: We sh but we should do the same for Java as well.
• 22:12
  SPEAKER_01: And we should do, you know, for all sorts of different things, like how do we get COBOL code into Python or whatever it is, you know?
• 22:18
  SPEAKER_01: Like um, those are the areas where I think we get more and more specialized agents that are part of this kind of framework and tools, and then more and more tools plugging in.
• 22:27
  SPEAKER_01: And that's where I think GitHub's uniquely well positioned to be kind of the hub of these tools because everybody integrates their stuff into GitHub.
• 22:34
  SPEAKER_01: And so we by using things like MCP, we can do it in an open way as well, which again is very much down to kind of GitHub's core culture and ethos.
• 22:43
  SPEAKER_03: That's interesting.
• 22:44
  SPEAKER_03: I mean, you talk about the idea of GitHub affecting Microsoft, and I think it's a big way that they're coming in and saying, hey, let's be open, let's be community focused rather than just the bad old days of Microsoft was you know, let's let's close the game.
• 22:57
  SPEAKER_03: Yeah.
• 22:57
  SPEAKER_03: Um I think that that's some of the agentic stuff, you know, and the the way they tie into things that are nothing to do with you know C sharp or Visual Studio or or something like that has just been amazing.
• 23:09
  SPEAKER_03: Yeah.
• 23:09
  SPEAKER_03: Um so let's talk a bit about you know the usage of models.
• 23:14
  SPEAKER_03: Yeah.
• 23:14
  SPEAKER_03: You know, models aren't free.
• 23:15
  SPEAKER_03: Yeah.
• 23:16
  SPEAKER_03: Um and you've got a free version of of Copilot.
• 23:20
  SPEAKER_03: Um, how does that work with paying for an LM or using a free LLM?
• 23:25
  SPEAKER_03: How does that work?
• 23:26
  SPEAKER_03: And is there kind of a stopping point or a a pay point or something like that?
• 23:30
  SPEAKER_01: Yeah.
• 23:30
  SPEAKER_01: So um yeah, we have a free version, and that's kind of enough.
• 23:34
  SPEAKER_01: You know, we looked at kind of the usage of average developers and how much we can afford to fund and kind of strike a balancing point.
• 23:42
  SPEAKER_01: So that if you're like coding on the weekends, if you're doing like um if you're in school, things like this, because we give copilot for free, copilot pro for free to students and teachers as well.
• 23:54
  SPEAKER_01: But if you're like coding and you know, doing some stool stuff, getting to learn, copilot usage of like um the sort of the good models um is f uh free for you.
• 24:09
  SPEAKER_01: Like it should you shouldn't have have to pay.
• 24:11
  SPEAKER_01: Uh because then but if you're coding for a job and you're doing it like, you know, eight well, we don't code for eight hours a day, we code for like four hours a day if it's a good day, but maybe we can go for eight hours a day with AI.
• 24:22
  SPEAKER_01: But um like if you're doing it for a job, then probably you want to be either getting um Copilot Pro, which is like the the sort of the uh a paid version, which is you know gives you more requests, more like um the it's all based on the number of LLM calls, basically, um which is how we pay for things in our back end.
• 24:41
  SPEAKER_01: We're a we're a consumer of um Azure AI, just like any other person out in the conference hall.
• 24:46
  SPEAKER_01: Um quite a big one, but we're you know, and and obviously um Claude Sonnet and um you know uh Google uh blanking again.
• 24:57
  SPEAKER_01: I'm sorry, yeah, the Gemini models like we're just consumers of those models kind of thing, and then we have a bunch of infrastructure below them and over them, which we can dig into if you're interested.
• 25:06
  SPEAKER_01: But um, yeah, those we're just consumers of those models, we pass them on, but we've got obviously a set budget where we're handling all the free stuff.
• 25:13
  SPEAKER_01: Um, and then we pay for pro that gives you more requests, and it gives you access to more models because different models have different price points, and then finally, uh we have like uh and then we also have business for enterprises to to go into, and then the final level is like Pro Plus and Enterprise, and that's where you get a lot more requests and you get access to a lot more models, including some of the really expensive ones like 01 or something like this, which have like advanced reasoning capabilities but are significantly s slower and more expensive, and so you use them when you are stuck.
• 25:48
  SPEAKER_01: Um, you don't use them for the um you don't use definitely don't use them for autocompletes, like you can't even enable it for that, but you use them for um you know when I when I when I need reasoning to help me solve a problem, otherwise, you know, GPT-4.1 is a fabulous model, and that's now like the default model kind of thing, or you can use um Claude Sonnet, and then different people have different preferences between 3.5 and 3.7.
• 26:13
  SPEAKER_01: What was really interesting when we started, and this is the biggest difference, there was clearly a model that could do this better than anything else on the market.
• 26:21
  SPEAKER_01: And then the advancements in LLMs, like new models coming out all the time, and we're keeping up with all that and making sure we we we we bring those to developers.
• 26:30
  SPEAKER_01: Um we've moved to a world now where we it's all about giving the developer the choice of which LLM they want to use because the LLMs are just advancing so quickly, it's starting to get to the point where no one is clearly better, but they have different not personalities because I don't want to anthropomorphise.
• 26:46
  SPEAKER_01: Well, they have different ways they work, sure.
• 26:48
  SPEAKER_01: Like Claude um sort of GPT-4.0 was quite verbose in its answers.
• 26:54
  SPEAKER_01: GPT-4.1 is a bit less verbose.
• 26:56
  SPEAKER_01: Um Claude 3.5 does really nice, tersy, sharp code and will use like lovely link statements pre usually.
• 27:04
  SPEAKER_01: Um 3.7 is good, but in agent mode, I love 3.7 personally, Claude 3.7, but in agent mode, but for some people they find it's um they call they almost say it's like too assertive in things it wants to touch.
• 27:17
  SPEAKER_01: Interesting.
• 27:17
  SPEAKER_01: And so they they would prefer to use say Claude 3.5 in agent mode, but use GPT-4.1 in ask mode, and like people have different preferences, so we need to give um developers those controls to be able to do that.
• 27:30
  SPEAKER_01: I I would love to give an auto mode.
• 27:33
  SPEAKER_01: Um the auto mode for us right now is go use GPT-4.1, that's like what we call base, that's the base model.
• 27:38
  SPEAKER_01: Um, go use that because it's good for most things.
• 27:41
  SPEAKER_01: Um, but um it would be great to have an auto mode where we're inferring a lot more, but right now the advancements in LLMs are so fast, it's easier to give you the stick shift and then you decide rather than us build a rule in which then changes tomorrow or the day after.
• 27:57
  SPEAKER_01: Like literally that this week that's what's happening as you know, new announcements are coming out and things.
• 28:02
  SPEAKER_03: Sure.
• 28:02
  SPEAKER_03: Yeah, and that's I mean it's an interesting analogy that stick shift because that's you know, I grew up with a stick shift.
• 28:07
  SPEAKER_03: Um, dad taught me that stick shift, which is awesome.
• 28:09
  SPEAKER_03: Um, but you know, now we have automatics throughout the country, and it's easy because of the fact that we've learned a lot and it's it's easy to put an automatic in a car, and I think that's where we're getting to is eventually we'll have that automatic capability of hey, this we know what's best for you, let's let's help you out within auto mode.
• 28:27
  SPEAKER_03: Yeah.
• 28:27
  SPEAKER_03: Um, but for now we're we're still in the early phases of like you said, the models continually change.
• 28:32
  SPEAKER_01: Yeah.
• 28:32
  SPEAKER_01: Um but the internal combustion engine hasn't changed that much, and then we've had the introduction of like hybrids and elect you know, electric motors and things, but even those, they haven't changed that much.
• 28:42
  SPEAKER_01: LLM advancements are just it's incredible how fast things are moving.
• 28:48
  SPEAKER_01: Like so much though that it's usually better.
• 28:52
  SPEAKER_01: We've we've done we've done experiments with customers, we've done lots of stuff like we have fine-tuned models of things for like say um different like semiconductor companies or like the office team, because their their C code has so many templates and stuff, it almost resembles a completely different language.
• 29:09
  SPEAKER_01: So we did a fine-tuned model for them and got a much greater like code acceptance, which is a thing that we tend to track.
• 29:16
  SPEAKER_01: Um but the the interesting thing is that um the next model comes out, and actually its acceptance is better than the fine-tuned version of this.
• 29:26
  SPEAKER_01: Interesting.
• 29:26
  SPEAKER_01: And so then it becomes, oh, okay, maybe we should focus on improving the speed of access of delivery of new models to people, so then you get those enhanced capabilities, and improving our ability to do retrieval uh rag.
• 29:40
  SPEAKER_01: So improving our ability to give that model the context it needs to give you the better answers.
• 29:45
  SPEAKER_01: Focus on those things because then you're gonna get better results rather than focusing on making yesterday's LLM really well suited to you, if you know what I mean.
• 29:55
  SPEAKER_03: So I think that I I've seen the agent mode, I love agent mode.
• 29:59
  SPEAKER_03: Oh right.
• 30:00
  SPEAKER_03: I love it.
• 30:00
  SPEAKER_03: Right.
• 30:01
  SPEAKER_03: I love the fact that you know, yes, models are smart, yeah.
• 30:05
  SPEAKER_03: There's a lot of great models, but it's more about the fact that it looks at my code and says, okay, here's a context.
• 30:10
  SPEAKER_03: Yeah.
• 30:10
  SPEAKER_03: You know, because that context is so huge.
• 30:12
  SPEAKER_03: Yeah.
• 30:12
  SPEAKER_03: You know, I I don't do things the same way the next developer does.
• 30:15
  SPEAKER_03: Let's get that context in place that that it can make an informed decision rather than a in-the-dark decision.
• 30:22
  SPEAKER_03: Um so let's talk a bit about vibe coding.
• 30:25
  SPEAKER_03: Yeah.
• 30:25
  SPEAKER_03: You mentioned it earlier.
• 30:26
  SPEAKER_03: Yeah, yeah, yeah.
• 30:27
  SPEAKER_03: Um, and it's kind of been, you know, this thing has grown over time of hey, I can do everything with an LLM.
• 30:33
  SPEAKER_03: Uh, what are your thoughts on vibe coding and and where we're at with that?
• 30:36
  SPEAKER_01: Um the term exists because using agent mode is joyful.
• 30:41
  SPEAKER_01: And you can do a ridiculous amount really, really quickly by you know looking at what's coming out of the result and like passing in requirements, viewing what was done, and like judging it based on this.
• 30:55
  SPEAKER_01: You know what I mean?
• 30:55
  SPEAKER_01: So it's it's almost rather than being the tech lead, you're now the business stakeholder in terms of how you're working with your team.
• 31:02
  SPEAKER_01: Um so I get it.
• 31:04
  SPEAKER_01: Um I I do have um the people who are most effective at using these AI tools are the people with the best DevOps guardrails.
• 31:16
  SPEAKER_01: Uh maybe an old-fashioned thing, but um because that then helps you accelerate your delivery safely, um you need kind of um you need to be checkpointing regularly.
• 31:29
  SPEAKER_01: Uh you need to be able to like because you might have a working version, go tell the agent another thing, and all of a sudden it's trashed what you've got.
• 31:36
  SPEAKER_01: And if you didn't, um if you weren't testing that particular area, um then you don't notice, and then you've got way far down, and then you realize you broke something.
• 31:45
  SPEAKER_01: So all of that good hygiene that we got used to doing, we need to be doing still.
• 31:49
  SPEAKER_01: So you need to be committing regularly.
• 31:51
  SPEAKER_01: Um, if you've got tests and things, like they help the agent get better because it can run your tests for you to make sure it hasn't broken stuff.
• 32:00
  SPEAKER_01: So vibe coding I love, um, and I do a lot of it in my spare time.
• 32:04
  SPEAKER_01: How we can take those productivity improvements and apply them into work is I think where we're doing things like agent, you know, uh the agentic stuff where you can assign it to a PR.
• 32:16
  SPEAKER_01: Because now you can do vibe coding, but within a PR with commits, with checkpoints, with CI on the back end, with uh with the ability to review and have peer review of your team.
• 32:28
  SPEAKER_01: You know, that's where I see things kind of going.
• 32:31
  SPEAKER_01: But yeah, I I enjoy it, so I get why it's there, but equally it makes me cringe when people people are like, oh just YOLO everything, you know what I mean?
• 32:37
  SPEAKER_01: And you get the classic like um, you know, I've deployed my website, is here's my address, local host, sort of thing.
• 32:43
  SPEAKER_01: Um like yeah, you need to still understand what it's doing and that like you still need you need devs more than you ever have need devs to that you make sure that actually what we're doing is the right thing to do, right?
• 32:56
  SPEAKER_01: Being that um informed manager of of agents, right, uh and um being able to give it the creative direction and the ability to solve like we've seen different you know different solutions to solve the same problem, and some work and some don't, and it's like how do you get that crisp minimum amount of work to solve a problem solution to something that's the most flexible?
• 33:21
  SPEAKER_01: Uh that's where the creativity comes, and that's what as developers, and as you rise in seniority, that's what you're learning is how to solve problems, how to do less work and solve the problems, not how do I how do I be a hundred X developer or something like uh a senior engineer writes less code than a junior engineer, and it's not because they're solving less customer problems, they just know how to solve more customer problems with right by writing less code.
• 33:45
  SPEAKER_03: Absolutely.
• 33:46
  SPEAKER_03: Yeah, I think that the the idea that it's the um the guardrails you put in place that's really important.
• 33:53
  SPEAKER_03: You know, the it's even more important to have good testing, it's even more important to have a good CI system, it's even more important to have those processes in place, code reviews, you know, where you're looking at what's happening and managing it well.
• 34:06
  SPEAKER_03: Yeah, you know, so making informed decisions about is this good code, is this the right code, is it secure code, is it you know, instrumented code, all that good stuff, um is still very important, but um but I do think it moves us forward a lot faster.
• 34:21
  SPEAKER_01: Oh, yeah, and um yeah, and then you also it's not just in the Devon loop, it's in the deployment inner loop as well, and making sure you can get it out into production quickly and easily.
• 34:30
  SPEAKER_01: Like um, do you have you know ephemeral production environments?
• 34:34
  SPEAKER_01: Are you able to deploy like to environments with no environment drift?
• 34:38
  SPEAKER_01: Are you able to learn from what's happening production and then use that to give you feedback of where you started?
• 34:45
  SPEAKER_01: Um that's where we're seeing people who are more advanced there are getting leveraging AI tools a lot better.
• 34:54
  SPEAKER_01: Um if you um I did some pithy statement that like um you know vibe coding um without doing that is a bit like doing test-driven development without the tests.
• 35:05
  SPEAKER_01: You're just iterating, you're just pushing stuff out there, but you don't know if you're going in a good way or not.
• 35:10
  SPEAKER_01: And so uh I think this is like responsible vibe coding or ethical vibe coding or something like that.
• 35:14
  SPEAKER_01: I'll try and coin a phrase that definitely won't catch on.
• 35:17
  SPEAKER_01: But there's there is one we're talking about with a gentic DevOps, you know, being able to use these agents in a DevOps way, because maybe what we can do with things like what's stopping you from having good DevOps principles today, maybe it's like your tech debt.
• 35:29
  SPEAKER_01: So if we can build agents to help you skip a line in terms of, well, I've you know what's stopping me from going to the cloud is I'm on a really old version of Java and none of the cloud providers support it.
• 35:39
  SPEAKER_01: Well, let's help you upgrade your Java version, like and then and then now you can get to the cloud, and now you can be a bit more cloud native.
• 35:44
  SPEAKER_01: And you know, so maybe we can help accelerate the people who are not as mature in DevOps to get there, but the people who are more mature in DevOps, man, they can just go so much faster now.
• 35:53
  SPEAKER_03: Excellent.
• 35:54
  SPEAKER_03: And they can write YAML, so that's that's a huge bonus for me.
• 35:57
  SPEAKER_01: Yeah, it really can.
• 35:58
  SPEAKER_01: And and when you are writing YAML, the tick is there as well in your prompt to give it an example of the YAML that you want to write.
• 36:05
  SPEAKER_01: Um, one of the tricky things I found as well is sometimes I'll want it to generate a snippet of YAML for me, but it will indent the YAML back as if it's at the root because it's it's trying to build a fully constructed correct YAML file.
• 36:18
  SPEAKER_01: And if you're if you're generating a subsection of it, it's like no, put the four spaces on the front or whatever.
• 36:23
  SPEAKER_01: That's a big deal in that.
• 36:24
  SPEAKER_01: It really is.
• 36:25
  SPEAKER_01: So I mean, but that's where you have to give it the whole context, you know what I mean?
• 36:28
  SPEAKER_03: But um yeah, but yeah, excellent.
• 36:31
  SPEAKER_03: Excellent.
• 36:31
  SPEAKER_03: I appreciate it.
• 36:32
  SPEAKER_03: Um before we go, I you know, you've been in development for a long time, and of course you've you know now you're at GitHub.
• 36:38
  SPEAKER_03: Um, do you have any advice for newer developers, developers getting in the field?
• 36:43
  SPEAKER_03: What are your thoughts um that you can share?
• 36:46
  SPEAKER_03: What are your wisdom?
• 36:46
  SPEAKER_01: Yeah, um I've got I've got skin in the game here.
• 36:49
  SPEAKER_01: I've got a kid who's like 21, he's in his third year of a four-year master's doing computer science.
• 36:55
  SPEAKER_01: Like um, and trying to sort of see how we're helping this new generation learn how to how to use these tools to be effective.
• 37:07
  SPEAKER_01: Um, the AI tools are undoubtedly make you quicker at developing, so we need to make sure we help them learn how to use them and get advantage of them, and when not to use them, when to switch them off because you need to learn or something like that.
• 37:20
  SPEAKER_01: So, um my advice for developers is to um learn these tools, learn how they can be a part of your workflow, and learn when when it makes sense to use them.
• 37:32
  SPEAKER_01: Don't be um, and this is for people coming into the industry as well as existing developers, don't be dismissive of them.
• 37:38
  SPEAKER_01: Um, and then for new developers coming in, it's focus on what the customer pain points like, focus on how do I solve this problem, focus on learning the different technology stacks and what they're capable of.
• 37:52
  SPEAKER_01: So that you're more of like a solutions architect or a lead dev when you're coming in as a junior dev.
• 37:57
  SPEAKER_01: Um you are helping your AI tools and your AI agents build the right answer for that solution, and that might well be build me a static website that does some stuff rather than build like a massive end-to-end application with like Redis and blah blah blah and all this, you know.
• 38:14
  SPEAKER_01: Um, so again, that's where a senior developer has experience over a junior developer is what's the simplest way to solve this problem, and you still need that with AI, because otherwise you risk the AI helping you go really quickly in the wrong direction.
• 38:27
  SPEAKER_01: So, um, yeah, so that's it's not very short pithy advice, but no.
• 38:30
  SPEAKER_01: Oh, excellent.
• 38:31
  SPEAKER_03: Okay, excellent.
• 38:31
  SPEAKER_03: I appreciate it.
• 38:32
  SPEAKER_03: Okay.
• 38:32
  SPEAKER_03: Well, thanks for being here.
• 38:34
  SPEAKER_03: Um, and as always, I am Tim Corey.
• 38:37
  SPEAKER_00: Thank you for joining us for this episode of Dev Questions.
• 38:40
  SPEAKER_00: When you're ready to learn to think and code like a professional developer, head over to imtimcorey.com and enroll in a course.