Latest / AI Builds It: Easy Coding Tools / Research Priorities for the Next 18 Months: Where Autonomous Coding Should Go Next

Transcript

Research Priorities for the Next 18 Months: Where Autonomous Coding Should Go Next · AI Builds It: Easy Coding Tools

• 0:00
  SPEAKER_00: Research Priorities.
• 0:01
  SPEAKER_00: The next 18 months of autonomous coding.
• 0:05
  SPEAKER_00: AI-powered coding assistants are already transforming software development.
• 0:09
  SPEAKER_00: By late 2025, tools like GitHub Copilot and AI chatbots are being used daily by most developers.
• 0:16
  SPEAKER_00: And even non-programmers can prototype code with simple prompts.
• 0:20
  SPEAKER_00: Google's CEO notes that this trend, often called vibe coding, is making programming more approachable for non-technical staff.
• 0:28
  SPEAKER_00: However, real-world deployments have exposed important gaps.
• 0:33
  SPEAKER_00: AI-generated code often contains subtle bugs, fails on complex projects, and raises accountability and policy issues.
• 0:41
  SPEAKER_00: To move from lab demos to reliable production systems, we need focused research on four fronts reliability, long horizon planning, verifiability, and socio-technical governance.
• 0:52
  SPEAKER_00: Below we outline key open problems and propose research agendas, benchmarks, and collaborations to tackle them.
• 0:59
  SPEAKER_00: Reliability and code quality.
• 1:02
  SPEAKER_00: A major issue is basic reliability.
• 1:04
  SPEAKER_00: Code written by AI assistants still contains significantly more errors than human code.
• 1:09
  SPEAKER_00: For example, an analysis of 470 GitHub pull requests found AI-written PRs had about 1.7x more issues than human-written ones.
• 1:19
  SPEAKER_00: On average, AI PRs triggered about 10.8 issues, logic bugs, naming or formatting problems, security flaws, etc., versus 6.5 for human PRs.
• 1:30
  SPEAKER_00: Notably, the AI-authored code had a heavier tail of severe bugs.
• 1:34
  SPEAKER_00: Logic errors and security vulnerability appeared nearly twice as often as in human code.
• 1:39
  SPEAKER_00: In practice, teams using AI tools have reported surprises.
• 1:43
  SPEAKER_00: Code that looks correct in isolation but fails integration or carries hidden flaws.
• 1:48
  SPEAKER_00: Indeed, a comprehensive survey of code generation tools observes that existing benchmarks do not capture the kinds of failure modes seen in production.
• 1:57
  SPEAKER_00: Hallucinated API calls, inconsistent naming, or subtle logic errors that slip past unit tests.
• 2:03
  SPEAKER_00: In short, AI can generate working code snippets, but those snippets are often not ready for production.
• 2:09
  SPEAKER_00: Developer experience echoes this mistrust.
• 2:12
  SPEAKER_00: A large sonar source survey found that while 72% of engineers use AI tools daily to write up to 42% of code, a staggering 96% admit they do not fully trust the AI output.
• 2:24
  SPEAKER_00: Yet fewer than half of teams always review AI-generated code before committing.
• 2:29
  SPEAKER_00: This gap, high usage but low trust, leads to what experts call verification debt.
• 2:34
  SPEAKER_00: Without better reliability, organizations risk introducing hard-to-catch bugs and technical debt whenever they adopt AI coding shortcuts.
• 2:42
  SPEAKER_00: Research agenda.
• 2:43
  SPEAKER_00: We need systematic study of error patterns in AI code and new methods to mitigate them.
• 2:49
  SPEAKER_00: Ideas include automated AI proofing, integrating static analyzers, or secondary models that scan AI output for common mistakes, similar to a second reviewer.
• 3:02
  SPEAKER_00: Better LLM training objectives could focus on stability, for example, training on buggy versus clean code examples, to teach the model to prefer safer solutions.
• 3:13
  SPEAKER_00: Researchers should analyze which types of code, algorithms, I.
• 3:33
  SPEAKER_00: We must codify these failure modes.
• 3:35
  SPEAKER_00: Educational solutions can help too.
• 3:38
  SPEAKER_00: As community guidelines stress, AI tools can only assist, humans must verify.
• 3:44
  SPEAKER_00: To encourage this, future tools could auto-generate warnings or even refuse to handle tasks without human approval.
• 3:51
  SPEAKER_00: Benchmarking should shift, moving beyond, does this code compile to how many subtle issues remain?
• 3:58
  SPEAKER_00: For example, code review AI models are emerging that specifically measure bug detection performance.
• 4:04
  SPEAKER_00: A community effort to produce a public data set of real AI versus human code changes with annotated defects akin to CodeRabbit's PR study would let researchers track progress in reliability.
• 4:17
  SPEAKER_00: Long horizon planning and maintenance AI code generators excel at small, self-contained tasks, but large projects expose their limits.
• 4:26
  SPEAKER_00: Real software evolves over time, with changing requirements, multiple files, and architectural decisions to manage.
• 4:34
  SPEAKER_00: Surveys note that generating correct isolated functions differs qualitatively from maintaining coherent architectural decisions across a large code base.
• 4:44
  SPEAKER_00: In practice, even state-of-the-art models struggle with multi-step, multi-file tasks.
• 4:49
  SPEAKER_00: Two recent benchmarks highlight this gap.
• 4:52
  SPEAKER_00: Roadmap Bent, May 2026, evaluates long horizon upgrades on real open source projects.
• 4:59
  SPEAKER_00: Each task gives the agent a base version of a project and a list of features to implement, with 3,700 lines change across 50 plus files.
• 5:08
  SPEAKER_00: Even Claude Opus 4.7, one of the strongest models, solved only 39% of tasks, and other models fell as low as 5%.
• 5:18
  SPEAKER_00: By contrast, simple one-shot bug fixes see near-perfect AI performance.
• 5:23
  SPEAKER_00: Roadmap bench authors conclude that long horizon software development remains a largely unsolved problem.
• 5:30
  SPEAKER_00: Slopcode Bench 2026 examines iterative development.
• 5:34
  SPEAKER_00: Agents were given a task and formed code, then over 20 rounds, the task specification changes, forcing code to evolve.
• 5:42
  SPEAKER_00: The result, even though all intermediate versions passed existing tests, the AI-generated code bases became 2.2x more verbose and much harder to maintain than human-maintained code.
• 5:54
  SPEAKER_00: In fact, none of the top models solved the full sequence, 5% by the final checkpoint.
• 6:00
  SPEAKER_00: This shows that small design mistakes accumulate under AI assistance, hindering future modifications.
• 6:07
  SPEAKER_00: These findings suggest research focus on planning and decomposition.
• 6:12
  SPEAKER_00: AI systems should not just write code per prompt, but plan multi-step strategies.
• 6:18
  SPEAKER_00: One emerging idea is plan and execute.
• 6:21
  SPEAKER_00: Let the model first outline a design or sequence of steps, then generate code for each step.
• 6:27
  SPEAKER_00: In fact, analyses of coding agents, clawed code, GitHub Copilot, etc., find that separating planning from execution and exposing the plan to the user dramatically improves performance on complex tasks.
• 6:40
  SPEAKER_00: Research should develop new architectures.
• 6:43
  SPEAKER_00: For example, nested agents, where a manager LLM breaks a big problem into subtasks for worker LLMs.
• 6:50
  SPEAKER_00: Long-term memory mechanisms are also needed.
• 6:53
  SPEAKER_00: Future models should remember code generated earlier in a session, even beyond the context window.
• 6:59
  SPEAKER_00: Benchmarks.
• 7:00
  SPEAKER_00: The community should define benchmarks that reflect real dev work.
• 7:04
  SPEAKER_00: Going beyond roadmap bench, we need tasks spanning multiple languages and integration challenges.
• 7:09
  SPEAKER_00: Front-end, back-end, databases, etc.
• 7:12
  SPEAKER_00: Simulated team projects would test how AI and humans collaborate over releases.
• 7:18
  SPEAKER_00: Borrowing ideas from software engineering, benchmarks could measure not just correctness, but maintainability.
• 7:24
  SPEAKER_00: How easy is it to add a new feature?
• 7:26
  SPEAKER_00: Performance.
• 7:27
  SPEAKER_00: Does AI code degrade as it evolves?
• 7:29
  SPEAKER_00: And integration, does it fit existing style conventions?
• 7:33
  SPEAKER_00: For example, benchmarks could start with an existing code base and ask the agent to implement a series of feature requests or refactorings with periodic tests.
• 7:42
  SPEAKER_00: Over the next 18 months, creating such open challenges, perhaps via academic industrial contests, will guide research into multi-stage coding.
• 7:52
  SPEAKER_00: Verifiability and formal interfaces.
• 7:55
  SPEAKER_00: As AI assistants attempt more critical tasks, ensuring correctness becomes essential.
• 8:00
  SPEAKER_00: Verifiability means linking code to precise specifications or test suites so we can be confident it does what we want.
• 8:08
  SPEAKER_00: In classic engineering, one writes a formal spec or thorough tests before coding.
• 8:12
  SPEAKER_00: How do we bring this mindset to AI-driven coding?
• 8:16
  SPEAKER_00: One opportunity is closed loop generation.
• 8:18
  SPEAKER_00: Recent work proposes that an AI-generated code, its dock string, and any formal annotations should be checked for consistency.
• 8:26
  SPEAKER_00: For instance, the Clover approach automatically generates formal specs using languages like Daphne alongside code and then uses proof tools to reject inconsistent solutions.
• 8:36
  SPEAKER_00: In early tests, this detected all incorrect programs on a textbook-level dataset.
• 8:42
  SPEAKER_00: Similarly, AutoACSL uses static analysis to prompt an LLM to write precise function contracts, pre-post conditions, and then verifies them with PROMACE.
• 8:52
  SPEAKER_00: By feeding back unsatisfied conditions, it dramatically improved the percentage of provably correct code.
• 9:00
  SPEAKER_00: These examples show that integrating formal methods at the code generation step can turn an uncontrolled AI guess into a verified program.
• 9:09
  SPEAKER_00: Apart from formal math, we also need better interfaces between informal specs, tests, and code.
• 9:16
  SPEAKER_00: Today, it is common to describe a function in English and hope the AI does the right thing.
• 9:22
  SPEAKER_00: But we should also have the AI generate or ask for test cases, type annotations, and design comments.
• 9:29
  SPEAKER_00: For example, a prompt can first ask the model to describe the algorithm or invariance in natural language or pseudocode, and only then to code it.
• 9:40
  SPEAKER_00: Or we could use contract-first development, write unit tests or property tests that the AI must satisfy.
• 9:47
  SPEAKER_00: Rough sketches of these ideas have shown promise.
• 9:50
  SPEAKER_00: Even generating a few example-based tests can steer the model away from trivial solutions.
• 9:56
  SPEAKER_00: Benchmarks.
• 9:57
  SPEAKER_00: New benchmarks should include formal check problems.
• 10:01
  SPEAKER_00: For example, we could add tasks where the correctness is verified by a theorem prover or symbolic checker, not just unit tests.
• 10:09
  SPEAKER_00: Data sets of user stories with LTL, TLA Plus, or alloy specs, and corresponding code would be valuable.
• 10:17
  SPEAKER_00: In education, competitions like the TLA Plus model check challenge show that specifying is hard.
• 10:23
  SPEAKER_00: One study found current LLMs achieve only about 8% semantic correctness on straightforward TLA plus specs.
• 10:31
  SPEAKER_00: Open source projects might release specification languages more widely, a sort of coding affidavit.
• 10:37
  SPEAKER_00: Standardized formats, YAML, JSON for API specs or data schemas, could be leveraged by AI to align code with intended behavior.
• 10:46
  SPEAKER_00: So swore technical governance and trust?
• 10:48
  SPEAKER_00: Finally, autonomous coding raises human and policy issues.
• 10:52
  SPEAKER_00: Who is responsible for AI code?
• 10:55
  SPEAKER_00: How do we ensure security, copyright compliance, and accountability?
• 10:59
  SPEAKER_00: Several organizations have begun addressing this, but open questions remain.
• 11:04
  SPEAKER_00: Developer practices.
• 11:05
  SPEAKER_00: As mentioned, industry surveys show a trust gap.
• 11:09
  SPEAKER_00: Developers know they should review AI output, but often skip it if it's easier, leading to unmanaged risk.
• 11:15
  SPEAKER_00: In response, major projects have set explicit rules.
• 11:19
  SPEAKER_00: For instance, the Open Infra Foundation permits AI assistance only if commits are labeled with an assisted by or generated by tag.
• 11:27
  SPEAKER_00: Google's Chromium project similarly requires that authors fully understand any AI suggested code, or else lose commit privileges.
• 11:35
  SPEAKER_00: Mozilla's Firefox policy states bluntly, AI can assist, but responsibility always stays with the human behind the change.
• 11:43
  SPEAKER_00: Even the NumPy project warns that you must be able to explain any code submitted.
• 11:48
  SPEAKER_00: These policies underscore that technical tools alone are insufficient.
• 11:53
  SPEAKER_00: We also need clear workflows and culture.
• 11:56
  SPEAKER_00: Regulation and standards.
• 11:58
  SPEAKER_00: On a broader scale, governments and standards bodies are catching up.
• 12:03
  SPEAKER_00: The EU is finalizing a code of practice for general-purpose AI, which will require transparency and safety measures from AI model providers.
• 12:12
  SPEAKER_00: While this is not specific to coding, it signals tighter scrutiny on training data licenses and model explainability.
• 12:19
  SPEAKER_00: Both highly relevant if your code assistant pulled from copyrighted code.
• 12:24
  SPEAKER_00: Similarly, ISO and IEEE have begun AI standards for governance and ethics, though only a few directly address code generation.
• 12:33
  SPEAKER_00: The AI Act EU and upcoming U.S.
• 12:36
  SPEAKER_00: guidelines will likely influence how companies vet AI code internally.
• 12:40
  SPEAKER_00: Collaboration needed.
• 12:42
  SPEAKER_00: Closing these socio-technical gaps will require joint efforts.
• 12:46
  SPEAKER_00: Academia can study how AI tools affect team productivity, vulnerability discovery, and licensing.
• 12:52
  SPEAKER_00: Industry can share anonymized data on real AI-related incidents.
• 12:57
  SPEAKER_00: Standard bodies like W3C, IEEE can incorporate coding scenarios into ethical AI guidelines.
• 13:04
  SPEAKER_00: For example, workshops could bring together SAT EL, software assurance experts, with ML folks to define evaluation criteria for AI code safety.
• 13:13
  SPEAKER_00: Guidelines could evolve into standards, e.g., IEEE 8201 AI-assisted software process, giving organizations a common framework.
• 13:22
  SPEAKER_00: Over the next 18 months, building consensus on best practices through white papers, consortia, or open source policy templates will help teams adopt these tools responsibly.
• 13:33
  SPEAKER_00: Research and Benchmarks Agenda.
• 13:35
  SPEAKER_00: To summarize, we suggest the following concrete steps for the research community.
• 13:40
  SPEAKER_00: Augmented benchmarks.
• 13:41
  SPEAKER_00: Develop a suite of benchmarks that mimic real software projects.
• 13:45
  SPEAKER_00: For example, multimodule frameworks, web apps, APIs, embedded systems, where the AI must implement new features and then maintain them.
• 13:54
  SPEAKER_00: Include evolving specs, simulating changing requirements.
• 13:57
  SPEAKER_00: Measure not just test pass rates, but code complexity, readability, security metrics, and review workload.
• 14:04
  SPEAKER_00: Work with industry to source real bug fix histories and feature requests as benchmark tasks.
• 14:09
  SPEAKER_00: Error taxonomy study.
• 14:11
  SPEAKER_00: Systematically categorize the kinds of bugs AI introduces.
• 14:15
  SPEAKER_00: CodeRabbit's report gave an initial breakdown, logic errors, naming issues, etc.
• 14:20
  SPEAKER_00: A larger academic study could gather PR data and classify AI versus human errors.
• 14:25
  SPEAKER_00: This would guide new model losses, e.g., extra weight on security and automated detectors, tools that flag typically AI-gone-wrong patterns, planning and multi-agent research.
• 14:36
  SPEAKER_00: Explore architectures like planner-executor agents.
• 14:39
  SPEAKER_00: Investigate how to give AI systems some form of memory across sessions or enforce hierarchical planning.
• 14:45
  SPEAKER_00: Collaborate with existing work in agentic AI and robotics, repurposing multi-step reasoning methods for code.
• 14:53
  SPEAKER_00: Formal methods integration.
• 14:55
  SPEAKER_00: Invest in research like Clover and Auto ACSL that tie in program synthesis and proofs.
• 15:02
  SPEAKER_00: Encourage formal methods researchers to partner with NLP ML groups.
• 15:07
  SPEAKER_00: For instance, academic competitions could pair LLM code assistants with provers on shared tasks.
• 15:13
  SPEAKER_00: Create competitions for AI-generated proofs or contract inference.
• 15:18
  SPEAKER_00: Governance frameworks.
• 15:20
  SPEAKER_00: Social science studies on team practices and liability.
• 15:23
  SPEAKER_00: For example, run developer studies, give teams AI tools, and observe how they review and debug.
• 15:30
  SPEAKER_00: Legal research on IP.
• 15:32
  SPEAKER_00: As one blog notes, the copilot copyright problem, unlicensed code, is an open issue.
• 15:38
  SPEAKER_00: Standards bodies should draft clear guidelines around data licensing and attribution for AI code.
• 15:45
  SPEAKER_00: Tooling and interfaces.
• 15:46
  SPEAKER_00: Final, build tool prototypes that demonstrate best practices.
• 15:50
  SPEAKER_00: An example, an AI coding IDE plugin that automatically runs static analysis or tests on any AI-generated code and warns the user.
• 16:00
  SPEAKER_00: Or a CLI that labels all AI-assisted sections in a code base.
• 16:04
  SPEAKER_00: Encourage open source projects to adopt AI-used badges or commit message conventions.
• 16:10
  SPEAKER_00: These informal standards can later be formalized.
• 16:13
  SPEAKER_00: By defining community benchmarks and holding multi-institution challenges, like an AI coding hackathon to meet certain security or maintainability targets, we can track progress.
• 16:24
  SPEAKER_00: Think of it like how ImageNet drove vision.
• 16:27
  SPEAKER_00: We need a shared ImageNet for code that reflects real development.
• 16:31
  SPEAKER_00: Early efforts, RAID map bench, slop code bench, Sigma bench, point the way.
• 16:37
  SPEAKER_00: But next we should scale them and make them widely available.
• 16:40
  SPEAKER_00: Formal interfaces, specs, tests, and code.
• 16:44
  SPEAKER_00: A central opportunity is tighter integration of specifications and tests into the coding loop.
• 16:50
  SPEAKER_00: In traditional development, a spec describes what the code should do, and tests check it.
• 16:55
  SPEAKER_00: AI tools can help connect these.
• 16:58
  SPEAKER_00: For example, a promising practice is spec-driven generation.
• 17:02
  SPEAKER_00: Write a possibly informal specification first, then prompt the AI to code it.
• 17:07
  SPEAKER_00: Even better, one could co-develop the spec with the AI.
• 17:10
  SPEAKER_00: For instance, ask the assistant, generate unit tests for this requirement.
• 17:15
  SPEAKER_00: Then use those tests to validate the code.
• 17:17
  SPEAKER_00: This creates a formal interface, the natural language spec, the tests it implies, and the code form a tight triangle.
• 17:24
  SPEAKER_00: On the research side, one could define a standard format for specs, e.g.
• 17:29
  SPEAKER_00: a YAML or JSON schema describing functionality, and require AI systems to consume it.
• 17:34
  SPEAKER_00: Efforts like TLA Plus, Alloy, or BDD style tools, Cucumber, might be integrated.
• 17:41
  SPEAKER_00: Imagine telling the AI, please generate code that satisfies this TLA Plus model.
• 17:46
  SPEAKER_00: Although LLMs today are not great at writing TLA Plus from scratch, combining a human-written abstract spec with AI-augmented code generation is worth exploring.
• 17:58
  SPEAKER_00: The goal is to make it easy for teams to produce a runnable specification, even if informal, that the AI respects.
• 18:06
  SPEAKER_00: Formal tests could then be auto-generated.
• 18:09
  SPEAKER_00: Recent work shows GPT models can produce property-based tests given a description of function behavior.
• 18:16
  SPEAKER_00: More ambitiously, we can create formal specification templates.
• 18:21
  SPEAKER_00: For cloud deployments or security-critical code, define a template, e.g., user authentication flow with fields.
• 18:30
  SPEAKER_00: The AI fills in the template and generates code.
• 18:33
  SPEAKER_00: A validator checks the contract.
• 18:35
  SPEAKER_00: By providing these interfaces, we transform coding from a black box into a more controlled pipeline.
• 18:42
  SPEAKER_00: Initiatives like the AI tools for TLA Plus or LLM to spec translation, ongoing in some research groups, are early examples.
• 18:51
  SPEAKER_00: In practice, even parcel adoption, asking AI to output comments or type signatures, can improve correctness.
• 18:58
  SPEAKER_00: As a first step for developers, incorporate simple spec test loops now.
• 19:03
  SPEAKER_00: For instance, if using Chat GPT, start your session by writing, we want a function that does X, write tests first.
• 19:11
  SPEAKER_00: Then ask it to generate the implementation.
• 19:13
  SPEAKER_00: Even without fancy formal tools, this enforces a discipline where AI always produces code with an accompanying check.
• 19:21
  SPEAKER_00: Over time, this habit can become formalized into standards for AI coding.
• 19:26
  SPEAKER_00: Collaboration, academia, industry, and standards.
• 19:30
  SPEAKER_00: Achieving these goals requires broad collaboration.
• 19:33
  SPEAKER_00: Academia can contribute by creating and sharing data and benchmarks, and by publishing rigorous evaluations.
• 19:40
  SPEAKER_00: Universities should partner with companies to get real code bases for testing.
• 19:44
  SPEAKER_00: Research labs can hold open challenges with prizes on tasks like long-term code quality or verified code generation.
• 19:52
  SPEAKER_00: Industry must provide feedback loops.
• 19:55
  SPEAKER_00: Firms deploying AI coding tools should anonymously share bug statistics, contributor experiences, and feature requests.
• 20:03
  SPEAKER_00: Tech companies can also fund AI for coding workshops or tracks at conferences, like ICSE, FSE.
• 20:10
  SPEAKER_00: They can open source parts of their policies, as Google did with Chromium's AI policy so others can learn.
• 20:17
  SPEAKER_00: Standards bodies, IEEE, ISO, W3C, etc., should incorporate coding into existing AI ethics and safety standards.
• 20:26
  SPEAKER_00: For example, ISO's ongoing work on AI governance, ISO IEC 38507, and AI lifecycle, ISO IEC 5338, could explicitly call out code generation.
• 20:38
  SPEAKER_00: The W3C has an ethical principles draft for WebML.
• 20:42
  SPEAKER_00: This could be extended with a section on programming use.
• 20:45
  SPEAKER_00: A lightweight code of practice for AI-reliant development teams should emerge, much like secure development standards, e.g.
• 20:52
  SPEAKER_00: OWASP, exist for security.
• 20:54
  SPEAKER_00: In short, the path forward is socio-technical.
• 20:58
  SPEAKER_00: Just as open source communities formed coding standards and review cultures, the emerging field of AI coding needs shared norms.
• 21:07
  SPEAKER_00: Joint roadmaps, e.g., Industry Consortia on AI code safety and transparency, publishing benchmarks and failure cases, will bring everyone onto the same page.
• 21:18
  SPEAKER_00: Who benefits and how to get started?
• 21:21
  SPEAKER_00: Crucially, AI-assisted coding is not just for expert developers.
• 21:25
  SPEAKER_00: These tools can democratize programming.
• 21:28
  SPEAKER_00: Beginners and subject matter experts can use AI to jumpstart projects they never have time to code by hand.
• 21:35
  SPEAKER_00: For example, a marketing analyst could ask an AI to write a data reporting script instead of learning Python from scratch.
• 21:42
  SPEAKER_00: An artist could prototype an app UI by sketching a prompt.
• 21:46
  SPEAKER_00: In each case, the AI lowers the barrier to creation.
• 21:50
  SPEAKER_00: To get started with these tools, follow the same agile iterative workflow that professional teams use.
• 21:56
  SPEAKER_00: Define a clear goal or specification.
• 21:58
  SPEAKER_00: Begin by What you want in concrete terms.
• 22:02
  SPEAKER_00: This could be a natural language description of a feature or a simple sketch of steps.
• 22:07
  SPEAKER_00: For programmers, even a list of bullet points or user stories can serve.
• 22:11
  SPEAKER_00: Use an AI assistant to draft code.
• 22:14
  SPEAKER_00: Run an AI coding tool.
• 22:16
  SPEAKER_00: Many are available, online chatbots or IDE extensions, and ask it to implement the spec.
• 22:22
  SPEAKER_00: For example, you might type create a Python function that reads a CSV and plots the data points.
• 22:28
  SPEAKER_00: The AI will generate a first version.
• 22:30
  SPEAKER_00: Verify and refine.
• 22:32
  SPEAKER_00: Crucially, take the AI's output and test it.
• 22:34
  SPEAKER_00: If it's code, run it in your environment.
• 22:37
  SPEAKER_00: Write or auto-generate some simple tests.
• 22:39
  SPEAKER_00: Does it give correct results on basic cases?
• 22:42
  SPEAKER_00: If something fails, it often will on the first try, give feedback to the AI.
• 22:47
  SPEAKER_00: For example, highlight the failing case and ask it to fix the code.
• 22:51
  SPEAKER_00: Many tools allow iterative prompting or multi-turn editing.
• 22:54
  SPEAKER_00: Ask for explanations and documentation.
• 22:57
  SPEAKER_00: Use the AI to produce doc strings or comments after the fact.
• 23:01
  SPEAKER_00: This helps you, the new coder, understand what was done.
• 23:04
  SPEAKER_00: You can also ask the AI to point out potential issues or suggest improvements.
• 23:09
  SPEAKER_00: Gradually increase complexity.
• 23:11
  SPEAKER_00: Once simple scripts work, you can attempt a small project, e.g.
• 23:14
  SPEAKER_00: a to-do app, a data analysis pipeline.
• 23:17
  SPEAKER_00: Divide the project into pieces.
• 23:19
  SPEAKER_00: Ask the AI for each component, database schema, front-end, business logic, one at a time.
• 23:25
  SPEAKER_00: Treat it like pair programming, where the AI is your junior partner.
• 23:29
  SPEAKER_00: First next step, pick a beginner-friendly AI coding tool and try a small experiment.
• 23:34
  SPEAKER_00: For instance, use an interface like GPT-4 with code capabilities or a free extension in your code editor.
• 23:40
  SPEAKER_00: Give it a trivial task.
• 23:42
  SPEAKER_00: Sort a list, make a graph, hello world web page.
• 23:45
  SPEAKER_00: Then read the code.
• 23:47
  SPEAKER_00: Even without coding experience, look at the structure, run it, and note any errors.
• 23:52
  SPEAKER_00: Then repeat, refine your prompt, maybe add more detail or constraints, and regenerate.
• 23:59
  SPEAKER_00: Over time, you'll learn how to communicate effectively with the tool and how to guide it toward correct solutions.
• 24:05
  SPEAKER_00: New coders should keep in mind AI is a powerful assistant, not an oracle.
• 24:11
  SPEAKER_00: Always check its work and use it as a learning opportunity.
• 24:15
  SPEAKER_00: Write your own tests for the AI's code, run them, and ask follow-up questions until you're confident.
• 24:22
  SPEAKER_00: This check-then-trust habit is how everyone, novice or expert, should safely build with AI.
• 24:28
  SPEAKER_00: Conclusion.
• 24:29
  SPEAKER_00: The rise of autonomous coding tools is a watershed moment.
• 24:33
  SPEAKER_00: But to fully reap the benefits, we must confront the open problems unveiled by early deployments.
• 24:39
  SPEAKER_00: In reliability, we see that code assistants make more mistakes than humans.
• 24:44
  SPEAKER_00: So research must focus on error detection and robust generation.
• 24:48
  SPEAKER_00: In planning, we see agents falter on long multi-step projects.
• 24:53
  SPEAKER_00: So we need new architectures and benchmarks for complex workflows.
• 24:57
  SPEAKER_00: In verifiability, we recognize that we need formal spec and testing support built into the AI coding process itself.
• 25:06
  SPEAKER_00: And on governance, companies and regulators are scrambling to set rules so that AI code is transparent, safe, and accountable.
• 25:14
  SPEAKER_00: Over the next 18 months, progress in each of these areas will be essential.
• 25:19
  SPEAKER_00: By building rigorous benchmarks, from project planning challenges to inspection of AI-induced bugs, integrating formal methods into AI coding pipelines, and forging collaborations across disciplines, we can close the gap between flashy demos and real-world reliability.
• 25:36
  SPEAKER_00: The vision is clear: an AI coding ecosystem where even beginners can safely create software, and where the code that AI generates is as trustworthy as human crafted code.
• 25:47
  SPEAKER_00: Meeting this vision will require shaping both the technology and the practices around it.
• 25:53
  SPEAKER_00: With focused research and broad community effort, the next generation of AI tools can truly unlock coding for everyone, starting today.
• 26:02
  SPEAKER_00: All links to sources are available in the text version of this article.
• 26:06
  SPEAKER_00: You can find the full article at easycoding.tools.blog.
• 26:11
  SPEAKER_00: Thanks for listening.
• 26:12
  SPEAKER_00: For more practical AI coding guides, tool comparisons, and resources for builders, visit easycoding.tools.
• 26:19
  SPEAKER_00: And if it's easier to remember, just go to aibuilds.it.
• 26:23
  SPEAKER_00: It redirects to easycoding.tools.