Latest / USA Daily News / NYC Council Sounds the Alarm on AI — The Risks Could Be Bigger Than We Think

Transcript

NYC Council Sounds the Alarm on AI — The Risks Could Be Bigger Than We Think · USA Daily News

• 0:00
  Speaker 1: Artificial intelligence has moved from being a specialized technology discussed
• 0:05
  Speaker 1: mainly by computer scientists and technology companies into a force
• 0:09
  Speaker 1: that is increasingly embedded in everyday life. In New York City,
• 0:14
  Speaker 1: AI is already connected to the way people communicate, work, study,
• 0:19
  Speaker 1: create content, search for information, and interact with businesses and
• 0:24
  Speaker 1: public institutions. That growing presence has also created a new
• 0:28
  Speaker 1: political question. What happens when systems designed to make decisions,
• 0:34
  Speaker 1: generate information, or act with increasing independence begin producing consequences
• 0:39
  Speaker 1: that humans cannot easily predict or control? That question was
• 0:44
  Speaker 1: at the center of a major New York City council
• 0:46
  Speaker 1: hearing on the risks of artificial intelligence. The council brought
• 0:51
  Speaker 1: together lawmakers, technology companies, researchers, former industry employees, and other
• 0:57
  Speaker 1: voices to examine whether existing safeguards are sufficient for a
• 1:01
  Speaker 1: technology developing at extraordinary speed. The hearing was unusual not
• 1:06
  Speaker 1: simply because of its subject. but because it placed the
• 1:09
  Speaker 1: company's building some of the world's most advanced AI systems
• 1:12
  Speaker 1: directly in front of local elected officials who wanted answers
• 1:16
  Speaker 1: about safety, accountability, and government oversight. The timing of the
• 1:21
  Speaker 1: hearing reflected a broader change in the public debate surrounding
• 1:25
  Speaker 1: artificial intelligence. Just a few years ago, much of the
• 1:29
  Speaker 1: discussion focused on the promise of AI. Generative systems could
• 1:33
  Speaker 1: write text, create images, summarize information, assist programmers, and help
• 1:38
  Speaker 1: businesses automate routine tasks. Supporters argued that these tools could
• 1:43
  Speaker 1: improve productivity, accelerate scientific research, and open new opportunities for
• 1:49
  Speaker 1: workers and entrepreneurs.
• 1:51
  Speaker 2: But as AI systems have become.
• 1:53
  Speaker 1: More capable, concerns have expanded alongside their benefits. Policymakers are
• 1:59
  Speaker 1: now confronting risks that range from the immediate and familiar
• 2:03
  Speaker 1: to the highly uncertain. Some involve misinformation and manipulated media.
• 2:09
  Speaker 1: Others concern privacy, discrimination, cybersecurity, fraud, employment, and the use
• 2:16
  Speaker 1: of AI in sensitive decisions. More advanced systems raise an
• 2:20
  Speaker 1: additional question about how much autonomy should be given to
• 2:23
  Speaker 1: software that can reason through complicated tasks. interact with digital environments,
• 2:28
  Speaker 1: and potentially make decisions with less direct human supervision. For
• 2:33
  Speaker 1: New York City officials, these issues are not abstract technological debates.
• 2:38
  Speaker 1: A city of millions of residents has a vast network
• 2:41
  Speaker 1: of schools, hospitals, businesses, government agencies, financial institutions, and public
• 2:47
  Speaker 1: services that can potentially be affected by AI. A problem
• 2:52
  Speaker 1: involving an AI system may therefore move quickly from the
• 2:55
  Speaker 1: digital world into people's daily lives. An inaccurate automated decision,
• 3:00
  Speaker 1: a convincing defake, A privacy failure or a vulnerable computer
• 3:04
  Speaker 1: system could have consequences for individuals as well as institutions.
• 3:09
  Speaker 1: The council's decision to devote a full hearing to the
• 3:12
  Speaker 1: subject showed that lawmakers increasingly view AI safety as a
• 3:16
  Speaker 1: matter of public policy rather than something that should be
• 3:19
  Speaker 1: left entirely to the technology sector. The central concern was
• 3:24
  Speaker 1: not simply whether AI is useful.
• 3:26
  Speaker 2: It was whether the.
• 3:27
  Speaker 1: Institutions developing and deploying increasingly powerful systems have adequate incentives, procedures,
• 3:34
  Speaker 1: and oversight to prevent serious harm. That concern has intensified
• 3:39
  Speaker 1: as competition among AI companies has accelerated. Technology firms are
• 3:44
  Speaker 1: racing to develop systems that are more capable, faster, and
• 3:48
  Speaker 1: more broadly useful. At the same time, Companies are competing
• 3:53
  Speaker 1: for enormous investments, customers, and strategic influence. The competitive environment
• 3:59
  Speaker 1: has created questions about whether commercial pressure can conflict with caution.
• 4:04
  Speaker 1: For policymakers, that creates a difficult balancing act. Artificial intelligence
• 4:10
  Speaker 1: could become an important source of economic growth, scientific progress,
• 4:14
  Speaker 1: and new employment. New York also has an interest in
• 4:18
  Speaker 1: remaining an important center for technology, finance, research, and innovation. Officials, therefore,
• 4:25
  Speaker 1: have to consider not only what could go wrong, but
• 4:28
  Speaker 1: also whether poorly designed regulation could discourage investment or push
• 4:32
  Speaker 1: technological development elsewhere. The hearing emerged against this background of uncertainty.
• 4:38
  Speaker 1: City lawmakers wanted to understand how AI companies evaluate the
• 4:42
  Speaker 1: risks associated with their systems and how prepared those companies
• 4:46
  Speaker 1: are to respond when something goes wrong. They also wanted
• 4:49
  Speaker 1: to explore whether government should establish minimum requirements rather than
• 4:53
  Speaker 1: relying primarily on voluntary commitments from the industry. That question
• 4:58
  Speaker 1: is particularly important because AI development does not stop at
• 5:02
  Speaker 1: the boundaries of a city. A model created by a
• 5:05
  Speaker 1: company in one location can be accessed by millions of
• 5:09
  Speaker 1: people elsewhere almost instantly. A decision made in New York
• 5:13
  Speaker 1: about AI regulation can therefore affect businesses that operate nationally
• 5:18
  Speaker 1: or globally. This creates both an opportunity and a challenge
• 5:22
  Speaker 1: for city government. The questions raised by the New York
• 5:26
  Speaker 1: City Council hearing became part of a much larger debate
• 5:29
  Speaker 1: over the pace of artificial intelligence development. At the center
• 5:33
  Speaker 1: of that debate is a concern that has become increasingly
• 5:36
  Speaker 1: difficult for policymakers, researchers, and even technology executives to ignore.
• 5:42
  Speaker 1: Artificial intelligence may be advancing faster than governments. Institutions and
• 5:47
  Speaker 1: society can adapt to its consequences. The issue is not
• 5:51
  Speaker 1: simply that AI systems are becoming more capable. Technology has
• 5:56
  Speaker 1: always progressed through successive generations, with newer systems generally outperforming
• 6:02
  Speaker 1: older ones. What makes artificial intelligence different is the speed
• 6:06
  Speaker 1: at which capabilities can be introduced, distributed, and scaled. A
• 6:11
  Speaker 1: new AI model can move from a research environment into
• 6:14
  Speaker 1: the hands of millions of users through an online service,
• 6:17
  Speaker 1: allowing its capabilities to influence businesses, schools, media organizations, and
• 6:23
  Speaker 1: households almost immediately. That speed creates a problem for traditional
• 6:27
  Speaker 1: approaches to regulation. Laws often take months or years to develop, debate,
• 6:33
  Speaker 1: and implement. By the time policymakers agree on a framework,
• 6:37
  Speaker 1: the technology they are attempting to regulate may already have
• 6:40
  Speaker 1: changed substantially. A system that seemed advanced when legislation was
• 6:45
  Speaker 1: drafted can quickly become outdated as companies release more powerful
• 6:49
  Speaker 1: models and new applications. For critics of the industry's current direction,
• 6:54
  Speaker 1: that cycle raises concerns about whether commercial competition is pushing
• 6:58
  Speaker 1: developers to release systems before every major risk is understood.
• 7:03
  Speaker 1: AI companies are competing aggressively over customers, investment, computing resources,
• 7:09
  Speaker 1: and technical talent. Each company has strong incentives to demonstrate
• 7:14
  Speaker 1: that its systems are more capable.
• 7:15
  Speaker 2: Than its competitors.
• 7:17
  Speaker 1: That competitive pressure can create a difficult environment in which
• 7:20
  Speaker 1: caution and speed do not always point in the same direction.
• 7:25
  Speaker 1: Supporters of rapid development reject the idea that innovation necessarily
• 7:29
  Speaker 1: means ignoring safety. Many companies have invested heavily in testing, monitoring,
• 7:35
  Speaker 1: and safeguards, arguing that AI can be developed responsibly while
• 7:39
  Speaker 1: still moving quickly. They point to improvements in reliability, system controls,
• 7:45
  Speaker 1: and misuse detection as evidence that safety measures can evolve
• 7:49
  Speaker 1: alongside capabilities. Yet the debate persists because the risks themselves
• 7:54
  Speaker 1: are changing. Some dangers are already visible. AI can generate
• 7:59
  Speaker 1: misleading information, imitate people's voices and images, automate fraudulent messages,
• 8:05
  Speaker 1: and produce convincing material that can be difficult to distinguish
• 8:09
  Speaker 1: from authentic content. These applications can be exploited for scams, manipulation,
• 8:15
  Speaker 1: or harassment without requiring the development of a hypothetical future superintelligence.
• 8:21
  Speaker 1: Other concerns are more structural. Businesses are increasingly experimenting with
• 8:26
  Speaker 1: AI to perform tasks that once required human judgment. Customer service, recruitment,
• 8:32
  Speaker 1: financial analysis, software development, marketing, and administrative work can.
• 8:38
  Speaker 2: All involve automated systems.
• 8:40
  Speaker 1: While such tools can improve efficiency, they also introduce questions
• 8:45
  Speaker 1: about what happens when an AI system produces a biased recommendation,
• 8:49
  Speaker 1: makes a serious error, or gives an answer that appears
• 8:52
  Speaker 1: confident but is wrong. The pace of development also creates
• 8:56
  Speaker 1: uncertainty about employment. Economists and business leaders disagree about the
• 9:01
  Speaker 1: scale and timing of AI-related job displacement. Some expect AI
• 9:07
  Speaker 1: to replace certain repetitive tasks while creating new occupations and
• 9:11
  Speaker 1: increasing productivity. Others warn that automation could affect large numbers
• 9:16
  Speaker 1: of workers before education systems and labor markets have time
• 9:20
  Speaker 1: to adjust. The uncertainty itself becomes a policy challenge because
• 9:24
  Speaker 1: governments must prepare for economic disruptions without knowing exactly how
• 9:29
  Speaker 1: quickly they will materialize. Privacy presents another area of concern.
• 9:34
  Speaker 1: AI systems can process enormous amounts of information, and the
• 9:38
  Speaker 1: expansion of AI-powered tools has increased attention on how personal.
• 9:43
  Speaker 2: Data is collected, stored, and used.
• 9:46
  Speaker 1: Facial recognition, voice analysis, Personalized systems and automated decision-making can
• 9:52
  Speaker 1: all raise questions about consent and surveillance. In a city
• 9:57
  Speaker 1: as large and technologically connected as New York, those questions
• 10:01
  Speaker 1: become especially significant because public and private institutions interact with
• 10:06
  Speaker 1: millions of residents every day. Cybersecurity has also become part
• 10:11
  Speaker 1: of the argument about rapid AI advancement. The same technology
• 10:15
  Speaker 1: that can help defenders identify suspicious activity can potentially make
• 10:20
  Speaker 1: cyberattacks more efficient. AI may allow attackers to automate parts
• 10:24
  Speaker 1: of phishing campaigns, generate deceptive communications, or adapt malicious activity
• 10:29
  Speaker 1: faster than traditional defenses can respond. At the same time,
• 10:34
  Speaker 1: cybersecurity professionals see AI as an important tool for identifying
• 10:39
  Speaker 1: threats and responding to them. As the New York City
• 10:42
  Speaker 1: Council hearing moved beyond broad warnings about artificial intelligence, lawmakers
• 10:48
  Speaker 1: turned their attention to the organization's building and deploying the technology.
• 10:54
  Speaker 1: Representatives from major AI companies were placed under public scrutiny
• 10:58
  Speaker 1: as council members sought answers to a basic but increasingly
• 11:02
  Speaker 1: consequential question. What exactly are these companies doing to make
• 11:06
  Speaker 1: sure increasingly powerful AI.
• 11:08
  Speaker 2: Systems remain safe?
• 11:10
  Speaker 1: The focus on corporate responsibility reflected the growing realization that
• 11:15
  Speaker 1: AI risks cannot be discussed solely in theoretical terms. The
• 11:19
  Speaker 1: companies developing advanced models are the ones deciding how those
• 11:23
  Speaker 1: systems are tested, what safeguards are placed around them, how
• 11:27
  Speaker 1: much autonomy they are given, and what happens when they
• 11:30
  Speaker 1: behave in unexpected ways. For lawmakers, understanding those processes was
• 11:35
  Speaker 1: therefore essential before deciding whether existing rules are adequate or
• 11:39
  Speaker 1: whether New York City needs new protections. Representatives from OpenAI, Anthropic, Google,
• 11:46
  Speaker 1: and Meta appeared before the council, creating a rare public
• 11:50
  Speaker 1: setting in which some of the largest technology companies were
• 11:53
  Speaker 1: questioned together about the dangers associated with AI. The companies
• 11:58
  Speaker 1: are major players in a rapidly expanding industry, but they
• 12:01
  Speaker 1: approach artificial intelligence from somewhat different business and technical positions.
• 12:07
  Speaker 1: Some develop highly advanced foundation models, while others integrate AI
• 12:12
  Speaker 1: into broader consumer platforms and digital services. Despite those differences,
• 12:17
  Speaker 1: they face a common challenge, demonstrating that their systems can
• 12:21
  Speaker 1: become more capable without creating unacceptable levels of risk. Lawmakers
• 12:26
  Speaker 1: wanted more than general statements about responsible innovation. They pressed
• 12:31
  Speaker 1: company representatives for explanations of how AI systems are evaluated
• 12:36
  Speaker 1: before release and how risks are monitored after deployment. This
• 12:40
  Speaker 1: includes questions about safety testing, internal review procedures, red team exercises,
• 12:46
  Speaker 1: human oversight, and the processes used to respond when a
• 12:49
  Speaker 1: model produces harmful or unexpected behavior. One of the most
• 12:54
  Speaker 1: difficult questions involved measuring risk itself. Policymakers can regulate a
• 12:59
  Speaker 1: technology more effectively when companies can provide clear information about
• 13:04
  Speaker 1: the probability and potential scale of dangerous outcomes. But AI
• 13:09
  Speaker 1: systems are complex, and the risks associated with future capabilities
• 13:14
  Speaker 1: are often difficult to quantify. That uncertainty became particularly important
• 13:19
  Speaker 1: as council members questioned representatives about extreme scenarios involving advanced AI.
• 13:25
  Speaker 1: The concern was not simply whether an AI chatbot might
• 13:28
  Speaker 1: give an incorrect answer. The more serious question was what
• 13:31
  Speaker 1: could happen if future systems became capable of carrying out
• 13:35
  Speaker 1: complicated tasks with relatively little human intervention and were able
• 13:40
  Speaker 1: to circumvent controls intended to restrict their behavior. For lawmakers,
• 13:45
  Speaker 1: a safety system is only meaningful if it continues to
• 13:48
  Speaker 1: work when circumstances change. A safeguard designed for today's AI
• 13:53
  Speaker 1: model may not necessarily be sufficient for a more capable
• 13:56
  Speaker 1: version released months later. This creates a moving target for
• 14:00
  Speaker 1: both companies and regulators. The issue of human oversight was
• 14:04
  Speaker 1: therefore another major component of the discussion. AI companies routinely
• 14:09
  Speaker 1: emphasize that people remain responsible for important decisions, but the
• 14:13
  Speaker 1: practical meaning of human supervision can vary. An individual reviewing
• 14:18
  Speaker 1: hundreds or thousands of AI-generated decisions may not have the
• 14:22
  Speaker 1: time or expertise to examine every output carefully. In other settings,
• 14:27
  Speaker 1: an AI agent may be given enough autonomy to complete
• 14:30
  Speaker 1: multiple stages of a task before a human becomes directly involved.
• 14:35
  Speaker 1: That distinction matters because the consequences of an error can
• 14:38
  Speaker 1: increase with autonomy. A system that produces an inaccurate paragraph
• 14:43
  Speaker 1: can usually be corrected.
• 14:45
  Speaker 2: By a user.
• 14:46
  Speaker 1: A system that interacts with computer networks, manages financial activity,
• 14:51
  Speaker 1: communicates with other software, or performs actions on behalf of
• 14:55
  Speaker 1: a company could potentially create much larger problems before a
• 14:59
  Speaker 1: person notices what is happening. The Council was also interested
• 15:03
  Speaker 1: in how companies respond when safety testing reveals a weakness.
• 15:07
  Speaker 1: Testing can identify vulnerabilities, but a safety program ultimately depends
• 15:12
  Speaker 1: on what happens afterward. Do companies delay a product launch,
• 15:17
  Speaker 1: reduce a system's capabilities, strengthen restrictions, or completely withdraw a
• 15:21
  Speaker 1: tool when serious problems are discovered? These questions go directly
• 15:26
  Speaker 1: to the credibility of voluntary corporate safety policies. Transparency was
• 15:31
  Speaker 1: another major point of tension. Companies possess detailed information about
• 15:36
  Speaker 1: their models that is often unavailable to government officials and
• 15:40
  Speaker 1: the general public. They know how systems were trained. what
• 15:44
  Speaker 1: evaluations were performed, and what internal incidents occurred, while outside
• 15:49
  Speaker 1: observers may see only the final product. That information imbalance
• 15:54
  Speaker 1: creates a challenge for regulators. The New York City Council
• 15:58
  Speaker 1: hearing took on a more confrontational tone as attention shifted
• 16:01
  Speaker 1: from the companies currently developing artificial intelligence to people who
• 16:05
  Speaker 1: had worked inside the industry and later raised concerns about
• 16:09
  Speaker 1: how those systems are being built. Former Employees Researchers and
• 16:14
  Speaker 1: other industry voices have increasingly become part of the public
• 16:17
  Speaker 1: debate because they can offer a perspective that is difficult
• 16:21
  Speaker 1: to obtain from official corporate statements. They have seen internal
• 16:25
  Speaker 1: testing procedures, development priorities, and workplace discussions firsthand, and some
• 16:31
  Speaker 1: argue that the public deserves to know more about the
• 16:33
  Speaker 1: risks they believe remain unresolved. The appearance of former industry
• 16:38
  Speaker 1: workers reflects a broader transformation in the artificial intelligence debate.
• 16:43
  Speaker 1: In the early stages of the current AI boom, much
• 16:46
  Speaker 1: of the public conversation was dominated by company executives and
• 16:50
  Speaker 1: enthusiastic researchers describing the possibilities of increasingly capable systems. As
• 16:57
  Speaker 1: concerns about safety have grown, however, Former employees have begun
• 17:01
  Speaker 1: challenging the idea that technological progress and adequate safeguards are
• 17:06
  Speaker 1: naturally moving forward together. Their warnings are varied, but several
• 17:11
  Speaker 1: themes have appeared repeatedly. Some former workers have questioned whether
• 17:15
  Speaker 1: companies are moving too quickly to release new products. Others
• 17:19
  Speaker 1: have raised concerns about the independence of internal safety teams,
• 17:24
  Speaker 1: the pressure created by intense competition, and the difficulty employees
• 17:28
  Speaker 1: may face, when trying to challenge decisions made by senior management.
• 17:33
  Speaker 1: Still others have argued that corporate safety commitments can appear
• 17:37
  Speaker 1: stronger publicly than they are in practice. These allegations are
• 17:41
  Speaker 1: significant because the AI industry operates under enormous commercial pressure.
• 17:46
  Speaker 1: Developing advanced systems requires substantial computing power, specialized talent, and
• 17:52
  Speaker 1: vast amounts of investment. Companies are competing for customers and
• 17:57
  Speaker 1: attempting to establish themselves as leaders in a market that
• 18:00
  Speaker 1: could influence nearly every major sector of the economy. In
• 18:05
  Speaker 1: such an environment, a decision to delay a product can
• 18:08
  Speaker 1: carry real financial consequences. For critics, that creates the possibility
• 18:14
  Speaker 1: of a structural conflict. The same organizations responsible for identifying
• 18:19
  Speaker 1: serious risks are also responsible for delivering products satisfying investors,
• 18:24
  Speaker 1: and keeping pace with competitors. Even when employees are acting
• 18:28
  Speaker 1: in good faith, the surrounding incentives may make it difficult
• 18:31
  Speaker 1: for safety concerns to receive the same priority as product performance,
• 18:36
  Speaker 1: market share, and release schedules. Former employees who speak publicly
• 18:41
  Speaker 1: can therefore become an important source of information for lawmakers.
• 18:45
  Speaker 1: They may describe situations that are not visible to consumers,
• 18:49
  Speaker 1: including disagreements over testing standards, Debates about how dangerous a
• 18:53
  Speaker 1: capability might be or disputes over whether a system should
• 18:57
  Speaker 1: be released at all. Their testimony can help policymakers understand
• 19:01
  Speaker 1: where internal safeguards may work and where institutional weaknesses could exist.
• 19:07
  Speaker 1: At the same time, whistleblower allegations can be complicated. Not
• 19:12
  Speaker 1: every criticism automatically proves that a company has acted improperly
• 19:17
  Speaker 1: and former employees can have personal disagreements. professional grievances, or
• 19:22
  Speaker 1: differing technical opinions. Lawmakers must therefore evaluate such testimony carefully,
• 19:28
  Speaker 1: separating verifiable safety concerns from disputes that may be more subjective.
• 19:34
  Speaker 1: That difficulty does not make the testimony irrelevant. In fact,
• 19:38
  Speaker 1: it highlights why independent oversight can become important. A system
• 19:43
  Speaker 1: of public accountability should not depend entirely on whether employees
• 19:47
  Speaker 1: are willing to risk their careers to speak out. If
• 19:50
  Speaker 1: safety information is critical to the public interest, policymakers may
• 19:55
  Speaker 1: want formal mechanisms that allow concerns to be reported without
• 19:58
  Speaker 1: requiring an employee to become a public whistleblower. That is
• 20:02
  Speaker 1: where the issue of whistleblower protection becomes especially important. Employees
• 20:07
  Speaker 1: working on advanced AI systems may discover problems before regulators, journalists,
• 20:13
  Speaker 1: or users do. They may have access to internal testing results,
• 20:17
  Speaker 1: incident reports, or discussions about potential weaknesses. But speaking out
• 20:22
  Speaker 1: can involve significant professional risks, particularly in a highly competitive
• 20:27
  Speaker 1: industry where specialized workers may depend on strong relationships and reputations.
• 20:32
  Speaker 1: The fear of retaliation can have a powerful effect on
• 20:36
  Speaker 1: organizational culture. Even when a company officially encourages employees to
• 20:40
  Speaker 1: report concerns, workers may hesitate if they believe their careers
• 20:45
  Speaker 1: could suffer as a result. A healthy safety culture, therefore,
• 20:49
  Speaker 1: requires more than formal policies. Employees must have confidence that
• 20:54
  Speaker 1: raising a legitimate concern will not automatically put them in
• 20:57
  Speaker 1: conflict with management. For policymakers, This raises a fundamental question
• 21:03
  Speaker 1: about how AI companies should be governed internally. As the
• 21:07
  Speaker 1: New York City Council examined the rapid development of artificial intelligence,
• 21:12
  Speaker 1: one of the most important parts of the hearing was
• 21:14
  Speaker 1: the effort to define exactly what lawmakers should be worried about.
• 21:18
  Speaker 1: Artificial intelligence is not a single technology with one predictable outcome.
• 21:24
  Speaker 1: It is a collection of systems being used for everything
• 21:27
  Speaker 1: from generating text and images to analyzing data, writing computer code,
• 21:32
  Speaker 1: and increasingly carrying out tasks with limited human intervention. That
• 21:37
  Speaker 1: means the potential risks exist across multiple levels, from problems
• 21:42
  Speaker 1: that are already affecting individuals to scenarios that remain uncertain,
• 21:46
  Speaker 1: but could become much more serious as AI capabilities advance.
• 21:51
  Speaker 1: some of the concerns discussed at the hearing were immediate
• 21:54
  Speaker 1: and familiar. Artificial intelligence can generate convincing false information, create
• 22:00
  Speaker 1: realistic images and audio, imitate people's identities, and produce content
• 22:05
  Speaker 1: at a scale that would have been difficult to achieve manually.
• 22:09
  Speaker 1: These capabilities can be exploited for scams, impersonation, political manipulation,
• 22:15
  Speaker 1: and harassment. A person trying to deceive a victim no
• 22:19
  Speaker 1: longer needs advanced technical knowledge to create a convincing message
• 22:23
  Speaker 1: or synthetic media when AI tools can automate much of
• 22:27
  Speaker 1: the process. The concern is especially serious because synthetic content
• 22:32
  Speaker 1: can spread faster than people can verify it. A fabricated
• 22:36
  Speaker 1: photograph or video may be viewed by thousands of people
• 22:39
  Speaker 1: before anyone has.
• 22:40
  Speaker 2: Established that it is false.
• 22:43
  Speaker 1: Even after the deception is exposed, the damage may not disappear.
• 22:47
  Speaker 1: People who saw the original material may continue to believe it,
• 22:51
  Speaker 1: while those who did not see the correction may never
• 22:54
  Speaker 1: become aware that the original claim was manipulated. For New
• 22:57
  Speaker 1: York City, this presents a significant public policy challenge. The
• 23:02
  Speaker 1: city is a major media center and a densely connected
• 23:06
  Speaker 1: digital environment, meaning information can move rapidly through social platforms,
• 23:11
  Speaker 1: messaging services, and news networks. Artificial intelligence can make the
• 23:16
  Speaker 1: production of misleading content cheaper and faster, potentially increasing the
• 23:21
  Speaker 1: volume of deceptive material that individuals and institutions must confront.
• 23:26
  Speaker 1: Fraud is another immediate concern. AI systems can help criminals
• 23:31
  Speaker 1: create highly personalized messages that appear to come from banks, employers, relatives,
• 23:36
  Speaker 1: or government agencies. Voice cloning can make a fraudulent phone
• 23:41
  Speaker 1: call sound familiar, while language generation tools can eliminate many
• 23:45
  Speaker 1: of the grammatical errors that traditionally helped people identify suspicious communications.
• 23:51
  Speaker 1: Criminal activity does not necessarily require an advanced autonomous machine.
• 23:56
  Speaker 1: In some cases, relatively ordinary AI capabilities can simply make
• 24:01
  Speaker 1: existing forms of crime more efficient. Privacy represents a different
• 24:06
  Speaker 1: category of risk. Artificial intelligence depends heavily on data, and
• 24:11
  Speaker 1: many modern systems are designed to process enormous quantities of information.
• 24:16
  Speaker 1: Questions about what information can be collected, how long it
• 24:19
  Speaker 1: can be stored, and who is allowed to use it
• 24:22
  Speaker 1: become increasingly important when.
• 24:24
  Speaker 2: AI is involved.
• 24:26
  Speaker 1: Residents may interact with AI without fully understanding what information
• 24:30
  Speaker 1: is being processed in the background. A company may use
• 24:34
  Speaker 1: AI to analyze customer communications, automate support services, or personalize recommendations.
• 24:41
  Speaker 1: Public institutions may consider AI for administrative tasks. The more
• 24:45
  Speaker 1: widely such systems are adopted, the more opportunities there are
• 24:49
  Speaker 1: for sensitive information to be exposed, misused, or retained in
• 24:54
  Speaker 1: ways users did not anticipate. The problem becomes more complicated
• 24:58
  Speaker 1: when AI systems are used to make or influence decisions
• 25:02
  Speaker 1: about people. Automated tools may assist employers in evaluating candidates,
• 25:07
  Speaker 1: businesses in assessing customers or institutions in allocating resources.
• 25:12
  Speaker 2: If the data used to develop a system contains.
• 25:15
  Speaker 1: Historical biases, an AI model can potentially reproduce or reinforce
• 25:20
  Speaker 1: those patterns. That creates concerns about discrimination and fairness. A
• 25:25
  Speaker 1: system can appear objective because it produces a numerical score
• 25:29
  Speaker 1: or automated recommendation, even though the underlying process may contain
• 25:34
  Speaker 1: biases that are difficult to detect. Individuals affected by such
• 25:38
  Speaker 1: decisions may not know that AI was involved, and even
• 25:41
  Speaker 1: when they do, they may not have access to an
• 25:44
  Speaker 1: understandable explanation of how the result was reached. For lawmakers,
• 25:49
  Speaker 1: transparency is therefore closely connected to fairness. People need to
• 25:54
  Speaker 1: know when an automated system is making an important decision
• 25:57
  Speaker 1: about them and what mechanisms exist to challenge an incorrect result.
• 26:02
  Speaker 1: Without those protections, AI could make certain decisions faster while
• 26:06
  Speaker 1: making them harder for individuals to contest. Employment disruption was
• 26:11
  Speaker 1: another major concern surrounding the technology. Artificial intelligence can increase
• 26:16
  Speaker 1: productivity by helping workers complete tasks more quickly and but
• 26:21
  Speaker 1: it can also automate responsibilities that previously required human labor.
• 26:26
  Speaker 1: With the potential dangers of artificial intelligence becoming clearer, the
• 26:30
  Speaker 1: New York City Council faced the next and more difficult question.
• 26:35
  Speaker 1: What should be done about them? Identifying risks is only
• 26:38
  Speaker 1: the first step in regulating a rapidly developing technology. The
• 26:43
  Speaker 1: real challenge is designing safeguards that are strong enough to
• 26:46
  Speaker 1: protect the public while still allowing useful AI systems to
• 26:50
  Speaker 1: develop and operate. That debate has led New York City
• 26:53
  Speaker 1: lawmakers toward several possible measures, including requirements for independent testing,
• 26:59
  Speaker 1: stronger transparency, additional accountability for companies, and a technical mechanism
• 27:05
  Speaker 1: that would allow a human operator to shut down an
• 27:07
  Speaker 1: AI system when necessary. Among these ideas, the proposed kill
• 27:12
  Speaker 1: switch has attracted particular attention because it raises a fundamental
• 27:16
  Speaker 1: question about the relationship between humans and increasingly capable machines.
• 27:22
  Speaker 1: Can people always maintain meaningful control over the systems they create?
• 27:26
  Speaker 1: The idea behind a shutdown capability is relatively straightforward. If
• 27:31
  Speaker 1: an artificial intelligence system begins behaving dangerously, violates established restrictions,
• 27:37
  Speaker 1: or produces an unacceptable level of risk, there should be
• 27:41
  Speaker 1: a way for an authorized person to stop its operation.
• 27:44
  Speaker 1: In principle, such a mechanism would provide a final layer
• 27:48
  Speaker 1: of protection when other safeguards fail. But the apparent simplicity
• 27:52
  Speaker 1: of the concept hides a much more complicated technical problem.
• 27:55
  Speaker 1: A shutdown system is useful only if it actually works
• 27:59
  Speaker 1: under real-world conditions. It must be accessible to authorized humans
• 28:04
  Speaker 1: protected against unauthorized interference, and designed so that it cannot
• 28:09
  Speaker 1: easily be bypassed. For highly integrated AI systems, shutting down
• 28:14
  Speaker 1: the model may also involve stopping connected applications, data pipelines,
• 28:19
  Speaker 1: automated processes, or other tools that depend on it. This
• 28:23
  Speaker 1: becomes especially important as AI moves beyond conventional chatbots. A
• 28:28
  Speaker 1: system that simply answers questions presents one category of risk.
• 28:33
  Speaker 1: An AI agent that can search the internet, operate software,
• 28:37
  Speaker 1: send communications, write and execute code or perform multiple tasks
• 28:42
  Speaker 1: on behalf of a user represents a different level of
• 28:45
  Speaker 1: potential exposure. The more actions a system can take, the
• 28:49
  Speaker 1: more important it becomes to establish boundaries around what it
• 28:52
  Speaker 1: can access and how a human can intervene. Supporters of
• 28:56
  Speaker 1: mandatory shutdown capabilities argue that no advanced technology should be
• 29:01
  Speaker 1: allowed to operate without a reliable means of human intervention.
• 29:05
  Speaker 1: They compare the idea to emergency controls found in other
• 29:09
  Speaker 1: high-risk environments where operators need a way to interrupt a
• 29:13
  Speaker 1: process when something goes wrong. Critics, however, can point to
• 29:17
  Speaker 1: a number of unanswered questions. Who would possess the authority
• 29:21
  Speaker 1: to activate the shutdown? What qualifies as an emergency? How
• 29:26
  Speaker 1: quickly would the system need to stop? Would shutting down
• 29:29
  Speaker 1: one component actually stop the entire AI-powered operation? And what
• 29:33
  Speaker 1: would happen if an AI system were embedded in a
• 29:36
  Speaker 1: service that millions of people depend on? These questions demonstrate
• 29:40
  Speaker 1: why lawmakers cannot treat a kill switch as a simple
• 29:44
  Speaker 1: software feature. It would need to be incorporated into a
• 29:47
  Speaker 1: broader safety architecture that establishes clear responsibilities and procedures. A
• 29:53
  Speaker 1: shutdown mechanism could become ineffective if companies do not regularly
• 29:57
  Speaker 1: test it or if employees are uncertain about when they
• 30:00
  Speaker 1: are authorized to use it. The council's proposals also put
• 30:04
  Speaker 1: significant emphasis on third-party validation. Under this approach, companies would
• 30:09
  Speaker 1: not necessarily be allowed to determine entirely by themselves whether
• 30:13
  Speaker 1: an AI model is ready for deployment. Independent evaluators could
• 30:18
  Speaker 1: examine issues such as performance, safety, privacy, and the potential
• 30:22
  Speaker 1: for discriminatory outcomes before a system reaches the market. The
• 30:27
  Speaker 1: reasoning behind independent validation is based on a basic principle
• 30:31
  Speaker 1: of accountability. Companies developing AI have enormous technical expertise, but
• 30:37
  Speaker 1: they also have commercial incentives. They're competing to release more
• 30:41
  Speaker 1: capable products, attract customers, and maintain their positions in a
• 30:45
  Speaker 1: rapidly expanding industry. An independent evaluator would theoretically have fewer
• 30:50
  Speaker 1: incentives to prioritize a product launch over safety concerns. Independent
• 30:55
  Speaker 1: testing could also improve public confidence. Consumers are often asked
• 31:00
  Speaker 1: to trust AI systems without knowing how extensively they have
• 31:04
  Speaker 1: been evaluated. A meaningful validation process could create a clearer
• 31:08
  Speaker 1: distinction between a system that has merely been released by
• 31:11
  Speaker 1: its developer and one that has undergone a structured external assessment. Still,
• 31:17
  Speaker 1: independent validation would introduce its own difficulties. Evaluators would need
• 31:22
  Speaker 1: sufficient technical expertise to understand increasingly complex AI models. They
• 31:28
  Speaker 1: would also have to remain independent from the companies whose
• 31:31
  Speaker 1: systems they assess. The debate over artificial intelligence in New
• 31:35
  Speaker 1: York City is not limited to questions of safety and
• 31:38
  Speaker 1: government oversight. Behind nearly every regulatory proposal is another issue
• 31:43
  Speaker 1: with enormous consequences, money. Artificial intelligence is becoming one of
• 31:48
  Speaker 1: the most important areas of technological investment. And the companies
• 31:53
  Speaker 1: developing it are attracting billions of dollars in capital while
• 31:56
  Speaker 1: promising to transform industries ranging from finance and healthcare to media, manufacturing,
• 32:03
  Speaker 1: and professional services. For New York City, that creates a
• 32:07
  Speaker 1: difficult balancing act. The city wants to protect residents from
• 32:11
  Speaker 1: the risks associated with AI, but it also wants to
• 32:14
  Speaker 1: remain one of the world's most important centers for innovation, investment,
• 32:19
  Speaker 1: and economic growth. That tension is central to the argument
• 32:23
  Speaker 1: over regulation. Supporters of stronger rules generally do not dispute
• 32:28
  Speaker 1: that artificial intelligence could produce major economic benefits. The question
• 32:33
  Speaker 1: is whether those benefits justify allowing development to proceed with
• 32:37
  Speaker 1: relatively limited government intervention. Critics of aggressive regulation argue that
• 32:43
  Speaker 1: excessive requirements could raise costs. slow product development, and make
• 32:48
  Speaker 1: it more difficult for smaller companies to compete with larger
• 32:51
  Speaker 1: technology firms that already possess substantial financial and technical resources.
• 32:57
  Speaker 1: For New York, the stakes are particularly high because the
• 33:01
  Speaker 1: city occupies a unique position in the American economy. It
• 33:05
  Speaker 1: is a global center for finance, media, advertising, consulting, law, healthcare, education,
• 33:13
  Speaker 1: and technology. AI is increasingly moving into all of those sectors.
• 33:18
  Speaker 1: Banks are exploring automated analytical tools. Media companies are experimenting
• 33:23
  Speaker 1: with synthetic content. Professional firms are using AI assistance. Retailers
• 33:29
  Speaker 1: are applying predictive systems and startups are building new products
• 33:33
  Speaker 1: around generative models. The potential productivity gains are significant. A
• 33:38
  Speaker 1: worker who once spent hours organizing information may be able
• 33:42
  Speaker 1: to complete the same task in a fraction of the time.
• 33:45
  Speaker 1: Software developers can use AI to accelerate coding. Analysts can
• 33:50
  Speaker 1: process large amounts of information more quickly. Businesses can automate
• 33:54
  Speaker 1: routine customer interactions and administrative work. Researchers can use machine
• 34:00
  Speaker 1: learning to examine complex datasets that would otherwise take much
• 34:04
  Speaker 1: longer to analyze. From an economic perspective, these improvements could
• 34:09
  Speaker 1: increase output without requiring businesses to expand their workforces at
• 34:13
  Speaker 1: the same rate. That possibility is one reason investors and
• 34:17
  Speaker 1: corporate leaders are so interested in artificial intelligence. If AI
• 34:22
  Speaker 1: can substantially raise productivity, companies may become more efficient, new
• 34:27
  Speaker 1: businesses may emerge, and entirely new markets could develop. but
• 34:32
  Speaker 1: those same gains can create disruption. When a technology makes
• 34:36
  Speaker 1: certain tasks easier to automate, companies may need fewer workers
• 34:40
  Speaker 1: to perform those tasks. The impact may not always appear
• 34:44
  Speaker 1: as sudden mass unemployment. Instead, organizations could reduce hiring, restructure departments,
• 34:51
  Speaker 1: or expect existing employees to manage much larger workloads with
• 34:55
  Speaker 1: AI assistance. That creates a complicated picture for policymakers. A
• 35:00
  Speaker 1: company may consider an AI deployment a productivity improvement, while
• 35:05
  Speaker 1: the workers affected by it may see the same change
• 35:08
  Speaker 1: as a reduction in opportunities. Government officials must therefore consider
• 35:12
  Speaker 1: how economic gains are distributed, not simply how large those
• 35:16
  Speaker 1: gains might eventually become. The challenge is especially relevant in
• 35:21
  Speaker 1: a city where the cost of living and labor market
• 35:23
  Speaker 1: pressures are already significant. If AI expands productivity but concentrates
• 35:29
  Speaker 1: the financial benefits among a relatively small group of companies
• 35:33
  Speaker 1: and investors, questions about inequality could become more prominent. On
• 35:38
  Speaker 1: the other hand, if AI helps businesses grow, lowers operating costs,
• 35:43
  Speaker 1: and creates new industries, the broader economy could benefit in
• 35:47
  Speaker 1: ways that are difficult to predict today. This uncertainty is
• 35:51
  Speaker 1: one reason some policymakers favor flexible regulation rather than rigid rules.
• 35:57
  Speaker 1: They argue that governments should create basic safety requirements while
• 36:01
  Speaker 1: allowing the market to determine which applications succeed. Under this model,
• 36:06
  Speaker 1: regulators would focus on high-risk uses while avoiding unnecessary restrictions
• 36:11
  Speaker 1: on lower-risk systems that could generate substantial economic benefits. Other
• 36:17
  Speaker 1: policymakers take a more cautious view. They argue that the
• 36:21
  Speaker 1: costs of AI-related harm should be considered alongside its economic potential.
• 36:26
  Speaker 1: A company may save money by automating a decision-making process,
• 36:30
  Speaker 1: but if that system systematically discriminates against certain people, exposes
• 36:35
  Speaker 1: sensitive information, or creates widespread financial losses, the broader social
• 36:41
  Speaker 1: cost could exceed the initial economic benefit. This is where
• 36:45
  Speaker 1: regulation becomes a question of incentives. Businesses naturally have strong
• 36:50
  Speaker 1: reasons to invest in AI when they expect.
• 36:52
  Speaker 2: Higher profits or lower costs. The debate over.
• 36:56
  Speaker 1: Artificial intelligence regulation becomes considerably more complicated when the question
• 37:01
  Speaker 1: shifts from what should be regulated to who should do
• 37:04
  Speaker 1: the regulating. The New York City Council can debate rules
• 37:07
  Speaker 1: affecting technology used within the city, but the companies developing
• 37:12
  Speaker 1: advanced AI systems operate far beyond municipal boundaries. Their products
• 37:17
  Speaker 1: can be created in one state, trained on computing infrastructure
• 37:21
  Speaker 1: spread across several regions, deployed through global online platforms, and
• 37:26
  Speaker 1: accessed by users thousands of miles away. That reality raises
• 37:30
  Speaker 1: a fundamental problem for policymakers. Can a city effectively regulate
• 37:35
  Speaker 1: a technology whose reach is national and international? For New
• 37:39
  Speaker 1: York City lawmakers, local action has obvious advantages. City officials
• 37:44
  Speaker 1: are closest to the residents and institutions directly affected by
• 37:48
  Speaker 1: decisions made at the municipal level. They can respond to
• 37:52
  Speaker 1: local concerns, create rules for city agencies, set standards for contractors,
• 37:58
  Speaker 1: and address specific problems emerging in the communities they represent.
• 38:02
  Speaker 1: A local government can also move more quickly than a
• 38:05
  Speaker 1: national legislature when a new risk becomes urgent. That flexibility
• 38:10
  Speaker 1: can be valuable in the artificial intelligence debate. AI is
• 38:14
  Speaker 1: developing too quickly for policymakers to assume that a single
• 38:18
  Speaker 1: set of rules will remain appropriate for years. A city
• 38:21
  Speaker 1: can experiment with specific requirements, evaluate their results, and adjust
• 38:26
  Speaker 1: them as the technology changes. New York could potentially become
• 38:30
  Speaker 1: an important testing ground for policies designed to address high-risk
• 38:34
  Speaker 1: uses without imposing identical restrictions on every AI application. But
• 38:40
  Speaker 1: local regulation also has clear limitations. A company developing an
• 38:45
  Speaker 1: AI model does not necessarily design one version specifically.
• 38:49
  Speaker 2: For New York.
• 38:50
  Speaker 1: The same system may be available to users across the country.
• 38:54
  Speaker 1: If New York establishes requirements that differ significantly from those
• 38:58
  Speaker 1: in other jurisdictions, technology companies could face multiple compliance regimes
• 39:03
  Speaker 1: for essentially the same product. That creates the possibility of
• 39:08
  Speaker 1: a fragmented regulatory landscape. One state might require extensive testing
• 39:13
  Speaker 1: before a powerful AI system is deployed. Another could rely
• 39:17
  Speaker 1: more heavily on voluntary standards. A third could focus mainly
• 39:22
  Speaker 1: on disclosure. Cities could then add their own rules on
• 39:25
  Speaker 1: top of those state requirements. For large technology companies such
• 39:30
  Speaker 1: a system may be manageable because they have the lawyers,
• 39:33
  Speaker 1: compliance teams, and technical specialists needed to navigate complex regulations.
• 39:39
  Speaker 1: Smaller companies could face a much greater burden. A startup
• 39:43
  Speaker 1: with limited resources may be forced to spend a substantial
• 39:46
  Speaker 1: share of its budget meeting different requirements in different places
• 39:50
  Speaker 1: rather than building its product. That could have an unintended
• 39:54
  Speaker 1: effect on competition. Regulations designed to improve safety could become
• 39:59
  Speaker 1: so complicated that they reinforce the dominance of the largest firms.
• 40:03
  Speaker 1: Established companies may be able to absorb the costs, while
• 40:07
  Speaker 1: smaller competitors struggle to enter the market. Policymakers, therefore, have
• 40:12
  Speaker 1: to consider not just whether rules are strong enough, but
• 40:15
  Speaker 1: whether they are designed in a way that preserves competition.
• 40:19
  Speaker 1: There is also a constitutional and economic dimension to the issue.
• 40:23
  Speaker 1: Artificial intelligence products can cross state lines instantly, making the
• 40:28
  Speaker 1: technology closely connected to interstate commerce. Questions, therefore, arise about
• 40:33
  Speaker 1: how much authority should belong to municipalities and states and
• 40:37
  Speaker 1: how much should be reserved for the federal government. Supporters
• 40:41
  Speaker 1: of national regulation argue that the United States needs a
• 40:44
  Speaker 1: coherent framework. A company should not have to determine entirely
• 40:49
  Speaker 1: different safety obligations depending on where its customers happen to
• 40:52
  Speaker 1: be located. A federal system could establish baseline requirements for testing, transparency, privacy, security,
• 41:01
  Speaker 1: and accountability while allowing states and cities to impose additional
• 41:05
  Speaker 1: protections in specific areas. A national framework could also make
• 41:09
  Speaker 1: enforcement more consistent. Federal regulators could potentially coordinate investigations, establish
• 41:16
  Speaker 1: technical standards, and maintain centralized systems for incident reporting. This
• 41:22
  Speaker 1: could be especially important for AI failures that affect people
• 41:26
  Speaker 1: across multiple jurisdictions at the same time. Imagine an AI-powered
• 41:31
  Speaker 1: service that contains a serious security vulnerability. If the vulnerability
• 41:36
  Speaker 1: affects customers in dozens of states, responding through separate local
• 41:40
  Speaker 1: regulators could become slow and fragmented. A coordinated national mechanism
• 41:45
  Speaker 1: could identify the problem more quickly and distribute information to
• 41:50
  Speaker 1: affected organizations. The same argument applies to emerging AI threats
• 41:55
  Speaker 1: that do not fit neatly within city or state boundaries. Cybersecurity,
• 41:59
  Speaker 1: synthetic media, financial manipulation, and advanced autonomous systems can operate
• 42:05
  Speaker 1: on a national or international scale. The New York City
• 42:09
  Speaker 1: Council hearing on artificial intelligence revealed that the relationship between
• 42:13
  Speaker 1: government and the technology industry is entering a new and potentially.
• 42:18
  Speaker 2: More demanding phase.
• 42:19
  Speaker 1: For years, much of the technology sector operated in an
• 42:23
  Speaker 1: environment where innovation often moved faster than formal regulations. Companies
• 42:28
  Speaker 1: introduced new digital products, expanded their platforms, and developed increasingly
• 42:33
  Speaker 1: sophisticated tools while governments frequently responded after problems became visible.
• 42:39
  Speaker 1: Artificial intelligence is now challenging that model because its capabilities
• 42:44
  Speaker 1: are expanding quickly and its influence is reaching areas that
• 42:47
  Speaker 1: have traditionally been governed by public institutions. The appearance of
• 42:52
  Speaker 1: major AI companies before elected officials, was therefore significant beyond
• 42:57
  Speaker 1: the immediate questions discussed at the hearing. It demonstrated that
• 43:01
  Speaker 1: artificial intelligence companies are increasingly being treated not merely as
• 43:06
  Speaker 1: technology providers, but as institutions whose decisions can have broad
• 43:10
  Speaker 1: social and economic consequences. When an AI system influences what
• 43:15
  Speaker 1: people see, how businesses operate, or how important decisions are made,
• 43:20
  Speaker 1: questions about accountability naturally move into the political arena. This
• 43:25
  Speaker 1: represents a substantial change in the industry's relationship with government.
• 43:30
  Speaker 1: Technology companies have traditionally emphasized their ability to innovate without
• 43:35
  Speaker 1: extensive bureaucratic interference. Their argument has often been that engineers
• 43:40
  Speaker 1: and researchers understand emerging technologies better than policymakers and that
• 43:45
  Speaker 1: excessive regulation can prevent society from benefiting from new inventions.
• 43:50
  Speaker 1: That argument becomes harder to maintain when AI systems begin
• 43:54
  Speaker 1: performing tasks that directly affect people's lives. The more deeply
• 43:59
  Speaker 1: AI enters employment, education, financial services, healthcare, public administration, and communications,
• 44:07
  Speaker 1: the more difficult it becomes to treat the technology as
• 44:11
  Speaker 1: simply another consumer product. The hearing showed that lawmakers are
• 44:15
  Speaker 1: increasingly unwilling to rely entirely on assurances from technology companies themselves.
• 44:22
  Speaker 1: They want evidence that safety procedures are meaningful, that risks
• 44:26
  Speaker 1: are being identified before systems are released, and that companies
• 44:30
  Speaker 1: are prepared to accept responsibility when something goes wrong. At
• 44:34
  Speaker 1: the heart of this change is the issue of trust.
• 44:37
  Speaker 1: Technology companies need users to trust their products. Government officials
• 44:42
  Speaker 1: need the public to trust that they are protecting residents.
• 44:46
  Speaker 1: But each side is increasingly asking the other to prove
• 44:49
  Speaker 1: that it deserves that trust. AI companies often argue that
• 44:53
  Speaker 1: they have strong internal safety processes and highly specialized teams
• 44:57
  Speaker 1: capable of evaluating their systems. From their perspective, the technology
• 45:02
  Speaker 1: is too complex to be governed effectively through simplistic rules.
• 45:07
  Speaker 1: They may believe that technical standards— Industry cooperation and voluntary
• 45:12
  Speaker 1: commitments can address many risks without requiring heavy government intervention. Lawmakers, however,
• 45:19
  Speaker 1: face a different responsibility. Their obligation is not to maximize
• 45:24
  Speaker 1: the speed of technological development. It is to protect the
• 45:27
  Speaker 1: public and establish rules that apply even when individual companies
• 45:31
  Speaker 1: make mistakes or face conflicting incentives. That difference in priorities
• 45:36
  Speaker 1: creates a natural tension.
• 45:38
  Speaker 2: A company might view.
• 45:39
  Speaker 1: A particular AI capability as an important breakthrough that should
• 45:43
  Speaker 1: reach customers quickly. A regulator might see the same capability
• 45:48
  Speaker 1: and ask whether sufficient testing has been completed. Neither position
• 45:52
  Speaker 1: necessarily reflects bad faith. The disagreement arises because the two
• 45:57
  Speaker 1: institutions are evaluating the same technology from fundamentally different perspectives.
• 46:03
  Speaker 1: The hearing also highlighted the growing importance of transparency. AI
• 46:08
  Speaker 1: systems can be difficult for outsiders to understand, and companies
• 46:12
  Speaker 1: often possess information about their models that is unavailable to regulators, researchers,
• 46:18
  Speaker 1: and the public. This creates an uneven relationship in which
• 46:21
  Speaker 1: the developers know far more about the technology than the
• 46:25
  Speaker 1: people responsible for overseeing it. That imbalance can become particularly
• 46:30
  Speaker 1: serious after an incident. If an AI system causes harm,
• 46:34
  Speaker 1: officials may need to know what testing took place, what
• 46:38
  Speaker 1: risks were identified beforehand, what warnings employees raised, and what
• 46:43
  Speaker 1: decisions were made before deployment. Without access to that information,
• 46:48
  Speaker 1: government oversight can become dependent on whatever the company chooses
• 46:52
  Speaker 1: to disclose. For that reason, future regulation may increasingly focus
• 46:58
  Speaker 1: on documentation and reporting rather than simply banning particular AI systems.
• 47:03
  Speaker 1: Governments could require companies to maintain records of risk assessments,
• 47:08
  Speaker 1: testing procedures, significant failures, and corrective actions. The goal would
• 47:13
  Speaker 1: be to create a trail of accountability that allows regulators
• 47:16
  Speaker 1: to understand not just the final result, but the process
• 47:20
  Speaker 1: that produced it. The New York City Council's hearing on
• 47:23
  Speaker 1: artificial intelligence risks was never likely to produce a final
• 47:28
  Speaker 1: answer in a single session. The technology is changing too quickly,
• 47:32
  Speaker 1: the potential consequences are too broad and the debate over
• 47:36
  Speaker 1: how much government should intervene remains.
• 47:39
  Speaker 2: Far from settled.
• 47:40
  Speaker 1: What the hearing did accomplish was to move the discussion
• 47:43
  Speaker 1: from general concern to a more concrete examination of what lawmakers, companies,
• 47:48
  Speaker 1: and the public might do next. The future of artificial
• 47:52
  Speaker 1: intelligence in New York will now depend heavily on whether
• 47:55
  Speaker 1: the concerns raised during the hearing lead to actual policy.
• 47:59
  Speaker 1: Public hearings can bring attention to an issue, but their
• 48:03
  Speaker 1: long-term significance.
• 48:04
  Speaker 2: Is measured by what follows.
• 48:07
  Speaker 1: For the city council, that means deciding which proposals deserve
• 48:11
  Speaker 1: legislative action, which risks require additional investigation, and which aspects
• 48:16
  Speaker 1: of AI regulation may need coordination.
• 48:19
  Speaker 2: With state or federal authorities.
• 48:22
  Speaker 1: One possible outcome is a stronger set of rules for
• 48:25
  Speaker 1: high-risk artificial intelligence systems. Instead of attempting to regulate every
• 48:30
  Speaker 1: AI application in exactly the same way, lawmakers could focus
• 48:35
  Speaker 1: their strongest requirements on systems capable of creating significant consequences
• 48:40
  Speaker 1: for people's lives. Such systems could face additional testing, documentation, transparency,
• 48:47
  Speaker 1: and human oversight requirements before being deployed. That approach could
• 48:52
  Speaker 1: become increasingly important as artificial intelligence moves into more sensitive areas.
• 48:58
  Speaker 1: An AI tool used to summarize routine documents presents very
• 49:02
  Speaker 1: different concerns from a system that evaluates employment candidates, assists
• 49:06
  Speaker 1: with financial decisions, or influences access to public services. The
• 49:11
  Speaker 1: future regulatory framework may therefore depend less on whether a
• 49:15
  Speaker 1: product is labeled AI and more on what that product
• 49:18
  Speaker 1: is capable of doing, and how much power it has
• 49:21
  Speaker 1: over real people. Another possibility is the creation of stronger
• 49:26
  Speaker 1: disclosure requirements. Governments may seek greater visibility into where AI
• 49:31
  Speaker 1: is being used, what systems are involved, and how organizations
• 49:35
  Speaker 1: respond when problems occur. The purpose would not necessarily be
• 49:39
  Speaker 1: to make every technical detail public, but to ensure that
• 49:43
  Speaker 1: residents and regulators are not left in the dark when
• 49:46
  Speaker 1: automated systems have important consequences. Transparency could become one of
• 49:51
  Speaker 1: the defining principles of AI governance. People may increasingly expect
• 49:56
  Speaker 1: to know when they are interacting with an automated system
• 49:59
  Speaker 1: when AI is being used to influence a decision and
• 50:02
  Speaker 1: who remains responsible for the result. that could become particularly
• 50:06
  Speaker 1: important when mistakes are difficult to reverse. The question of
• 50:10
  Speaker 1: accountability will therefore remain central. As AI systems become more sophisticated,
• 50:16
  Speaker 1: governments will have to establish who is responsible when something
• 50:20
  Speaker 1: goes wrong. Companies that develop models, businesses that deploy them,
• 50:24
  Speaker 1: and institutions that rely on their outputs may each have
• 50:28
  Speaker 1: different responsibilities. Without clear rules, disputes over responsibility could become
• 50:34
  Speaker 1: common after serious incidents. The Council's attention to human control
• 50:39
  Speaker 1: also points toward another possible development, stronger requirements for intervention mechanisms.
• 50:46
  Speaker 1: As AI systems become capable of performing increasingly complex tasks,
• 50:51
  Speaker 1: regulators may expect organizations to maintain reliable ways for humans
• 50:55
  Speaker 1: to interrupt and override or disable those systems when necessary.
• 51:00
  Speaker 1: But the implementation of such safeguards will be more complicated
• 51:04
  Speaker 1: than simply creating an emergency button. Regulators will need to
• 51:08
  Speaker 1: determine when a shutdown should occur, who is authorized to
• 51:11
  Speaker 1: initiate it, and how organizations can test the mechanism before
• 51:15
  Speaker 1: an emergency happens. The more interconnected an AI system becomes,
• 51:20
  Speaker 1: the more difficult it may be to guarantee that disabling
• 51:23
  Speaker 1: one component will immediately stop every related process. That means
• 51:28
  Speaker 1: future legislation may need to establish principles rather than highly
• 51:32
  Speaker 1: specific technical instructions. Technology evolves rapidly, and a rule designed
• 51:38
  Speaker 1: around one generation of AI could become obsolete as new
• 51:42
  Speaker 1: capabilities emerge. Policymakers may therefore seek standards that require effective
• 51:47
  Speaker 1: human control without prescribing exactly how every company must engineer
• 51:52
  Speaker 1: its systems. Independent testing is another area that could gain momentum.
• 51:58
  Speaker 1: A major question raised by the hearing is whether companies
• 52:01
  Speaker 1: should be able to determine entirely on their own whether
• 52:04
  Speaker 1: an advanced AI system is safe enough to deploy. Third-party
• 52:08
  Speaker 1: evaluations could provide another layer of accountability by examining systems
• 52:13
  Speaker 1: from outside the organization that created them. Such testing could
• 52:17
  Speaker 1: potentially become a routine part of the AI development process.
• 52:21
  Speaker 1: Before the release of particularly powerful systems, independent evaluators could
• 52:25
  Speaker 1: examine their performance, security, misuse potential, and other risks. After deployment,
• 52:32
  Speaker 1: additional assessments might be required when systems receive major upgrades,
• 52:37
  Speaker 1: or are given new capabilities.