Latest / ELECTE's Podcast: AI Frontiers / ChatGPT and the Reporting Dilemma: When AI Becomes an Informant

Transcript

ChatGPT and the Reporting Dilemma: When AI Becomes an Informant · ELECTE's Podcast: AI Frontiers

• 0:00
  SPEAKER_00: The Big Change.
• 0:01
  SPEAKER_00: OpenAI admits to reporting to authorities.
• 0:06
  SPEAKER_00: In September 2025, OpenAI made a revelation that shook the global tech community.
• 0:12
  SPEAKER_00: ChatGPT actively monitors user conversations and reports potentially criminal content to law enforcement.
• 0:18
  SPEAKER_00: The news which emerged almost casually in a company blog post revealed that when automated systems detect users who are planning to harm others, conversations are routed to specialized pipelines where a small team trained in usage policies reviews them.
• 0:32
  SPEAKER_00: If human reviewers determine that there is an imminent threat of serious physical harm to others, the case may be referred to law enforcement.
• 0:41
  SPEAKER_00: Sources, OpenAI says it's scanning users' chat GPT conversations and reporting content to the police.
• 0:50
  SPEAKER_00: Futurism chat GPT privacy questioned as OpenAI confirms police access in extreme cases.
• 0:56
  SPEAKER_00: Storyboard 18, the contrast with protected professions, the privilege of professional secrecy.
• 1:04
  SPEAKER_00: When we talk to a psychologist, solicitor, doctor, or priest, our words are protected by a well-established legal mechanism, professional secrecy.
• 1:13
  SPEAKER_00: This principle, rooted in centuries of legal tradition, establishes that certain conversations are inviolable, even in the face of criminal investigations.
• 1:22
  SPEAKER_00: Characteristics of traditional professional secrecy, very extensive protection.
• 1:28
  SPEAKER_00: Communications remain confidential even in the presence of confessed crimes.
• 1:33
  SPEAKER_00: Limited and specific exceptions.
• 1:35
  SPEAKER_00: Only in extreme cases defined by law can must certain professionals break their silence and us.
• 1:42
  SPEAKER_00: Qualified human control.
• 1:44
  SPEAKER_00: The decision to breach confidentiality always remains in the hands of a trained professional ethical responsibility.
• 1:51
  SPEAKER_00: Professionals are bound by codes of conduct that balance duties to the client and society.
• 1:56
  SPEAKER_00: The real limits of professional secrecy, contrary to common perception, professional secrecy is not absolute.
• 2:04
  SPEAKER_00: There are well-defined exceptions that vary by professional category.
• 2:08
  SPEAKER_00: For solicitors, Art 28 of the code of conduct for solicitors, disclosure is permitted when necessary for the performance of defense activities, preventing the commission of a particularly serious crime, defending oneself in a dispute against one's client, disciplinary proceedings, critical example.
• 2:30
  SPEAKER_00: If a client declares to their solicitor that they intend to commit murder, the protection of life must prevail over the protection of the right to defense, and the solicitor is released from their duty of confidentiality.
• 2:43
  SPEAKER_00: For psychologists, Art 13 Code of Ethics.
• 2:47
  SPEAKER_00: Confidentiality may be breached when there is an obligation to report or file a complaint for offenses that are prosecutable ex officio.
• 2:57
  SPEAKER_00: There is a serious threat to the life or mental and physical health of the subject andor third parties.
• 3:02
  SPEAKER_00: There is valid and demonstrable consent from the patient, important distinction.
• 3:07
  SPEAKER_00: Private psychologists have greater discretion than public psychologists, who, as public officials, have more stringent reporting obligations.
• 3:17
  SPEAKER_00: Sources, PUCE law firm, professional secrecy of lawyers, the law for all.
• 3:24
  SPEAKER_00: Must a psychologist who knows of a crime report the patient?
• 3:27
  SPEAKER_00: AI, as a non-professional chat GPT, operates in a completely different gray area.
• 3:33
  SPEAKER_00: Lack of legal privilege.
• 3:35
  SPEAKER_00: Conversations with AI do not enjoy any legal protection.
• 3:38
  SPEAKER_00: As Sam Altman, CEO of OpenAI, admitted, if you talk to a therapist or a solicitor or a doctor about those issues, there is legal privilege for that.
• 3:48
  SPEAKER_00: There is doctor-patient confidentiality, there is legal confidentiality, whatever, and we haven't solved that yet for when you talk to ChatGPT.
• 3:57
  SPEAKER_00: Automated process.
• 3:59
  SPEAKER_00: Unlike a human professional who evaluates each case individually, ChatGPT uses algorithms to identify problematic content, removing qualified human judgment from the initial screening stage.
• 4:11
  SPEAKER_00: Sources.
• 4:12
  SPEAKER_00: Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist.
• 4:21
  SPEAKER_00: A new paradigm of surveillance.
• 4:23
  SPEAKER_00: The paradox of technological trust.
• 4:26
  SPEAKER_00: The situation creates a troubling paradox.
• 4:29
  SPEAKER_00: Millions of people use ChatGPT as a digital confidant, sharing intimate thoughts, doubts, fears, and even criminal fantasies that they would never share with a human being.
• 4:39
  SPEAKER_00: As Sam Altman reports, people talk about the most personal things in their lives to ChatGPT.
• 4:45
  SPEAKER_00: People use it, especially young people, as a therapist, life coach.
• 4:51
  SPEAKER_00: The risk of criminal self-censorship.
• 4:54
  SPEAKER_00: The awareness that conversations may be monitored could paradoxically push criminals towards more hidden channels, prevent people with violent thoughts from seeking help, create a chilling effect on digital communications, expertise versus algorithms.
• 5:10
  SPEAKER_00: Who decides what is criminal?
• 5:13
  SPEAKER_00: A crucial issue highlighted by critics concerns the expertise of those making the final decisions.
• 5:19
  SPEAKER_00: Human professionals have years of training to distinguish between fantasies and real intentions, codes of ethics that define when to break confidentiality, personal legal responsibility for their decisions, ability to assess context and credibility.
• 5:35
  SPEAKER_00: The Chat GPT system operates with automated algorithms for initial detection, open AI staff who do not necessarily have clinical or criminological training, non-public and potentially arbitrary evaluation criteria, no external control mechanisms, problematic example.
• 5:52
  SPEAKER_00: How does an algorithm distinguish between a person writing a thriller and seeking inspiration for violent scenes?
• 5:59
  SPEAKER_00: Someone fantasizing with no intention of acting.
• 6:03
  SPEAKER_00: An individual who is actually planning a crime sources, tech crunch.
• 6:07
  SPEAKER_00: Sam Altman warns there's no legal confidentiality when using chat GPT as a therapist Shinkai blog.
• 6:14
  SPEAKER_00: OpenAI's chat GPT conversations scanned reported to police igniting user outrage sources.
• 6:20
  SPEAKER_00: OpenAI's chat GPT conversations scanned reported to police igniting user outrage and privacy.
• 6:26
  SPEAKER_00: Fears OpenAI's contradiction, privacy versus security.
• 6:30
  SPEAKER_00: The double standard OpenAI's admission creates a glaring contradiction with its previous positions.
• 6:35
  SPEAKER_00: The company has strongly resisted requests for user data in lawsuits, citing privacy protection.
• 6:42
  SPEAKER_00: In the case against the New York Times, OpenAI argued strenuously against the disclosure of chat logs to protect user privacy.
• 6:49
  SPEAKER_00: The irony of the situation.
• 7:02
  SPEAKER_00: The situation has been further complicated by a court order requiring OpenAI to retain all chat GPT logs indefinitely, including private chats and API data.
• 7:11
  SPEAKER_00: This means that conversations that users believe to be temporary are now permanently archived.
• 7:16
  SPEAKER_00: Sources.
• 7:18
  SPEAKER_00: OpenAI slams court order to save all chat GPT logs, including deleted chats OpenAI lawsuit, 2025, appeals NYT over ChatGPT data possible solutions and alternatives towards an AI privilege.
• 7:32
  SPEAKER_00: As suggested by Sam Altman, it may be necessary to develop a concept of AI privilege, a legal protection similar to that offered to traditional professionals.
• 7:42
  SPEAKER_00: However, this raises complex questions, possible regulatory options, licensing model, only certified AI can offer conversational privilege.
• 7:53
  SPEAKER_00: Wide mandatory training, those who handle sensitive content must have specific qualifications, professional supervision, involvement of qualified psychologists, lawyers in reporting decisions, algorithmic transparency, publication of the criteria used to identify dangerous content, intermediate technical solutions, compartmentalized, AI, separate systems for therapeutic versus general use, end-to-end encryption for sensitive conversations, explicit consent for each type of monitoring, tripartite approach, automatic detection only for immediate and verifiable threats, mandatory qualified human review appeal process for contested decisions, the precedent of digital professionals lessons from other sectors, telemedicine, developed protocols for digital privacy, online legal advice, uses encryption and identity verification, digital therapy, specialized apps with specific protection sources, Simon Willison, OpenAI court order analysis, what this means for AI companies, lessons for the industry.
• 9:04
  SPEAKER_00: The OpenAI case sets important precedents for the entire artificial intelligence industry.
• 9:09
  SPEAKER_00: Mandatory transparency AI companies will need to be more explicit about their monitoring practices.
• 9:16
  SPEAKER_00: Need for ethical frameworks.
• 9:18
  SPEAKER_00: Clear regulation is needed on when and how AI can interfere with private communications.
• 9:24
  SPEAKER_00: Specialized training.
• 9:27
  SPEAKER_00: Those who make decisions about sensitive content must have appropriate skills.
• 9:32
  SPEAKER_00: Legal liability.
• 9:34
  SPEAKER_00: Define who is responsible when an AI system makes an incorrect assessment, operational recommendations for companies developing conversational AI, implement multidisciplinary teams, legal, psychologists, criminologists, develop public and verifiable criteria for reporting create appeal processes for users.
• 9:55
  SPEAKER_00: Invest in specialized training for review staff for companies using AI, assess privacy risks before implementation.
• 10:03
  SPEAKER_00: Clearly inform users about the limits of confidentiality.
• 10:07
  SPEAKER_00: Consider specialized alternatives for sensitive uses, the future of digital confidentiality.
• 10:14
  SPEAKER_00: The central dilemma how to balance the prevention of real crimes with the right to privacy and digital confidentiality.
• 10:22
  SPEAKER_00: The issue is not merely technical but touches on fundamental principles presumption of innocence.
• 10:28
  SPEAKER_00: Monitoring private conversations implies generalized suspicion.
• 10:33
  SPEAKER_00: Right to privacy includes the right to have private thoughts, even disturbing one's preventive effectiveness.
• 10:40
  SPEAKER_00: It is not proven that digital surveillance actually prevents crime conclusions.
• 10:45
  SPEAKER_00: Finding the right balance, OpenAI's revelation marks a watershed moment in the evolution of artificial intelligence.
• 10:52
  SPEAKER_00: But the question is not whether reporting is right or wrong in absolute terms.
• 10:57
  SPEAKER_00: It is how to make it effective, fair, and respectful of rights.
• 11:01
  SPEAKER_00: The need is real.
• 11:02
  SPEAKER_00: Concrete threats of violence, plans for attacks, or other serious crimes require intervention.
• 11:08
  SPEAKER_00: The issue is not whether to report, but how to do so responsibly.
• 11:12
  SPEAKER_00: The fundamental differences to be resolved.
• 11:16
  SPEAKER_00: Training and competence.
• 11:19
  SPEAKER_00: Human professionals have established protocols for distinguishing between real threats and fantasies.
• 11:24
  SPEAKER_00: AI systems need equivalent standards and qualified supervision.
• 11:29
  SPEAKER_00: Specialized training is needed for those who make final decisions.
• 11:33
  SPEAKER_00: Transparency and control.
• 11:35
  SPEAKER_00: Professionals operate under the supervision of professional associations.
• 11:39
  SPEAKER_00: Open AI needs public criteria and external control mechanisms.
• 11:43
  SPEAKER_00: Users need to know exactly when and why they might be reported.
• 11:47
  SPEAKER_00: Proportionality.
• 11:48
  SPEAKER_00: Professionals balance confidentiality with security on a case-by-case basis.
• 11:54
  SPEAKER_00: AI systems need to develop similar mechanisms, not binary algorithms.
• 11:58
  SPEAKER_00: For companies in the sector, the challenge is to develop systems that effectively protect society without becoming tools for indiscriminate surveillance.
• 12:06
  SPEAKER_00: User trust is essential, but must coexist with social responsibility.
• 12:11
  SPEAKER_00: For users, the lesson is twofold.
• 12:14
  SPEAKER_00: Conversations with AI do not have the same protections as traditional professionals.
• 12:19
  SPEAKER_00: This is not necessarily bad if done transparently and proportionately, but it is important to be aware of it.
• 12:27
  SPEAKER_00: The future of conversational AI requires a new framework that recognizes the legitimacy of crime prevention, establishes professional standards for those who handle sensitive content, ensures transparency in decision-making processes, protects individual rights without ignoring security.
• 12:46
  SPEAKER_00: The right question is not whether machines should report crimes, but how we can ensure that they do so with at least the same wisdom, training, and responsibility as human professionals.
• 12:58
  SPEAKER_00: The goal is not to return to AI that is blind to real dangers, but to build systems that combine technological efficiency with ethics and human expertise.
• 13:07
  SPEAKER_00: Only then can we have the best of both worlds, security and protected individual rights, references and sources, futurism.
• 13:17
  SPEAKER_00: Open AI says it's scanning users chat GPT conversations and reporting content to the police.
• 13:24
  SPEAKER_00: Studio Legale Putz, Segreto Professionale del Avocato, Lawyer's Professional Secrecy.
• 13:31
  SPEAKER_00: La Lege Pertuti Must a Psychologist Who Knows of a Crime Report the Pentent?
• 13:36
  SPEAKER_00: Four.
• 13:38
  SPEAKER_00: TechCrunch.
• 13:39
  SPEAKER_00: Sam Altman warns there's no legal confidentiality when using chat GPT as a therapist.
• 13:55
  SPEAKER_00: Simon Willison OpenAI slams court order to save all chat GPT logs, including deleted chat 7.
• 14:02
  SPEAKER_00: SuccessKnox OpenAI Lawsuit 2025, appeals NYT over ChatGPT data article by the AI research team.
• 14:10
  SPEAKER_00: For more insights on artificial intelligence, privacy, and regulation, follow us in our weekly newsletter.
• 14:16
  SPEAKER_00: Share the newsletter.
• 14:17
  SPEAKER_00: Welcome to Electe's Newsletter, English.
• 14:20
  SPEAKER_00: This newsletter explores the fascinating world of how companies are using AI to change the way they work.
• 14:25
  SPEAKER_00: It shares interesting stories and discoveries about artificial intelligence in business, like how companies are using AI to make smarter decisions, what new AI tools are emerging, and how these changes affect our everyday lives.
• 14:37
  SPEAKER_00: You don't need to be a tech expert to enjoy it.
• 14:40
  SPEAKER_00: It's written for anyone curious about how AI is shaping the future of business and work.
• 14:46
  SPEAKER_00: Whether you're interested in learning about the latest AI breakthroughs, understanding how companies are becoming more innovative, or just want to stay informed about tech trends, this newsletter breaks it all down in an engaging, easy to understand way.
• 15:01
  SPEAKER_00: It's like having a friendly guide who keeps you in the loop about the most interesting developments in business technology.
• 15:07
  SPEAKER_00: Without getting too technical or complicated, subscribe now, subscribe to get full access to the newsletter and publication archives.