Victorian Case Worker Employs ChatGPT to Create Child Protection Report
We independently review everything we recommend. When you buy through our links, we may earn a commission which is paid directly to our Australia-based writers, editors, and support staff. Thank you for your support!
Use of ChatGPT by Victorian Case Worker Leads to AI Prohibition in Child Protection
The Department of Families, Fairness, and Housing (DFFH) in Victoria has been mandated to prohibit and restrict access to generative AI tools, such as ChatGPT, following an incident in which a case worker employed the AI to compose a child protection report for the Children’s Court. The employment of ChatGPT in this situation raised significant issues surrounding privacy, data security, and accuracy, prompting a more extensive investigation into the department’s practices.
Quick Overview
- A case worker in Victoria utilized ChatGPT to create a child protection report submitted to the Children’s Court.
- The report included major inaccuracies, incorporating incorrect personal data.
- The Office of the Victorian Information Commissioner (OVIC) identified this as a serious violation of privacy and security protocols.
- An internal audit revealed that nearly 900 DFFH staff accessed ChatGPT, raising alarms about possible widespread abuse.
- DFFH has been instructed to outlaw all generative AI tools and restrict access to these resources across the department by November 5, 2023.
- The case worker implicated is no longer part of the department.
ChatGPT and the Child Protection Report Situation
The Office of the Victorian Information Commissioner (OVIC) disclosed that a child protection worker relied on ChatGPT to generate material for a report sent to the Children’s Court. The report aimed to evaluate the risks facing a child living with parents accused of sexual offences. Nevertheless, the AI-produced document included inaccuracies and minimized the associated risks, which could have led to severe consequences if it had affected the court’s ruling.
“Thankfully, it did not alter the outcome of the child’s case, but the potential harm that could have occurred is evident,” OVIC’s inquiry noted. The investigation emphasized that the report should have reflected the case worker’s evaluation rather than a summary generated by an external party.
Concerns Regarding Privacy Breach and Data Security
The case worker provided “personal and sensitive” information related to the case to ChatGPT, a tool created by OpenAI, an overseas entity. This resulted in a significant infringement of Victoria’s privacy regulations as the data was shared outside the control of the DFFH. OVIC voiced apprehensions that OpenAI, which now possesses this information, might use or disclose it further.
The OVIC inquiry detected several indicators in the report that hinted at the use of ChatGPT, such as inaccuracies in personal details, inappropriate sentence construction, and language that did not align with the training given to child protection staff.
Greater Utilization of Generative AI in the Department
In light of this specific case, the DFFH undertook an internal review which indicated that ChatGPT may have been utilized in nearly 100 cases managed by the same unit over the course of a year. Additional investigations uncovered that around 900 employees throughout the department accessed ChatGPT, which accounts for nearly 13% of the DFFH workforce.
Although the department initially downplayed the incident, asserting it was an isolated event, OVIC’s discoveries suggested a more encompassing issue within the department. The extensive use of generative AI tools without proper policies or training raised concerns about the security of sensitive data and the reliability of AI-generated material in official documents.
Department Instructed to Prohibit AI Tools
In light of the findings, OVIC issued a compliance notice ordering the DFFH to prohibit the use of generative AI tools, including ChatGPT, ChatSonic, Claude, Copy.AI, Grammarly, Jasper, and Microsoft 365 Copilot. The department was instructed to restrict access to these tools by November 5, 2023, and to establish technical safeguards to prevent future violations.
The case worker associated with the report is no longer in the department’s employ, and DFFH acknowledged the unauthorized use of ChatGPT. However, the department maintained that no evidence of widespread AI tool usage for sensitive matters existed—a claim OVIC contested based on its analysis.
Consequences for AI Utilization in the Public Sector
This situation raises significant concerns about the deployment of generative AI tools in critical government functions. While AI tools like ChatGPT provide convenience and effectiveness, their application in fields such as child protection, healthcare, and law enforcement necessitates strict oversight to ensure the safety of sensitive data and the accuracy of the produced information.
As AI technologies advance, public sector entities must create clear policies, provide sufficient training, and implement technical measures to avert unauthorized usage and potential privacy infringements.
Summary
The incident involving a Victorian child protection worker employing ChatGPT to generate a court report has highlighted the dangers of utilizing generative AI tools in sensitive and private work settings. The report was found to be inaccurate and violated privacy regulations by disclosing personal information to an overseas AI provider. Consequently, Victoria’s DFFH has been mandated to prohibit the use of generative AI tools and bolster data protection protocols.
FAQs
Q: What actions did the case worker take with ChatGPT?
A:
The case worker utilized ChatGPT to draft a child protection report submitted to the Children’s Court. The document was designed to evaluate the risks to a child in a sensitive situation, but the AI-generated content included inaccuracies and minimized the associated risks.
Q: Why is this situation regarded as a privacy violation?
A:
The case worker inserted personal and sensitive information regarding the child and parents into ChatGPT, a tool created by OpenAI, an overseas company. This led to the data being shared outside the DFFH’s jurisdiction, breaching state privacy laws.
Q: What was the extent of generative AI use within the DFFH?
A:
A subsequent internal review revealed that nearly 900 employees accessed ChatGPT, with signs of AI use in drafting documents across approximately 100 cases managed by the same unit over a year. This indicates that the utilization of generative AI tools might have been more prevalent than initially reported.
Q: What measures has the DFFH implemented to resolve the situation?
A:
After the OVIC investigation, the DFFH received an order to ban all generative AI tools, including ChatGPT, and to establish technical controls to restrict employee access to these resources. The department has until November 5, 2023, to conform to these directives.
Q: Is the case worker still part of the department?
A:
No, the case worker who used ChatGPT to draft the report is no longer part of the DFFH.
Q: Will the DFFH offer AI tool training to its personnel?
A:
Currently, there is no indication that specific training will be provided to staff regarding AI usage. Nonetheless, the incident underlines the necessity for defined guidelines and training on the responsible use of AI tools, particularly in sensitive areas like child protection.
Q: What broader implications does this incident have for AI use in government?
A:
This incident emphasizes the critical need to regulate AI usage in government, especially in areas that involve sensitive data. The risks of data breaches and inaccuracies highlight the importance of stringent oversight, appropriate training, and clear directives to ensure the responsible and secure use of AI tools.
