Social and Ethical Risks Posed by General-Purpose LLMs for Settling Newcomers in Canada

TL;DR

This paper discusses the potential social and ethical risks of using general-purpose LLMs like ChatGPT in Canada's settlement sector for newcomers, emphasizing the need for tailored, trustworthy AI solutions.

Contribution

It highlights the risks of unguarded AI use in settlement services and advocates for developing customized, community-aligned LLMs with human oversight.

Findings

Unregulated AI tools may harm immigrant communities.

Need for AI literacy and tailored models in settlement services.

Importance of integrating AI responsibly into existing workflows.

Abstract

The non-profit settlement sector in Canada supports newcomers in achieving successful integration. This sector faces increasing operational pressures amidst rising immigration targets, which highlights a need for enhanced efficiency and innovation, potentially through reliable AI solutions. The ad-hoc use of general-purpose generative AI, such as ChatGPT, might become a common practice among newcomers and service providers to address this need. However, these tools are not tailored for the settlement domain and can have detrimental implications for immigrants and refugees. We explore the risks that these tools might pose on newcomers to first, warn against the unguarded use of generative AI, and second, to incentivize further research and development in creating AI literacy programs as well as customized LLMs that are aligned with the preferences of the impacted communities. Crucially, such technologies should be designed to integrate seamlessly into the existing workflow of the settlement sector, ensuring human oversight, trustworthiness, and accountability.