The Tamesis
When I asked ChatGPT for a list of experts on cotton and water management, the results were exclusively Western academics and climate NGOs. Another prompt asking about the source of the water-saving data read: “Training data is unevenly distributed globally. Indigenous, local, or unpublished farmer knowledge is underrepresented.” (OpenAI, the U.S. developer of ChatGPT, was not immediately available for comment.)
Nor is it as simple as inviting traditional communities and First Nations to the table. Even if asked to participate, many people do not want their knowledge to be exploited by AI. But if underlying biases are not addressed, they risk undermining progress on sustainability, diversity and inclusion.
Who benefits from artificial intelligence?
Taylor Sparklingeyes is a senior data sovereignty specialist (involving the collection, ownership and use of data) at Shared Value Solutions, an environmental and community development consultancy, and a member of Goodfish Lake First Nation, part of Canada’s Treaty 6 Territory. After members of the Indigenous communities she was working with began asking what AI was and whether they should use it, Sparklingeyes joined the AI Indigenous Pathfinders program run by Montreal-based AI research institute Mila, which aims to empower First Nations, Inuit and Métis participants to learn Indigenous-centered approaches to engaging with AI.
Sparklingeyes warns that the speed at which the technology is developing – it is the fastest spreading in human history – could lead to Indigenous communities neglecting safety, security and privacy concerns. “Working with Indigenous communities is one thing, and if you want to be true allies, sometimes you don’t have to worry about timing and expectations. It takes a long time to build those trusting relationships, and they should be the foundation of that work, whether it’s around governance, co-design of data, or the impact of these systems on communities,” she said.
Some experts worry that bias in AI is not only real, but intentional. Deepak Varuvel Dennison, an artificial intelligence researcher and doctoral student at Cornell University, believes that artificial intelligence platforms have direct financial incentives to pay for knowledge that reflects the majority of the paying user base, rather than niche or underrepresented topics. Reiterating user biases is more likely to keep people on the platform because their beliefs will not be challenged, while a user base concentrated in the Global North further contributes to the “silicon gaze” and marginalizes indigenous knowledge. “Things that have economic value to those in power are promoted, [what isn’t] Loss of legitimacy,” Denison said.
Calculate access rights
Complicating indigenous representation in AI is the larger question of whether traditional communities want the technology to have access to their data and insights. For many creators in the northern hemisphere, this is the first time they are considering how their data is used and how they can take back ownership. For Indigenous communities, however, the fight for data sovereignty is nothing new.
“[Indigenous communities] “All people have unique experiences when it comes to the historical harms of knowledge and data extraction,” said Sparklingeyes, noting that many communities don’t even have access to data about them because it is often extracted through force or under unfair and misleading conditions. This data can range from maps to artwork, and some of this data may be used to train artificial intelligence, if it exists online, in scientific journals, or in government databases, all of which is mined for training purposes. However, it is likely that it will be removed from its original context and presented in many forms. According to ChatGPT’s contextual description of the source material, Western papers and research materials, as there is a disproportionate share of freely available English-language research materials from high-income countries.
