Indigenous Data Sovereignty & Ethical Guardrails

So far, we’ve explored some of the risks associated with AI and considered how these systems can affect learning, information, and decision-making. Responsible AI also requires us to think about the choices we make around these systems: what information they can access, how they are used, who has authority to make those decisions, and who remains responsible for the outcomes.

These questions become especially important when AI is used with information or knowledge connected to communities. When we hear the word data, we might think of numbers, records, or personal information. In Indigenous contexts, data can include much more than that.

For Indigenous Peoples, questions about data are also connected to a longer history of who has had the authority to collect, interpret, store, and use that information. These questions are central to Indigenous Data Sovereignty, which we’ll explore more closely in the next section.

At its core, Indigenous Data Sovereignty asks us to think differently about access and authority. Being able to find, view, or download information does not necessarily mean having the authority to use, reproduce, share, or transform it (Bush et al., 2026; First Nations Information Governance Centre [FNIGC], n.d.).

This is particularly important when information and knowledge are connected to a community. Indigenous data can include information about people, communities, lands, languages, cultures, histories, and knowledge. Decisions about how that information is collected and used may involve rights, relationships, and responsibilities that extend beyond individual consent.

Why Does this Matter?

When Indigenous Peoples do not have meaningful authority over how their data and knowledge are used, the harm can extend beyond privacy. Cultural knowledge can be removed from its original context, sacred or restricted knowledge can be reproduced inappropriately, and information can be collected, interpreted, or used without meaningful community authority or consent.

These concerns are connected to a longer history in which information about First Nations was often collected, interpreted, and used by outside institutions rather than governed by First Nations themselves (FNIGC, n.d.).

One Important Response: OCAP

The First Nations Principles of OCAP emerged from First Nations efforts to change this relationship and maintain greater authority over their own information. Developed by the First Nations Information Governance Centre (FNIGC), the principles describe how First Nations exercise jurisdiction over how their information is collected, used, interpreted, and shared.

A Bit More Context

The four principles work together. For example, ownership and possession are related, but they are not the same. A First Nation may own information even when it is physically stored by another organization. However, without possession or meaningful control over where that information is held, exercising ownership can become more difficult.

OCAP also has an important scope. It is specifically a First Nations framework, rather than a framework for all Indigenous Peoples. Different First Nations may apply the principles according to their own laws, protocols, worldviews, and priorities, while Inuit and Métis communities may have their own approaches to data governance (FNIGC, n.d.).

What Does This Mean When Using AI?

AI adds another layer to these considerations. Educators should be extremely cautious when using AI with Indigenous knowledge. Indigenous knowledge is connected to particular people, relationships, places, and protocols. Generative AI reproduces information without preserving all of the necessary context because it is largely probabilistic. This means an output may sound credible while missing where the knowledge comes from, who holds it, or the protocols that shape how it should be shared (Groten et al., 2024).

Responsible AI use always requires us to consider the source, context, authority, and appropriateness of the technology for a given task. When working with Indigenous data and knowledge, these considerations take on additional significance, including respecting Indigenous Data Sovereignty, cultural protocols, and the rights of Indigenous Peoples and communities to determine how their knowledge is shared, represented, and used.

Before integrating AI into your teaching practice, consider why you might use it in the first place. What need does it address? Does it offer a meaningful benefit over existing approaches? Could it support student learning, or might it take away opportunities for students to develop important skills themselves?

Education researcher and consultant Leon Furze makes a similar case in his Need/Potential Matrix, which encourages schools to consider the needs a technology addresses rather than adopting new tools simply because they are available. He also cautions that, given the speed of AI development, sometimes waiting is more appropriate than rushing to adopt a poorly designed or unsuitable tool. A new capability, on its own, is not a reason to introduce a tool into classrooms. Once there is a clear purpose, schools and school systems can consider what safeguards need to be in place.

Of course, deciding whether AI serves a meaningful educational purpose is only one part of the picture. Educators should not be expected to independently evaluate the technical safety, security, or privacy practices of AI tools, or understand the internal practices of the companies that build AI models. School authorities have an important responsibility to establish clear expectations, review tools, and provide guidance about what is appropriate for classroom use. Educators can then draw on that guidance alongside their own professional and pedagogical judgment.

School systems don’t need to start from scratch when deciding how AI can be used safely. In recognition that many educators and students are using AI, many education systems and organizations across Canada have already developed AI guidelines and resources to help school boards and educators navigate the evolving AI landscape. These resources offer useful starting points for thinking about responsible AI use, including privacy, student safety, equity, academic integrity, and the role of educator professional judgment.

Some Canadian examples include:

• British Columbia Ministry of Education and Child Care – Considerations for Using AI Tools in K-12 Schools
• Alberta School Boards Association (ASBA) – AI Policy Guidance
• New Brunswick Department of Education – Recommended Approaches to Generative AI
• Ontario Association of School Business Officials and the Educational Collaborative Network of Ontario – Guidelines for Responsible Use of Generative Artificial Intelligence

These guidelines act as guardrails. They set boundaries around how AI can be used while giving educators practical guidance for navigating AI in their everyday work. Educators are encouraged to check their school board or provincial ministry of education website to understand what, if any, AI guidelines or policies are already in place and how they might apply to their teaching practice.

Questions for Evaluating AI Use

Educators make important decisions every day about what technologies to invite into classrooms. The following questions can help you think through whether and how an AI tool might be appropriate for your teaching practice. You don’t need to have all the technical answers yourself. Knowing what to ask, what to look for, and when to seek guidance from your school authority is an important part of responsible AI use.

Match the Guardrail to the Risk

Not every use of AI requires the same level of oversight. As the potential impact of an AI-supported decision increases, so should the safeguards around it.

Guardrails Should Support Educators

Guidance is most useful when educators are supported in applying it. UNESCO's AI Competency Framework for Teachers emphasizes that responsibility for safe and ethical AI use cannot rest with educators alone. Schools, policymakers, and teacher-education institutions also have a responsibility to prepare and support educators in using AI appropriately (UNESCO, 2024).

Professional learning can help educators better understand how AI works, recognize potential risks, and make informed decisions about appropriate use. In this way, guardrails provide both boundaries and support, empowering educators to use their professional judgment within clear shared expectations.

Continuously Monitor and Evolve

AI technologies, their capabilities, and the ways they are used in schools will continue to change. Guardrails should evolve with them.

UNESCO emphasizes that the selection, use, and monitoring of AI should remain human-controlled and human-accountable. Its framework also positions educators’ AI development as an ongoing process, supported through continuous professional learning as technologies and their implications evolve (UNESCO, 2024).

In practical terms, schools can monitor how AI is being used, gather feedback from educators, students, and families, and revisit their guidance over time to respond to emerging risks as AI technologies and their uses continue to evolve.

In Chapter 2, we explored how the data used to train an AI model shapes the patterns it learns. When some communities, languages, cultures, or perspectives are missing or poorly represented in that data, those gaps can show up in the model’s outputs.

An AI system might produce stereotypes, overlook important cultural differences, struggle with less-represented languages, or present one perspective as though it represents everyone. This becomes particularly important when AI is used to create educational content about cultures or communities. Cultural safety, however, involves more than whether an AI-generated resource is accurate. It also means considering where knowledge comes from, who has the authority over its use, and what context might be lost when AI reproduces or changes it.

Scenario: Bridging Indigenous Knowledge Into a Science Lesson

You’re planning a science lesson about the night sky and want to meaningfully include Indigenous perspectives and knowledge. You’ve found that connecting with appropriate local knowledge holders isn’t always straightforward, and you want to avoid leaving these perspectives out simply because they are harder to access.

You consider using a generative AI tool to help you find and understand Indigenous stories and knowledge connected to the night sky.

Cultural safety means approaching AI-generated or AI-mediated content with care, particularly when it represents cultures, languages, or communities beyond our own experience. Community-developed resources and guidance can help educators work from appropriate sources and avoid relying on AI to fill in the gaps in knowledge it may not be equipped to represent ethically.

Responsible AI use in education is a shared responsibility. Students, educators, school and system leaders, technology providers, families, and communities can all have different roles in shaping how AI is used and what protections are needed.

For educators, this means using professional judgment to decide when and how AI is appropriate in their practice. School and system leaders have a broader role in evaluating tools, establishing clear guardrails, and creating processes for responding when concerns arise. Students also need opportunities to understand not only how to use AI, but how to question it, recognize its limitations, and make thoughtful choices about when to use it.

Communities matter in these conversations too. Decisions about technology can affect different people in different ways. Creating meaningful opportunities for those affected by these decisions to inform them can help surface perspectives, priorities, and concerns that might otherwise be missed.

This work also requires ongoing reflection. AI technologies will continue to change, and our understanding of their benefits and risks will change with them. Policies, practices, and guardrails will need to be revisited as new evidence, technologies, and concerns emerge.

Building an ethical AI culture requires us to keep asking thoughtful questions, involve the people affected by the decisions, and keep human judgment and responsibility at the centre of how AI is used.

These same questions about responsibility, transparency, and appropriate AI use also show up in students’ everyday learning. Next, we’ll explore what academic integrity looks like in the age of AI, including how educators can set clear expectations while supporting meaningful student learning.