AI in Education and Research: Towards a More Ethical Engagement
From a technological point of view, AI development up to now has been focussed primarily on extracting data to produce large language models and developing the algorithms necessary to exploit these models. From a human point of view, however, the successful deployment of AI depends also on how well we are able to ethically regulate the technology so as to use it for the common good.
There is a large amount of public writing about the promise of artificial intelligence (AI) for supporting social inclusion, freeing up researchers from grunt work and the “top x AI tools for teachers”. What is less well understood is where “AI” comes from and whose oppression is needed for it to succeed. In the case of AI and education that interests us here, we will show what is at stake in these seemingly user-friendly AI interfaces by answering three interrelated questions about large language models (LLMs) and algorithms.
1. What lies behind AI?
AI relies on datasets scraped from the internet in large language models (LLMs) and algorithms (which are just sets of rules to be followed in calculations). These large – but narrow – datasets encode the values of privileged members of WEIRD (Western/White, educated/English-speaking, industrialised, rich, democratic) societies who design them and profit from them. A minority of humanity is represented on the web; the majority is not.
Students cannot give consent to their data being profited from by platforms that are mandated by their institutions. Moreover, the contribution of African scientists who collected data during the 2020 Covid pandemic was not recognised even as their data were analysed to benefit us all. How do we break with practices of coloniality where raw materials – now data – are extracted and sent to distant elites for value-added manufacture?
AI is also environmentally extractive since the data centres hosting LLMs require enormous amounts of electricity and water for cooling, diverted from humans and places that need it. AI tools reinforce and promulgate dominant epistemologies, further marginalising other ways of knowing and doing. Datafication introduces new actors – digital platforms – as powerful intermediaries in educational processes and decision making. Recent advances appear to owe more to business practices than technical breakthroughs. What research agendas can investigate these new education actors and their opaque value creation models and motivations?
Your open access or pirated articles and books are fodder for LLMs, but you will not be cited. “CAPTCHAs” on websites identify you as a human, and also train LLMs. Voluntary human efforts to produce public goods are reused without permission to generate profits as well as answers to your questions. Even more exploitative are the working conditions of data workers in low-income countries without whom AI tools would not be marketable to schools. AI needs HI (human intelligence) to nudge it – but in which direction?
2. What is data used for and with what effects?
Historical and contemporary biases are captured by data, encoded in LLMs, and perpetuated by algorithmic outputs. This matters since AI is used in selection decisions for colleges and jobs, and to monitor humans in work, exams and society, despite evidence of significant failures and injustices built into theses systems.
LLMs generate a silicon ceiling – an imperceptible, but systemic, barrier to opportunity for marginalised peoples – through biased algorithmic decision making. AI facial recognition systematically misidentified Black women but not White men, and ChatGPT predicted “terrorist” as the word to follow “Muslim” in 23% of test cases. AI replicates, reinforces and exacerbates historical disadvantages and human biases under a veneer of objectivity and scientificity. Actual harms now are as much existential threats to marginalised people as future harms are to us all.
AI benefits those who resemble its techno-optimistic, corporate designers. It is not by chance that most of the research in these hyperlinks is produced by people who do not.
3. How to move towards more ethical engagement with AI?
- Using technology is as essentially human as ethics; let’s lead with ethics. Move from values based on extraction and exploitation towards values more usually found in education (ethics of care) and research (informed consent, precautions against causing harm).
- Centre humans – and education and research – and then consider how technologies support human flourishing for “tech on our terms”.
- Urgently address AI governance to redress existing inequalities and regulate how technology industries structure human choices and make it harder to “click wisely”.
- Recognise the corporate origins and capture of AI narratives, purposes and governance.
- Teach age-appropriate digital literacy (and general literacy) to teachers and students without adding to teachers’ workload.
- Demand algorithmic transparency and ethical audits.
- Use language carefully: generative AI generates; it does not think, predict, decide or hallucinate.
- Change citation practices and institutional and funding incentives to avoid the “compound effect” of rewarding privileged groups and further marginalising others.
- “Imagine and craft the worlds you cannot live without, just as you dismantle the ones you cannot live within” (Ruha Benjamin).
Drawing attention to issues that AI generates for education does not deny that digital technologies can, and might, be used for the common good. But we should not confuse corporate techno-optimism for human-centred AI use. Even if the genie is out of the bottle, it is still bound to it and can be put back in.
In November 2021, 193 states adopted UNESCO’s global standard on AI ethics. Governments, international organisations and the private sector could govern this technocosm in the common interest to overcome allocational and representational harms.
Using technology is as essentially human as ethics; let’s lead with ethics. Governance and regulation need to be based on the duties of decision makers to safeguard rights for humans, and ensure that human and planetary purposes guide algorithms.
Now that would be intelligent.
TABLE: Roles and Missions of the University
|Role and missions
|Serving God and Church; Serving Science
|Serving Science; Serving State and Nation
|Serving Society and Humanity; Serving the Market
Source: Marie-Laure Salles-Djelic, “Scholars in the Audit Society: Understanding our Contemporary Iron Cage”, in Scholars in Action: Past – Present – Future, ed. Lars Engwall (Uppsala Universitet, 2012), p. 99.
DEFINITIONS: Selection of Terms Related to Higher Education
A university (from Latin universitas “a whole”) is an institution of higher (or tertiary) education and research which awards academic degrees in several academic disciplines. Universities typically offer both undergraduate and postgraduate programmes. The word university is derived from the Latin phrase universitas magistrorum et scholarium, which roughly means “community of teachers and scholars”. (Wikipedia)
Higher education, also called post-secondary education, third-level or tertiary education, is an optional final stage of formal learning that occurs after completion of secondary education. This consists of universities, colleges and polytechnics that offer formal degrees beyond high school or secondary school education. The right of access to higher education is mentioned in a number of international human rights instruments. The UN International Covenant on Economic, Social and Cultural Rights of 1966 declares, in Article 13, that “higher education shall be made equally accessible to all, on the basis of capacity, by every appropriate means, and in particular by the progressive introduction of free education”. (Wikipedia)
A college (Latin collegium) is an educational institution or a constituent part of one. In most of the world, a college may be a high school or secondary school, a college of further education, a training institution that awards trade qualifications, a higher-education provider that does not have university status (often without its own degree-awarding powers), or a constituent part of a university. The word is generally also used as a synonym for a university in the US. (Wikipedia)
Apprenticeship is a system for training a new generation of practitioners of a trade or profession with on-the-job training and often some accompanying study (classroom work and reading). Apprenticeships can also enable practitioners to gain a license to practice in a regulated occupation. Most of their training is done while working for an employer who helps the apprentices learn their trade or profession, in exchange for their continued labor for an agreed period after they have achieved measurable competencies. (Wikipedia)
An information society is a society where the usage, creation, distribution, manipulation and integration of information is a significant activity. Its main drivers are information and communication technologies, which have resulted in rapid growth of a variety of forms of information. Proponents of this theory posit that these technologies are impacting most important forms of social organisation, including education, economy, health, government, warfare, and levels of democracy. (Wikipedia)
A knowledge society generates, shares and makes available to all members of the society knowledge that may be used to improve the human condition. A knowledge society differs from an information society in that the former serves to transform information into resources that allow society to take effective action, while the latter only creates and disseminates the raw data. The capacity to gather and analyse information has existed throughout human history. However, the idea of the present-day knowledge society is based on the vast increase in data creation and information dissemination that results from the innovation of information technologies. (Wikipedia)
TABLE: The Top 12 Host Countries of International Students in 2022 (by number of students)
Source: Project Atlas.
GRAPH: The Twelve Countries with the Most Universities in 2023
BOX: The Humboldtian Model of Higher Education
The Humboldtian model of higher education or Humboldt’s Ideal is a concept of academic education that emerged in the early 19th century and whose core idea is a holistic combination of research and studies. It integrates the arts and sciences with research to achieve both comprehensive general learning and cultural knowledge. Several elements of the Humboldtian model heavily influenced and subsequently became part of the concept of the research university. The Humboldtian model goes back to Wilhelm von Humboldt, a Prussian philosopher, government functionary and diplomat who, in the time of the Prussian reforms, relied on a growing, educated middle class to promote his claims about general education.
As a privy councillor in the Interior Ministry, he reformed the Prussian education system according to humanist principles. He founded the University of Berlin, appointing distinguished scholars to both teach and conduct research there. Several scholars have labeled him the most influential education official in German history. Humboldt sought to create an educational system based on unbiased knowledge and analysis, combining research and teaching while allowing students to choose their own course of study. The University of Berlin was later named the Humboldt University of Berlin, after him and his brother, the naturalist Alexander von Humboldt. His educational model went beyond vocational training in Germany.
In a letter to the Prussian king, he wrote: “There are undeniably certain kinds of knowledge that must be of a general nature and, more importantly, a certain cultivation of the mind and character that nobody can afford to be without. People obviously cannot be good craftworkers, merchants, soldiers or businessmen unless, regardless of their occupation, they are good, upstanding and – according to their condition – well-informed human beings and citizens. If this basis is laid through schooling, vocational skills are easily acquired later on, and a person is always free to move from one occupation to another, as so often happens in life.”
The philosopher and former State Minister for Culture of the Federal Republic of Germany, Julian Nida-Rümelin, has criticised discrepancies between Humboldt’s ideals and the contemporary European education policy, which narrowly understands education as preparation for the labor market, arguing instead that one needs to decide between McKinsey’s and Humboldt’s ideals.