Insufficient Data for an Ethical Answer

A Case for User-In-Loop (‘UIL’) and ‘Dataset Governance’ Applications elling in Examinations of the Illicit Trade of Antiquities. Catalysed by alacritous technological advancements and ever-so demanded by the changing geopolitical environment of competing superpowers, artificial intelligence has far evolved from its origins in university conference workshops, in order to become unarguably one of the cornerstones of human functionality. Professor Melanie Mitchell, in her insurmountable treatise, ‘Artificial Intelligence: A Guide for Thinking Humans’ summarises this technological trolley-problem perfectly, through tracing the historical intersection between cognitive decision making and the new digital frontier. Her emphasis on the importance of understanding the ontological psychology of Deep Learning Models and Natural Language Processing tasks, have allowed specialists from a range of disciplines to understand how important the internal investigation of our own psyches are. This is especially vital within the cultural heritage sector, as we aim to discern the reasoning that drives and adjudicates a range of decisions made The analysis of illicit markets and their subsequent trading behaviours requires researchers, law enforcement, preservationists, military strategists, and diplomatic attachés to work with 20% of the available intelligence. The drive, and raw motivation to recover the other 80%, hidden beneath the surface, is what pushes us all to work under such incredible pressure and circumstances. However, this age-old conundrum (that has stumped the archaeological, legal, and intelligence sectors for at least five decades) has led to notably dangerous policy outcomes, disputing social consensus, and a general practitioner bias that limits one’s ability to action any reliable change. With the black market of art and antiquities effervescently becoming one of the largest security issues of the 21st century, all personnel working to of Psychology in the Further Integration of Artificial Intelligence Modcombat heritage crime have been rightfully forced to reckon with the morality of the tactics, techniques and procedures (TTP’s) used to analyse and execute relevant research operations. This article will specifically comment on the increased integration of TTP’s associated with artificial intelligence, in a variety of cultural heritage projects and missions currently in operation.

The morality concept I have subsequently coined, ‘dataset governance’ in this article, was first introduced to me in theory regarding User-In-Loop (‘UIL’) processes. A concept used temporally and spatially (principles familiar to archaeologists), computer science uses UIL to monitor, and equalise decisions made between human and artificial users. In simpler terms, User-In-Loop integrations allow us to continuously improve behaviours without risk of bias swinging too far to one side of the pendulum. It encourages us to work alongside machine learning processes to evaluate our own decisions, as well as the unconscious bias often overlooked in datasets. It has already been successfully applied to Deep Learning Models, whereby AI is trained to mimic and extract information drawn from human neurological networks, replicating the brain’s architecture to generate unique responses. This includes Convolutional Neural Networks, Restricted Boltzmann Machine Networks, as well as Long or Short-Term Memory Recurrent Networks. Research projects utilising these UIL computer psychologies within studies of the illicit trade of antiquities, have proven incredibly successful, in both their results and in the limitation in their margins of error. The Provenance Lab at Leuphana University is one example amongst many research projects currently utilising such AI tasking, but its significant mission We will always

focus on incorporating UIL protocols means it is worth mentioning further. The provenance data for suspectedly looted artefacts is delimited into plain text and divided into conclusive statements, dividing the known intelligence regarding the history of particular items, into information ‘spans’ that are deemed confirmed, before being disambiguated into categorisation tasks. These provenance spans for the artefacts are subsequently classified into further sections of ‘knowledge’, when fed through their program, and monitored by human experts via UIL, provide a resulting definitive statement regarding the known and unknown provenance gaps for objects. In comparison to other forms of black market trafficking, such as firearms, wildlife, or narcotics, the antiquities trade (due to its physical and cultural age) has survived and often profited off of the lack of digitised evidence. Bolstered by the amount of licenses, attributive reviews, ownership statements, and receipts that often go “missing to time”, false provenance histories understandably mark a significant amount of the artefacts that are successfully repatriated. Thus, if we are to continue hailing artificial intelligence as a significantly more efficient and absolute analytical solution, how can we further integrate computer principles, such as those aforementioned, without identifying the necessity in making sure that datasets are agreed upon with limited bias? Looking past the headlines suggesting impending doom and continuous references to apocalyptic computing systems such as HAL 9000 or Skynet (additionally, see this article’s Asimov title), AI continues to make mistakes in our world. Despite our technological advances that have assimilated neural architecture into computational algorithms, the aforementioned models still cannot recognise the nuances and complexities often dictated in historical records. History is malleable, undefined.

In spite of artificial intelligence existing since the 1960’s, the mention of any discussion nearly resembling a ‘dataset governance’, did not truly come into fruition until 2014, alongside the creation of Generative Adversarial Networks, such as America’s ChatGPT and its Chinese counterpart in the form of DeepSeek – now banned on all federal government systems and devices in my home country of Australia. Graham, Yates, and El-Roby’s article on investigating illicit trafficking through knowledge-graphs assembled through these types of pre-trained transformers, touched on this ethical concern in quantitative terms. Rightly so, the authors comment that a theoretical algorithm for allowing large language models to work with semantic (rather than definitive) statements, to translate what is true and what is unknown (rather than false), simply does not exist yet. As such, traffickers in their study were labelled ‘occasional art thiefs’, when their convictions would certainly say otherwise.

In addition to this, I call upon the earlier argument made regarding my hypothetical ‘20%’ measure, ultimately representing the baseline of information available to investigators before further intelligence is uncovered. The same aforementioned study regarding knowledge maps, noted that certain statements fed into the generative pre-trained transformers, missed directionality amongst trafficking events. For example, confirmed events such as the import, transfer, or facilitation of sale amongst individual artefact cases, could not be differentiated by the models. Figure A which sold to Figure B, was presented in the model as, Figure B sold to Figure C, with no mention of Figure A. This example summarises how cultural heritage trafficking and its recovered documentation (whether it be conclusive or forged) is often vague, incomplete, subjective, and uncertain. Unless we can develop a learning model or appropriate algorithm to account for these nuances, we must treat all AI-exploited findings with caution, ethical allowance, or an oversight mentality that allows for both further User-In-Loop approval and testing set upon standards determined by this ‘dataset governance’.

Despite clear separations from the public sphere, both the archaeological and intelligence sectors have been using automated systems of knowledge development for decades. However, with the increased utilisation of generative pre-trained transformers into workflow and operation development/execution, many warning signs have correctly been issued regarding the results that these investigative projects produce. It is worth noting additionally, that the headlines written regarding artificial intelligence have been largely based in political fear-mongering, whereby an absence of understanding has been easily supplemented by an asymmetrical warfare conducted by media outlets. As we continue to understand provenance events and the overall economic cognition of the illicit trade of antiquities, we must call upon the further integration of User-In-Loop processes and an overall ‘dataset governance’. Until we can develop Deep Learning Models that can precisely, completely, objectively, and certainly understand the minds of our adversaries, human experts will always be needed. We will always need practitioners with open hearts and minds to recover missing heritage. Humans created the items trafficked themselves, and as such, we must be included in its safe return.

---

The original pages

As published in Issue 4: Limitless. Click any page to view it full size.

First published in Issue 4: Limitless, the Innately Science newspaper.