EL TURCO

The avatar to the right is operated directly and live by a human hidden in the system. The setup is similar to the “Mechanical Turk”, an automated chess player that surprised the 18th century aristocrat courts, but later appeared to hide a human player. The left avatar is operated by an LLM machine. We use Claude 3.5 Sonett from Anthropic (updating to any best future Large Language Model) which is chained to a pipeline of different AI tools. It presents an identity that is constituted as a mix of different fictional and historical personalities. In essence the construct can be seen as a format, a shell, since in principle it could be holding any desired profile and narrative. It seems to evoke allusions to what French philosopher Jean Baudrillard describes as a simulacrum: a system that generates a world of symbols and signs without any original or “grounded” relationship to the world. Despite the machine displaying impressive and surprising capabilities in mimicking knowledge and understanding, the effortless “talking machine” seems to reveal itself, according to some critical observers, as just quite an advanced computational “magician’s trick”, while the discussion about the machine’s true degree of intelligence remains the subject of controversy.

Technical Description of the Live Dialogue System

The avatar to the right in the green field (presented in the menu in different gender and racial formats to choose from) is operated directly and live by a human hidden in the system. The actor’s face is captured through a geometrical smart grid and mapped onto the avatar’s face. An attached live data stream application enables the actor to operate the avatar directly and in real time, generating a lifelike mimicry of human expressions on the avatar’s face. The actor’s spoken words then trigger a pre-encoded pipeline of multiple AI components to enable a live conversation: The human voice output is converted simultaneously into text which prompts the second avatar that can take on different identities as well (see menu). The avatar to the left in the red field is generating text through a natural language processing AI system. We use statistical based probabilistic Large Language Models called LLM to generate text. The text is simultaneously converted into audio. Through an audio-to-face AI module, the machine-generated audio output also operates his own lip sync, gestures, facial expressions, and body movements.

Some thoughts on the current state of AI Large Language Models and AI technology in general

Although the current AI Large Language Models generate representations of semantic dimensions, at this point within the accelerating development of AI technology the machines seem to hold no conceptual understanding of language nor the world. Nevertheless, renowned AI specialists are claiming Large Language Models as Claude 3.5 Sonnet or others can perform a certain degree of reasoning. Predicting the next word still seems far removed from consistent realistic or hypothetical modeling, critical and strategic thinking or drawing logical and causal inferences for example, which allows us to extract meaning in so many intricate ways instead of being exposed to showers of noise. Nevertheless, the increasing level of coherence in Large Language Models can present a forceful challenge for our ability to even discriminate between true and false, not even considering additional social disruption and change that comes with the technology.

In addition: who codes into the machine what is true and false on subjects that involve value statements and a specific world view? All this could implicate unpredictable manipulative powers. And when considering who encodes goals and motivations into machines, for example into lethal AI weapons, we could start to doubt whom we must fear more — AI machines or ourselves?

Would additional new technology and the chaining of AI modules elevate AI results to a whole new scenario of the much-discussed “singularity”? Could AI involve the end of the lead of human intelligence by its manipulative omnipresence even if artificial general intelligence will be just a fiction? In addition we never see a relationship in which the less intelligent entity controls the smarter one except for the mother-child relationship. Or will AI be the most potent extension of man’s technological grip on the world, that can tackle the biggest problems through a vast array of applications across many fields in the most promising way as AI advocates like to convey? In any case AI strongly confronts us with its impactful powers in practical applications and the challenge it presents to rethink intelligence.

Artist

I am a German-American, US-Based artist, working with different international institutions, but mostly with MIT faculty. At MIT I have been Artist in Residence from 2017-2019.

My works link art and science to address contemporary issues, often incorporating themes related to philosophy and literature. I explore the crossover between science and art through media such as living biological materials, nanomaterials, algorithms & numbers, experimental setups and installations. This practice includes scientific concepts, methods and tools, engaging with science directly. The collaborations with scientists take place for example in the fields of human and plant-genetics, quantum- and astrophysics, AI, math and various types of engineering. The heterogeneous appearance of my works, results from the variety of topics and strands in science I deal with.

Focused on the advanced science of our era, I feel compelled to re-affirm the Romantic paradigm of “the new” as well as the role of avant-garde art relating to radical experiments of Classical Modern Art throughout the medium and its combination with the arts.

Collaborators

Eli Barzilay, PhD

Hacker, Teacher, Programmer; Northeastern University; website.

Ben Greenman,

University of Utah website.

Zhiling Huang

Northeastern University; website.

Xiangqing Guan

Digital kitsch maker; Northeastern University; website.

With many thanks for the kind support to:

Jianchi Zhou

Northeastern University; website.

Hongyin Luo

Massachusetts Institute of Technology.

Varun Shankar

University of Utah; website.

El Turco #1 / Conversations with an AI (recorded version)

Steve DiPaola, PhD, Prof

School of Interactive Arts & Technology (SIAT); Past Director: Cognitive Science Program at Simon Fraser University; research site; artwork site.

Gaines Semler

is a Brooklyn-based actor, writer, and filmmaker. He graduated with a BFA in performing arts from the Savannah College of Art and Design. His specialities include voice acting, classical theatre, and sketch comedy.

With many thanks to Gary Marcus.