
A.I. Anne: neurodiverse Inclusion in creative collaborations between humans and A.I. entities
-
BFA: Artist, director, researcher
-
Ph. D. candidate, 2017 - 2021: Composer and music technologist
-
the New Museum’s cultural incubator, NYC: Institution affiliation
The creative and performing arts have often been effective in communicating the range of challenges faced by autistic individuals, but the arts have not been used as a tool to facilitate and drive research, communicate rigorous scientific inquiry, and spotlight the potential of creative collaborations with the autistic community.
Janet Biggs, Richard Savery and Mary Esther Carter have presented live and live-streamed performances that feature “A.I. Anne,” a machine learning entity that exhibits a range of responses based on Biggs’ autistic aunt’s behaviors and communication methods. The performances test novel ways of identifying and considering neural diversity and equity. A.I. Anne was created to drive multifactorial neurodevelopmental diverse inclusion in building and expanding technology.
Our team has been training and patterning an AI entity on one autistic individual’s phenotypic data to produce a series of innovative performances that explore the inclusion of neural diversity in creative collaborations with technology. The performances combine improvisation, vocalization and physical movement interwoven with storytelling.
Our research and performances drive partnerships meant to improve and enable greater public understanding of autism spectrum disorder (ASD). Prompted by memories of lived experiences with her aunt, artist Janet Biggs asked composer and music technologist Richard Savery to create an A.I. algorithm that includes some of her aunt’s phenotypic traits. Biggs’ aunt vocalized and hummed, but never communicated through spoken language due to apraxia. A.I. Anne can vocalize, communicate emotions and respond to human emotions, but does not use traditional spoken language. A.I. Anne can “hear” and respond in real time, allowing for interactive exchanges that range from utterances to sophisticated tonal duets. These exchanges elicit deep emotional responses from both performers and audience members, while acting as a feedback loop for future research and generations.
The current generations use state of the art convolutional conditional variational autoencoders to react dynamically based on emotional responses from singer Mary Esther Carter. These emotional interactions utilize a narrow form of emotional exchange, limited by a narrow range of data input. Increased datasets will enable Biggs and Savery to leverage more diversity in response type and drive new understandings and performative outcomes.
Biggs and Savery have expanded their collaborative network to include experience designer/creative director, Kate Machtiger, who is autistic, and improvisational violinists Earl Maneein and Mylez Gittens.
In addition to musical emotion interactions, widening our datasets will expand A.I. Anne’s language processing and understanding. Currently A.I. Anne is able to listen to text and respond musically based on the semantic meaning and vocal prosody, but has a limited range of responses and a confined understanding of input. Preliminary work has been completed focusing on A.I. Anne generating language, however this is based only on a generalized, non-specific text data set.
Our project will not only facilitate broader avenues of communication and representation, but will also lay the groundwork for future inclusion of neurodiversity in building and expanding technology while promoting innovative research and understanding.
Documentation of performances: https://vimeo.com/488543925
References:
Shimon Sings - Robotic Musicianship Finds its Voice
Richard Savery, Lisa Zahray, Gil Weinberg
Handbook of Artificial Intelligence and Music, Springer, 2020
A ConvNet for Ethical Robotic Musical Generation and Interaction
Richard Savery, Lisa Zahray, Gil Weinberg
Artificial Intelligence & Creative Music Practice, Routledge, 2021
Emotional Musical Prosody for the Enhancement of Trust in Robotic Arm Communication
Richard Savery, Lisa Zahray, Gil Weinberg
Trust, Acceptance and Social Cues in Human-Robot Interaction, Ro-MAN 2020
Emotional Musical Prosody: Validated Vocal Dataset for Human Robot Interaction
Richard Savery, Lisa Zahray, Gil Weinberg
2020 Joint Conference on AI Music Creativity
ABstract
Meet the Team
-
A.I. Anne
A.I. Anne is a machine learning entity developed by Richard Savery and Janet Biggs. A.I. Anne was, trained on Mary Esther Carter’s voice and patterned on Janet Biggs’ aunt. Biggs’ aunt was autistic and non-speaking due to apraxia. The entity A.I. Anne, is also non-speaking, but able to vocalize and hum. A.I. Anne can improvise vocally, creating a duet between human and technology. Using deep learning and semantic knowledge A.I. Anne can respond to emotion and with emotion.
-
Jason Barnes
Jason Barnes is a drummer from Georgia, USA. He was electrocuted in a workplace accident in 2012, and as a result he lost his right arm. Despite this setback, in 2013 he was accepted into the Atlanta Institute of Music. In 2014, a team from the Georgia Institute of Technology led by Professor Gil Weinberg fitted Jason with a robotic arm. He has since toured internationally, most recently performing at Robotronica at the Queensland University of Technology.
-
Mary Esther Carter
Mary Esther Carter is a singer-songwriter as well as a dancer, and actor. She has cultivated her voice as an artist in New York City for the past 10 years. Performing in dance companies, performance art, immersive theater, and film as well as creating her own music and choreography. Her debut EP "DOULA” was released in the summer of 2019.
-
Richard Savery
Richard Savery is a developer of artificial intelligence and robotics, using music and creativity as a medium to program better interactions, understandings and models. He is currently a PhD student at Georgia Institute of Technology working with the Robotic Musicianship lab led by Gil Weinberg. He has composed and orchestrated many video games, films and ads including Fast Four featuring Roger Federer and has worked for 15 years as a professional saxophonist, clarinetist and flutist. His current projects focus on computer and robot generated scores for visual media, and a NVIDIA funded deep learning application of the NVIDIA Jetson in a musical companion robot.
-
Brian Reynolds
Brian Reynolds is a double amputee distance runner. At the age of 4, Brian contracted a deadly form of meningitis called meningococcemia; making him a bilateral below the knee amputee. Sports have been a part of Brian's entire life. While in college, he was a competitive powerlifter, and after college, he started running! The strides he has made in his running career have been huge, specifically with distance running. In 2016, Brian ran under 1 hour and 30 mins in the Philadelphia Half Marathon, which may make him the first bilateral amputee to have ever accomplished this feat. His goal? Run a sub 3 hour marathon. Finally, and most importantly, he wants to help other amputees realize that nothing is impossible when you just try!
-
Earl Maneein
Earl Maneein was born and raised in Queens, NYC and began studying classical violin at age four. He received a Bachelor of Music Degree from Queens College and a Master of Music Degree for the Mannes College of Music in New York City. As a composer, Earl has received commissions from Rachel Barton Pine, Tito Muñoz, Masumi Rostad, The Brooklyn Conservatory of Music, The Dance Theater of Harlem, The Francesca Harper Dance Company and Zentripetal Duo. Earl regularly composes for his two projects, the experimental grindcore/improvisatory/jazz/metal duo Black Heart Sutra, and his music/hardcore crossover string quartet SEVEN)SUNS. Earl tours as a featured play of the Vitamin String Quartet and has recorded and played with artists including Albert Hammond Jr. of the Strokes, Aretha Franklin, Alicia Keys, Avraham Fried, Florence+The Machine, Jay-Z, Mordechai Ben David, Paul Weller of The Jam, Rhianna, The Roots and Sean Lennon.
-
Mylez Gittens
Born & raised in Barbados, Mylez Gittens moved to New York three years ago. He has been playing the violin for sixteen years, eleven being classical study and the last five have been a mixture of jazz, funk, hip-hop and everything in-between.
-
Kate Machtiger
Kate Machtiger is the founder of Extra Terrestrial Studio, where she designs and builds “deep space” — surreal places that build connection through exploration and play.She was formerly Design Director at Assignment Studios, where she helped clients bring their brand visions to life in the form of retail spaces, pop ups, and events. In 2016, Kate was selected to the Forbes 30 Under 30 for her role in developing emerging technology programs with the federal government. . She began her career as an investment banker at Goldman Sachs. Kate holds a B.S.E. in Civil Engineering from Princeton University and an M.B.A. from Said Business School at the University of Oxford.
-
Janet Biggs
Janet Biggs is an interdisciplinary artist known for her immersive work in video, film and performance. Biggs’ work focuses on individuals in extreme landscapes or situations, navigating the territory between art, science and technology. She has collaborated with neuroscientists, Arctic explorers, aerospace engineers, astrophysicists and a robot. Last year, Biggs sent a project up to the International Space Station as part of MIT Media Lab’s Space Exploration Initiative. She has had solo exhibitions and screenings at the Museos de Tenerife, Neuberger Museum of Art, SCAD Museum of Art, Blaffer Art Museum, Musée d'art contemporain de Montréal, Hirshhorn Museum and Sculpture Garden. Her work has been supported by the John Simon Guggenheim Foundation, the New York State Council on the Arts and the National Endowment for the Arts. Reviews of her work have appeared in the New York Times, New Yorker, ArtForum, ARTNews, Art in America, and many others. She works with Cristin Tierney Gallery, CONNERSMITH, Galerie Analix Forever and Hyphen-Hub.