mostly using AI
An Introduction for Artists
from Google Arts & Culture Creative Lab
Lynn Cherny, Ph.D., ML Artist in Residence
To view all slides, just press SPACE
Story vs. Poem, first...
By K Allado-McDowell (and GPT-3)
Collaborative story generation with a model trained on genre fiction, or game play
See, e.g., challenges in Automatic Story Gen (article)
Also lots of articles on text gen tools in general, e.g., A Robot Wrote This Entire Article. Are You Scared Yet, Human? (The Guardian)
And recent Verge piece on independent writers and SudoWrite / JasperAI
GPT-3 / GPT-2 / Relatives (autoregressive language models)
not AI models (but could be combined)!
Rules define combinations (see "sentence" above) and the output can be many variations of these words.
a project of mine... written up.
A Twine game
"National Novel Generation Month" (2021 link)
Write a novel using generative text, post the code and output. (Mix of AI and non-AI methods used.)
Related but professional: A procedurally generated book in which no two copies are the same: Subcutanean by Aaron A Reed
Kevan Davis's entry of Moby Dick as a CYOA game (link)
First, AI and Machine Learning Poetry....
TECH: Uses an LSTM model, 2016-17, trained on 19th c. poetry
Also in Lions in Trafalgar Square exhibit for London Design Festival 2018
TECH: Uses a GPT-2 model, much improved over LSTM, trained on 19th + modern poetry
We trained on poetry reformatted as:
<keyword>: 2 lines containing <keyword> somewhere
So that the model would take an input word from a person and generate a couplet containing it.
Text input: "shuttle"
TECH: LSTMs (older than GPT2/3)
the code no longer runs
but it could be replicated with
"Reconstructions" uses a VQAE model to generate "chiasmus"
"Compasses", words generated in the phonetic space between 4 real words
See also GAC Nonsense Laboratory
TECH: Image recognition algorithms
An embedding results in giving you words (or sentences, or images) that are "similar" in some way, such as context of usage
Color represents a normalized distance between the original word (random "nouny"*-words) and its next closest relative in the embedding model.
Blue: Closer, Pink: Further Away. (So, options for "sound" are going to be more different.)
Click on a colored word like "sound"... pick a replacement from it's nearest neighbors in the model....
and you edited the poem :)
Original haiku by Bashō:
With a different embedding model:
Chinese (Classical, Modern, Lyrics...), Urdu and Hindi, Finnish, Japanese, .... (see my page here)
"AfriKI: Machine-in-the-Loop Afrikaans Poetry Generation" - trained model on fiction by South African novelist Etienne van Heerden in Afrikaans to generate poetry (LSTM)
A person chooses and orders the lines generated, in a co-collaboration. Like the word2vec edits.
Based on data, interactions, etc.
Open Street Map Haiku (not AI) by Satellite Studio - based on data in the map location
Every Thing Every Time - Naho Matsudo (Manchester, Newcastle..)
"Watch the live story of Newcastle unfold with an installation that turns city smart data into a constantly changing poem outside the Theatre Royal."
the web site is dead now, which makes me very sad.
Digitally remixed poetry, depends on the weather report (multi-award winning plus book)
Color Name Poems
Ranjit Bhagnar's Pentametron, a twitter bot that got a lot of press in 2012+ for creating iambic pentameter poetry composed of found tweets.
Lots of tools/libs online to make it, e.g.
One Hundred Thousand Billion Poems by Raymond Queneau
(could be made with code easily)
since 2013... or earlier (link)
Search for rhyming lines in a
corpus of poems / text
input a word, get a couplet containing it, and build up a poem doing this.
TECH: ElasticSearch db
text2image and image2text, primarily
Try yourself, in open beta now.
search maybe uses the CLIP algorithm in text2image models
"MAGIC" Demo site
1. Starting prompt: "A lonely traveler walked by a lake"
2. Generate image using text with a code tool
3. Caption by an image2text app
4. Write out the story.
From my Wordplay invited talk
my input text is on 0; the top caption is the prompt for 1
A grid view of the prompt used to generate each image, and the image. Each prompt is the top caption of the previous image. In a perfect AI model world, there would be no drift at all in contents/images.
link, and see thread of recent attempts with non-binary persons imagined by Midjourney and Dalle-2... which follow:
The caption spawns an image, which is classified and divided into colored segments depending on what the neural network sees, another image is created using those abstract sections as a guide and a final caption is generated.
visualizing transgender and non-binary people
a very few examples, to stimulate some ideas?
Selections from IG poetry poster fossilisedflowers
Variation piece based on 47 translations into English of Dante’s opening lines of The Inferno.
listen: there is a good universe beside her; let's go listen: there is a good universe nearby; go listen: there's a damn good universe; gone Listen: Next door is a very good corner. So let's go Listen: The side door is a very good angle. Let's go listen: there is a damn good universe; go Please try listening. There is a very good universe around. go Please try to listen. There is a very nice corner on the side. Okay, let's go Listen: The side door is a very good corner. Let's go Listen: there is a very good environment. Let's go Listen: The neighbor has a very good angle. So let's go
English-Kurdish English-Albanian Kurdish-Finnish Belarusian-German Kurdish-Finnish German-Albanian Japanese-Albanian Japanese-Kurdish Somali-Kurdish Haitian Cr.-Kurdish German-Finnish
ee cummings "listen: there's a hell of a good universe next door; let's go"
universe → corner | corner → angle | next door → side door or neighbor | let's go → go...
telling immersive VR folk stories with multimodal AI conversational agents
(work by Melisa Allela)
"The winding geometry’s angle and its base ascent are respectively driven by sampling a sentence from the book and by performing a sentiment analysis on it."
"Stendhal", by Damien Seguin
Is it positive, negative, neutral; happy, sad, excited, etc....
See also, Exercises in Style, Raymond Queneau
You might then use the facts in a new form, like a game!
Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Go (recent paper at ACL 2022)
An article on AI Art in Africa from AyaData by Luba Elliott
Sigana, Tales from Lawana by Melissa Link
If you have tech stack questions, I can point you to tools or methods!