mostly using AI
An Introduction for Artists
from Google Arts & Culture Creative Lab
Lynn Cherny, Ph.D., ML Artist in Residence
Aug 2022
To view all slides, just press SPACE
"Tech Topic"
Story vs. Poem, first...
By K Allado-McDowell (and GPT-3)
NovelAI, SudoWrite
GPT3 Playground/API - registration needed
Writing / using code in colab notebooks or github repos
NovelAI shown
Collaborative story generation with a model trained on genre fiction, or game play
See, e.g., challenges in Automatic Story Gen (article)
Also lots of articles on text gen tools in general, e.g., A Robot Wrote This Entire Article. Are You Scared Yet, Human? (The Guardian)
And recent Verge piece on independent writers and SudoWrite / JasperAI
GPT-3 / GPT-2 / Relatives (autoregressive language models)
not AI models (but could be combined)!
Rules define combinations (see "sentence" above) and the output can be many variations of these words.
TECH: Tracery.js
a project of mine... written up.
A Twine game
Nested by Orteil, 2011 (thanks to Aaron A Reed for link)
"National Novel Generation Month" (2021 link)
Write a novel using generative text, post the code and output. (Mix of AI and non-AI methods used.)
Related but professional: A procedurally generated book in which no two copies are the same: Subcutanean by Aaron A Reed
Kevan Davis's entry of Moby Dick as a CYOA game (link)
First, AI and Machine Learning Poetry....
TECH: Uses an LSTM model, 2016-17, trained on 19th c. poetry
Also in Lions in Trafalgar Square exhibit for London Design Festival 2018
TECH: Uses a GPT-2 model, much improved over LSTM, trained on 19th + modern poetry
We trained on poetry reformatted as:
<keyword>: 2 lines containing <keyword> somewhere
So that the model would take an input word from a person and generate a couplet containing it.
Text input: "shuttle"
2016 link
TECH: LSTMs (older than GPT2/3)
"Reconstructions" uses a VQAE model to generate "chiasmus"
"Compasses", words generated in the phonetic space between 4 real words
See also GAC Nonsense Laboratory
TECH: Image recognition algorithms
An embedding results in giving you words (or sentences, or images) that are "similar" in some way, such as context of usage
Text
Color represents a normalized distance between the original word (random "nouny"*-words) and its next closest relative in the embedding model.
Blue: Closer, Pink: Further Away. (So, options for "sound" are going to be more different.)
word2vec model:
pre-configured poms:
Click on a colored word like "sound"... pick a replacement from it's nearest neighbors in the model....
and you edited the poem :)
Original haiku by Bashō:
With a different embedding model:
one edit:
Chinese (Classical, Modern, Lyrics...), Urdu and Hindi, Finnish, Japanese, .... (see my page here)
"AfriKI: Machine-in-the-Loop Afrikaans Poetry Generation" - trained model on fiction by South African novelist Etienne van Heerden in Afrikaans to generate poetry (LSTM)
A person chooses and orders the lines generated, in a co-collaboration. Like the word2vec edits.
Based on data, interactions, etc.
Open Street Map Haiku (not AI) by Satellite Studio - based on data in the map location
Every Thing Every Time - Naho Matsudo (Manchester, Newcastle..)
"Watch the live story of Newcastle unfold with an installation that turns city smart data into a constantly changing poem outside the Theatre Royal."
the web site is dead now, which makes me very sad.
Digitally remixed poetry, depends on the weather report (multi-award winning plus book)
Color Name Poems
by me
Ranjit Bhagnar's Pentametron, a twitter bot that got a lot of press in 2012+ for creating iambic pentameter poetry composed of found tweets.
Amelia was just fourteen and out of the orphan asylum;
at her first job—in the bindery, and yes sir, yes
ma'am, oh, so anxious to please.
She stood at the table, her blond hair hanging about
her shoulders, "knocking up" for Mary and Sadie,
the stichers ("knocking up" is counting books and
stacking them in piles to be taken away).
Lots of tools/libs online to make it, e.g.
One Hundred Thousand Billion Poems by Raymond Queneau
The book is a set of ten sonnets printed on card with each line on a separate strip. As all ten sonnets have not just the same rhyme scheme but the same rhyme sounds, any lines from a sonnet can be combined with any from the nine others, allowing for 1014 (= 100,000,000,000,000) different poems
(could be made with code easily)
...
since 2013... or earlier (link)
my rhyme-finder
Search for rhyming lines in a
corpus of poems / text
TECH: RiTa.js
input a word, get a couplet containing it, and build up a poem doing this.
Text
(my work)
TECH: ElasticSearch db
text2image and image2text, primarily
Given a text "prompt", generate an image. You can now do this in Midjourney and at OpenAI's DALL-E for payment -- or use an online code notebook (Colab) that takes a long time to run.
OpenAI's
Try yourself, in open beta now.
demo it.
Artstation
search maybe uses the CLIP algorithm in text2image models
"MAGIC" Demo site
1. Starting prompt: "A lonely traveler walked by a lake"
2. Generate image using text with a code tool
3. Caption by an image2text app
(Goto 2)
4. Write out the story.
From my Wordplay invited talk
my input text is on 0; the top caption is the prompt for 1
A grid view of the prompt used to generate each image, and the image. Each prompt is the top caption of the previous image. In a perfect AI model world, there would be no drift at all in contents/images.
link, and see thread of recent attempts with non-binary persons imagined by Midjourney and Dalle-2... which follow:
The caption spawns an image, which is classified and divided into colored segments depending on what the neural network sees, another image is created using those abstract sections as a guide and a final caption is generated.
visualizing transgender and non-binary people
AttnGAN
2017
Midjourney
now
Dalle-2
now
thread link
a very few examples, to stimulate some ideas?
Selections from IG poetry poster fossilisedflowers
Variation piece based on 47 translations into English of Dante’s opening lines of The Inferno.
English
Hi
"hi"
French
Salut
"hi"
Korean
etc.
안녕
listen: there is a good universe beside her; let's go listen: there is a good universe nearby; go listen: there's a damn good universe; gone Listen: Next door is a very good corner. So let's go Listen: The side door is a very good angle. Let's go listen: there is a damn good universe; go Please try listening. There is a very good universe around. go Please try to listen. There is a very nice corner on the side. Okay, let's go Listen: The side door is a very good corner. Let's go Listen: there is a very good environment. Let's go Listen: The neighbor has a very good angle. So let's go
English-Kurdish
English-Albanian
Kurdish-Finnish
Belarusian-German
Kurdish-Finnish
German-Albanian
Japanese-Albanian
Japanese-Kurdish
Somali-Kurdish
Haitian Cr.-Kurdish
German-Finnish
ee cummings "listen: there's a hell of a good universe next door; let's go"
universe → corner | corner → angle | next door → side door or neighbor | let's go → go...
telling immersive VR folk stories with multimodal AI conversational agents
(work by Melisa Allela)
"The winding geometry’s angle and its base ascent are respectively driven by sampling a sentence from the book and by performing a sentiment analysis on it."
"Stendhal", by Damien Seguin
Is it positive, negative, neutral; happy, sad, excited, etc....
See also, Exercises in Style, Raymond Queneau
You might then use the facts in a new form, like a game!
Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Go (recent paper at ACL 2022)
An article on AI Art in Africa from AyaData by Luba Elliott
Sigana, Tales from Lawana by Melissa Link
If you have tech stack questions, I can point you to tools or methods!