Thank you, Francis for this very interesting work, and for the responses from all. Some of you may not know me at all, so by way of introduction, I am an independent researcher/writer/feminist artist working between film, media and cultural studies on history and theory of new media in the context of post-war arts culture. I have been teaching about AI, AL, and varied artists works in these fields for a few years with great internet as a new medium (AI) comes in.
Some salient features of this analysis and the varied responses stand out for further critique.
If I am correctly reading your text, Francis, and I thank you Francis for the separation of the two modi, on the one hand there is the potential for new forms of data sets (as assets and commodities) which are “walled” and controlled - as collections or arrays - which will undoubtedly become, indeed already are being made: Some examples here: Lev Manovich’s PhotoTime projects, Trevor Paglen’s experimental arrays in which he inputs the dataset and then works with it; or experiments with MS-made Genesis Maps software used on/with the METs digital collections. Some of you may have seen or been at this event written about here: https://www.metmuseum.org/perspectives/articles/2019/2/artificial-intelligence-machine-learning-art-authorshipFrancis writes of something similar - controlled data spaces (like online communities maybe, where contributing or donating is controlled by membership?) “We are going to create separate data ecologies, which prohibit spamming the data space. These would be spaces, comparable to the no-photo-policy in clubs like Berghain or IFZ with a no-synthetics policy. While vast areas of the information space may be indeed flooded, these would be valuable zones of cultural exchange.”
On the other hand, there is the ability for AIs to remake data based on anything it is allowed to pull in, and to then feed that invention back into data pools and streams.
So the question arises about who and how data will be controlled. This question is familiar. Capitalist interests will have their reasons and methods - maybe what Brian alluded to about understanding that we have seen these mechanisms at work before…and there is the decided problem of having capitalist interests further empowered through the accelerated process of reproduction and distribution of it’s wares, especially when this uncontrolled reproduction/replication is prone to prejudices, biases, falsehood, and omissions.
Let’s see what we can foresee.
Can we foresee cultural progress continuing as a growing alienation among larger sectors of global populations, no longer able to find any useful or rational meaning in a plethora of false production? (- a great resignation from the virtual spectacle and an end to capitalism of this kind - literally it’s collapse from over extension)
Will there be regulation of AI in the net, further regulating the circulation of information?
Will those with the least access, those most vulnerable to data mining of their lives, become even more susceptible to exploitation, as they fail to understand how to navigate this new synthetic reality to their own advantage or where self-interest is a manufactured byproduct of the spectacle?
Whatever we might foresee, the bandaid of tweaking SEO for the sake of a false surface of BIPOC to add in is lame (Francis’ example about a few more pictures of BIPOC being worked in)
Molly
On Dec 23, 2022, at 12:48 PM, Luke Munn <luke.munn@gmail.com> wrote:
Hey Francis,
Thanks for your response. Just briefly I think there was some misframing of my response. It was not meant as a takedown or critique of your text, but rather a more general response to this idea tabled in the opening line, For the last time in human history the cultural-data space has not been contaminated.
This is an idea I had seen a few times from others the same day. 'The internet is now forever contaminated with images made by AI,” Mike Cook, an AI researcher at King’s College London' ..... 'How AI-Generated Text Is Poisoning the Internet' etc.
I was kicking around this idea and seeing what it implied, hence delving into this idea of post-truth authenticity etc which your text doesn't talk about.
I know you get this stuff, and can see issues with some of these assumptions, which is precisely why I mentioned your dnb example as culture of remixing.
Nevertheless I did think your original piece (and most people's work including mine) can be strengthened by thinking longer term / historically, as the presentist framing of tech is often very powerful. That was my only real feedback - but i can see how this more general 'essay as jumping off point' discussion could be misread, so apologies for that.
-L
Dear Luke, dear All
Interesting essay Francis, and always appreciate
Brian's thoughtful comments. I think the historical angle Brian
is pointing towards is important as a way to push against the
claims of AI models as somehow entirely new or revolutionary.
In particular, I want to push back against this idea that
this is the last 'pure' cultural snapshot available to AI
models, that future harvesting will be 'tainted' by automated
content.
At no point did I allude to the 'pureness' of a cultural
snapshot, as you suggest. Why should I? I was discussing this from
a material perspective, where data for training diffusion models
becomes the statistical material to inform these models. This data
has never been 'pure'. I used the distinction of
uncontaminated/contaminated to show the difference between a
training process for machine learning which builds on an snapshot,
that is still uncontaminated by the outputs of CLIP or GPT and one
which includes generated text and images using this techique on a
large scale.
It is obvious, but maybe I should have made it more clear, that
the training data in itself is already far from pure. Honestly I'm
a bit shocked, you would suggest I'd come up with a nostalgic
argument about purity.
Francis' examples of hip hop and dnb culture, with sampling
at their heart, already starts to point to the problems with
this statement. Culture has always been a project of cutting
and splicing, appropriating, transforming, and remaking
existing material. It's funny that AI commentators like Gary
Marcus talk about GPT-3 as the 'king of pastiche'. Pastiche is
what culture does. Indeed, we have whole genres (the romance
novel, the murder mystery, etc) that are about reproducing
certain elements in slightly different permutations, over and
over again.
Maybe it is no coincidence that I included exactly this example.
Unspoken in this claim of machines 'tainting' or
'corrupting' culture is the idea of authenticity.
I didn't claim 'tainting' or 'corrupting' culture, not even
unspoken. Who am I to argue against the productive forces?
It really reminds me of the moral panic surrounding
algorithmic news and platform-driven disinformation, where
pundits lamented the shift from truth to 'post-truth.' This
is not to suggest that misinformation is not an issue, nor
that veracity doesn't matter (i.e. Rohingya and Facebook). But
the premise of some halcyon age of truth prior to the digital
needs to get wrecked.
I agree. Only, I never equaled 'uncontaminated' to a "truth prior to
the digital", I equaled it to a snapshot that doesn't contain
material created by transformer models.
Yes, Large language models and other AI technologies do
introduce new conditions, generating truth claims rapidly and
at scale. But rather than hand-wringing about 'fake news,'
it's more productive to see how they splice together several
truth theories (coherence, consensus, social construction,
etc) into new formations.
I was more interested in two points:
1.) Subversion: What I called in my original text the 'data
space' (created through cultural snapshots as suggested by Eva
Cetinic) is an already biased, largely uncurated information space
where image data and language data are scaped and then
mathemtically-statistically merged together. The focus point here
is the sheer scale on which this happens. GPT-3 and CLIP are
techniques that both build on massive datascraping (compared for
instance to GANs) so that it is only possible for well funded
organizations such as Open-AI or LAION to build these datasets.
This dataspace could be spammed a) if you want to subvert it and
b) if you'd want to advertise. The spam would need to be on a
large scale in order to influence the next (contaminated)
iteration of a cultural snapshot. In that sense only I used the
un/contaminated distinction.
2). In response to Brian I evoked a scenario that builds on what
we already experience when it comes to information spamming. We
all know, that mis-information is a social and _not_ a machinic
function. Maybe I should have made this more clear (I simply
assumed it). I ignored Brians comment on the decline of culture,
whatever this would mean, and could have been more precise in this
regards. I don't assume culture declines. Beyond this, there have
been discussions about deepfakes for instance and we saw that
deepfakes are not needed at all to create mis-information, when
one can just cut any video using standard video editing practices
towards 'make-believe'. I wasn't 'hand-wringing' about fake news,
in my comment to Brian, instead I was quoting Langlois with the
concept of 'real fakes'.
Further I'm suggesting that CLIP and GPT make it more easy to
automate large scale spamming, making online communities
uninhabitable or moderation more difficult. Maybe I'm
overestimating the effect. We can already observe GPT-3 automated
comments appearing on twitter or the ban of GPTChat posts on
Stackoverflow
(https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned),
the latter already being a Berghain-no-photo-policy.
Finally, I'm interested in the question of bias and
representation, and how a cultural snapshot, that builds on a
biased dataset (and no, I'm not saying there are unbiased datasets
at all), can further deepen these biases with each future
interation, when these bias get statistically reproduced through
'AI' and the become basis for the next dataset.
best
Francis
Hi Brian,
While some may argue that generated text and
images will save time and money for businesses, a
data ecological view immediately recognizes a
major problem: AI feeds into AI. To rephrase it:
statistical computing feeds into statistical
computing. In using these models and publishing
the results online we are beginning to create a
loop of prompts and results, with the results
being fed into the next iteration of the cultural
snapshots. That’s why I call the early cultural
snapshots still uncontaminated, and I expect the
next iterations of cultural snapshots will be
contaminated.
Francis, thanks for your work, it's always
totally interesting.
Your argumentation is impeccable and one can
easily see how positive feedback loops will form
around elements of AI-generated (or perhaps
"recombined") images. I agree, this will become
untenable, though I'd be interested in your ideas as
to why. What kind of effects do you foresee, both on
the level of the images themselves and their
reception?
Foresight is a difficult field, as most estimates can
extrapolate maximum 7 year into the future and there are a
lot of independent factors (such as e.g. OpenAI, the
producer of CLIP could go bankrupt etc.).
It's worth considering that similar loops have
been in place for decades, in the area of market
research, product design and advertising. Now, all
of neoclassical economics is based on the concept of
"consumer preferences," and discovering what
consumers prefer is the official justification for
market research; but it's clear that advertising has
attempted, and in many cases succeeded, in shaping
those preferences over generations. The preferences
that people express today are, at least in part,
artifacts of past advertising campaigns. Product
design in the present reflects the influence of
earlier products and associated advertising.
That's an great and interesting argument. Because it
plays into the cultural snapshot idea.
Obviously Language wise, people already use translation
tools, such as Deepl and translate Text from German to
English and back to German in order to profit off the
"clarity" and "orthographic correction" brought by the
statistical analysis that feeds into the translator and
seems to straighten the German text. We see the same stuff
appearing for products like text editors and thus widely
employed for cultural production. That's one example.
Automated forum posts using GPT-3, for instance on Reddit
are another, because we know that the CLIP Model also
partly build on Reddit posts.
Another example is images generated using diffusion
models and prompts building on cultural snapshots and
being used as _cheap_ illustrations for editorial
products, feeding off stock photography and to a certain
extend replacing stock photography. This is more or less
an economic motivation with cultural consequences. The
question is what changes, when there is not sufficiently
'original' stock photography circulating, but the majority
is syntheticly generated? Maybe others want to join in, to
speculate about it.
We could further look into 1980s HipHop or 1990s Drum'n
Bass sample culture, which for instance took (and some
argue: stole) one particular sound break, the Amen Break,
from an obscure 1969 Soul music record by The Winston
Brothers and build a whole cultural genre from it. Cf. https://en.wikipedia.org/wiki/Amen_break
Here the sample was refined over time, with generations of
musicians cleaning the sample (compression, frequencies,
deverbing, etc.) and providing many variations of it, then
reusing it, because later generation did not build on the
original sample, but on the published versions of it.
We can maybe distinguish two modi operandi where a) "the
cultural snapshot" is understood as an automated feedback
loop, operating on a large scale, mainly through automated
scraping and publication of the derivates of data,
amplifying the already most visible representations of
culture and b) "the cultural snapshot" is a feedback loop
with many creative human interventions, be it through
curatorial selection, prompt engineering or intended data
manipulation.
Blade Runner vividly demonstrated this cultural
condition in the early 1980s, through the figure of
the replicants with their implanted memories.
I dont know if I get your point. I'd always say that Blade
Runner is a cultural imaginary, one of the many phantasms
about the machinisation of humans since at least 1900 if not
earlier, and that's an entirely different discussion then. I
would avoid this as an metaphor.
The intensely targeted production of postmodern
culture ensued, and has been carried on since then
with the increasingly granular market research of
surveillance capitalism, where the calculation of
statistically probable behavior becomes a good deal
more precise. The effect across the neoliberal
period has been, not increasing standardization or
authoritarian control, but instead, the rationalized
proliferation of customizable products, whose
patterns of use and modification, however divergent
or "deviant" they may be, are then fed back into the
design process. Not only the "quality of the image"
seems to degrade in this process. Instead, culture
in general seems to degrade, even though it also
becomes more inclusive and more diverse at the same
time.
When looking for a plausible scenario regarding synthetic
text and synthetic images, Steve Bannons “The real
opposition is the media. And the way to deal with them is
to flood the zone with shit.” is sadly a good candidate.
This ties in with what Ganaele Langlois posits:
„Therefore: communicative fascism posts that what is
real is the opposite of social justice, and we now see
the armies of ‚Social Injustice Warriors‘ as Sarah
Sharma (2019) calls them, busy typing away at their
keyboards to defend the rights to keep their fear of
Others unchallenged and to protect their bigotry,
misogyny, and racism from being debunked as inept
constructions of themselves“ Langlois 2021:3
„The first aspect of this new communicative fascism is
related to what can be called ‚real fakes_ that is to
say, the construction of a fictional and alternative
reality where the paranoid position of fear and rage can
find some validation … Real fakes are about what reality
ought to be: they are virtual backgrounds on which
fascists can find their validity and raising’être.“
Langlois 2021:3f
So this is to be expected both for political or consumer
marketing purposes.
AI is poised to do a lot of things - but one of
them is to further accelerate the continual remaking
of generational preferences for the needs of
capitalist marketing. Do you think that's right,
Francis?
That's one possible reading. I would insist, to not use
an active verb with AI however, rephrasing your point
towards "AI may be used for a lot of things". Better even
replace 'AI' with the term 'statistical computation'.
Currently I would read 'AI' as a mixture of imaginations
and phantasms about automation, of which some may become
true – just in another way from what was expected or
promoted. For certain, the inner logics of capital
circulation command to deploy statistical computation to
replace living, human labor. We already see how the job
description of translators changes towards an
human–statistical_computation entanglement and how the
repetetive parts of the illustrator job, like coloring get
automated away and put people out of jobs and it is
plausible to expect the consolidation of jobs like photo
editor, news editor, author with prompt-engineering. Since
we are concentrating on the cultural sphere here, I'll
limit the examples to this field. Human Labor in
production, logistics, care labor would need their own
thoughts.
What other consequences do you see? And above
all, what to do in the face of a seemingly
inevitable trend?
We are going to create separate data ecologies, which
prohibit spamming the data space. These would be spaces,
comparable to the no-photo-policy in clubs like Berghain
or IFZ with a no-synthetics policy. While vast areas of
the information space may be indeed flooded, these would
be valuable zones of cultural exchange. (The answer would
be much longer indeed, but we're not writing a book here).
--
Researcher at Training The Archive, HMKV Dortmund
Artistic Practice http://www.irmielin.org
Ph.D. at Bauhaus University Weimar http://databasecultures.irmielin.org
Daily Tweets https://twitter.com/databaseculture
Peter and Irene Ludwig guest professorship at the Hungarian University of Fine Arts in Budapest 2022/23
# <nettime> is a moderated mailing list for net
criticism,
# collaborative text filtering and cultural politics of the
nets
# more info: http://mx.kein.org/mailman/listinfo/nettime-l
# archive: http://www.nettime.org
contact: nettime@kein.org
# @nettime_bot tweets mail w/ sender unless #ANON is in
Subject:
--
Researcher at Training The Archive, HMKV Dortmund
Artistic Practice http://www.irmielin.org
Ph.D. at Bauhaus University Weimar http://databasecultures.irmielin.org
Daily Tweets https://twitter.com/databaseculture
Peter and Irene Ludwig guest professorship at the Hungarian University of Fine Arts in Budapest 2022/23
# <nettime> is a moderated mailing list for net criticism,
# collaborative text filtering and cultural politics of the nets
# more info: http://mx.kein.org/mailman/listinfo/nettime-l
# archive: http://www.nettime.org contact: nettime@kein.org
# @nettime_bot tweets mail w/ sender unless #ANON is in Subject:
|