Tag Archive for Stable Diffusion

One Minute Mistake – The Appliance of Science – A Short Film

I’m pleased to finally share a short film that, some time ago, Director Martin Cooper and I were commissioned to make for inclusion in a project called “One Minute Mistake”.
“The Appliance of Science” is what we came up with.

Somewhat of a departure from our usual documentary projects, it was a lot of fun to make. Despite my reluctance to appear onscreen!
Filmed on a green screen with all the backgrounds of the ‘laboratory’ generated in Stable Diffusion (AI Image generator).
The editing and compositing process, done entirely in After Effects, was a very interesting challenge.

We hope you enjoy it.

Staring – Martin Cooper, Adrian “Uchujin” Storey & Steve Gorton
Director – Martin Cooper
Director Of Photography, Editor & Digital Compositor – Adrian “Uchujin” Storey
Hand-drawn animations – Parham Ghalamdhar
Opening Title Card – Michael Knaggs
Music – Ethan Uslan (Used under a CC 4.0 Licence)

Ed Boxall – Dance Beneath A Diamond Sky (Music Video)

Ed Boxall an artist/musician and dear old friend, recently asked if I’d be interested in making a video for a track on his upcoming “Wild Ones” album release.
Back in the (distant) day, Ed and I had a band called “Sunshine Head” and I have always loved his songwriting and art work, so it was a swift and simple ‘yes’.
The track he chose was titled “Dance Beneath A Diamond Sky”, the Bob Dylan reference immediately endeared me to it.

Ed was happy for me to have free rein to do whatever I wanted and after listening to the track a few times an idea presented itself.

I took some existing footage I had shot on a beautiful old Russian Helios 40 85mm lens in a nearby wood and a whole stack of illustrations Ed had sent me.

I had been planning on experimenting more with AI image generators after my initial experiments a year ago as much had evolved in the space. This seemed like the perfect project on which to play.

I fed both the footage and the drawings into the Deforum extension for Stable Diffusion, using prompts generated from Clip Interrogator. This produced some interesting shifting animations of the static footage and dreamlike hallucinations of Ed’s original drawings. I also generated some additional elements to layer into the video using variations on the song’s title as prompts.
Everything was blended together in After Effects and a grade applied to complete the look.

I wanted to make something that didn’t look like it had been generated by AI, something that was using it more as a tool to play with my original footage and Ed’s original drawings. Not only was it more satisfying to use these tools to create based on work done by our own hands but (for me at least) it dismissed a lot of the ethical hand wringing that rightfully accompanies a lot of things being produced by it.

We hope you like it.

Listen to the full album “Wild Ones”

See and order the “Wild Ones” book and Ed’s other work on his website www.edboxall.co.uk

Peripheral Neuropathy – A Short Film

Allow me to present my latest short film “Peripheral Neuropathy”

I started thinking about this idea about 8 months ago and even filmed a few bits to check and see if it was going to work.
I had drawn up a storyboard and planned out 80% of it but had hit a brick wall with how to end it.
It got shelved but kept bubbling away in the back of my mind every time I looked at the crudely hand drawn storyboard taped to my speaker stand.

Then text to image AI models exploded into my world, adding an entirely new tool to the toolbox.
Quite simple film making and some finessed After Effects skills had gotten me most of the way, could I leverage this new tool to finally get this finished?

I’d already been using Stable Diffusion on another short Directed by Martin Cooper to generate backgrounds for some green screen footage we had shot and was increasingly excited about the possible uses in other work.

There ended up being very little actual Stable Diffusion stuff in this film, but what has ended up in there made it possible to finish the film, something I was beginning to worry might not happen. It had helped in other ways by inspiring experimentation and ultimately provided a much needed creative spark, so deserves more credit than the amount of it’s inclusion would suggest.

I used the Deforum animation plugin in the Automatic web UI of Stable diffusion to create a low poly moving “field” to mirror the actual field seen in the piece.
I also used Stable diffusion to generate an image of an aged me from a video frame, I attempted to use EbSynth to map the image back onto the video but was not really getting anything I was happy with. I ended up using FILM to map the image onto the footage, which gave a much more satisfying result.

I knew it needed more than the natural sounds present to bring it all together and asked my long time friend and musical talent Danny Griffin to see what he could do, the only real brief being I didn’t want it to end up being like a music video. I think he absolutely nailed the soundtrack, adding so much more to the piece.

You can draw your own conclusions on what it’s about or why it has the title it does.

I’m very pleased with how it’s turned out. I hope you like it.

*Note to self: Figure out the whole story before starting something in future otherwise it just gnaws away at you :)*

Experiments in AI Art – Part 2 – Animations & Ethics

“magical wet-plate portrait photography, man crying, flowers, scratches to magical wet-plate portrait photography, woman crying, flowers, scratched” Stable Diffusion with Deforum Animation

After my initial foray into AI art and discovering Stable Diffusion I was totally hooked but also trying to find ways to use the technology to make something interesting to me.
It had become very clear it was easy to generate amazing looking images, particularly if you were prepared to put in some time learning the syntax and craft of prompt writing to sculpt the outputs into ever more beautiful/weird creations.

I needed a direction or at least some idea beyond just single beautiful images.
Then I discovered Deforum.

Put simply Deforum allows you to make animations with Stable Diffusion generated images. You can morph between prompts, add zooms or rotations and so much more.
It is, for someone who has spent most of their adult life playing with moving images in one form or another, insanely addictive and inspiring.

The first animation I created after installing and figuring out how to use it was the one at the head of this post.
As was becoming a daily occurrence whilst playing with this tech, I was blown away.
I just sat there gawping at the screen, once again struck by the insane possibilities of this cutting edge tech I could run at home on my slightly aging desktop PC – for FREE!

The next animation I made (below) I played with a little more, taking the output from Deforum into After Effects, tweaking and creating the reverse half of the loop.
Now we were getting somewhere! I could already think of a thousand use cases for this in my own work.
It also took me down a rabbit hole of ethical considerations……..

“0”: “an old man on a motorbike by mark rothko, Ewald Rübsamen”,
“20”: “a beautiful tree, trending on Artstation, by mark rothko, Ewald Rübsamen”,
“40”: “water by mark rothko, Ewald Rübsamen –neg photo, realistic”,
“60”: “the moon, trending on Artstation, by mark rothko, Ewald Rübsamen”
Stable Diffusion with Deforum Animation

You’ll notice in the prompts the names of two artists. Stable diffusion knows what those artists work looks like and so is able to generate images in their style.
I’m a huge fan of Mark Rothko, his room at the Tate Modern in London is one of my favorite places on earth. I chose to add his name as I wanted something with big blocks of colour, but not bright and garish.
Ewald Rübsamen, however, I had never heard of.
The “insert random artist” button in the Stable Diffusion web-ui added it for me.

Something felt, odd, wrong even about using a model trained on these artists work, without their permission, to do what I was doing.

But was it?

(Interestingly, I have discovered that the LAION model used to train Stable Diffusion contains at least 3 of my photographs (LINK) scraped from this blog all clearly marked as “©Uchujin-AdrianStorey” so it could be argued I have skin in this game)

The ethics of these models is a debate that is raging across the internet (Search it)
There are far more intelligent and erudite people than me discussing these issues and I would urge you to do some reading if this interests you at all.
I think it needs some serious thought by anyone looking to interact with these technologies.

My current feelings on it (which may change) are this:

– “Good artists borrow, great artists steal” a quote attributed to Picasso.
I know as a photographer and videographer that I have been heavily influenced by work I love, I have blatantly “stolen” ideas or techniques from others and integrated them into my own work.
Surely all artists do this? even if it’s subconsciously.
No one who is a painter/photographer/designer/musician has never seen/heard another painting/photograph/graphic design/piece of music!
And those works produced by others have influenced us all.
Isn’t this some form of the same?

-There seems to me to be a very real difference between playing with this technology to generate images/animations for social media (or this blog) with attribution if artists names were explicitly used and attempting to pass the work off as entirely our own, without reference to how it was made, and trying to monetise it.
One of those feels like artistic exploration the other feels blatantly wrong.

-Every time a new technology comes along it shakes up the status quo.
I’m old enough to remember when Digital SLR Cameras exploded into the world. Traditional film photographers were not happy.
The Canon 5D mk I was the single most important thing that made me start making films. Because suddenly I could. The technology to do so had been democratised in a way it had never been before.
Let’s not even get started on CDs, Photoshop, Smartphones or The Internet for fuck sake!
We will have to address how these models are trained, how attribution is given and have open conversations about what this all means, but the genie is well and truly out of the bottle and sat on your sofa with his/her feet up sipping a nice glass of red and has no intention of going back in.

It really does feel like we are witnessing something that is set to change the world in ways we cannot even begin to imagine yet, and much like “the internet” that will equally inspire, amaze and bring us together with one hand and disgust, oppress and divide us with the other.

We really should be used to it by now.

Part 3 of this mini blog series will deal with my narcissism :)