Photoshop AI Generative Fill – A Benefit for Photographers?

In May 2023, Adobe release a Beta version¹ of Photoshop with a feature called Generative Fill. Basically, it is an artificial intelligence (AI) based tool which allows you to alter photos with a minimum of effort.

Almost immediately upon the release, there were concerns voiced that the tool was going to invalidate legitimate photography since it is now possible to create an entire ‘photograph’ entirely from AI generated images. I was one of those concerned.

And, in fact, that’s how it was initially promoted – at least on a limited scale.

One of the first demonstrations I saw was an instance where something which did not appear in the original photo in any way was added to a photo. Much of the hype focused on the fact that the AI could not only generate portions of an image that weren’t present initially, but that it could do a credible job of matching light and shadow with the existing photo.

It Wasn’t There – But Now It Is

This is a re-creation of the first time I saw Generative Fill demonstrated. The presentation took a photo which contained a meadow and then inserted a flock of sheep, which were nowhere to be found in the original photo. Since I don’t have the photo that was used for the demonstration, I am using one of my own.

This is a shot from a park near my house. It has a nice grassy area which is partially in sunlight and partially in shadow.

But let’s say that I decided that there needed to be a flock of sheep grazing in this public park. (Don’t ask me why anyone would decide that, but that was the concept in the original demonstration.)

So in the new Photoshop (still in Beta as this is written), I simply circle an area with a selection tool. Then in the Generative Fill dialog box, I type “flock of sheep grazing.” I click on the Generate button and wait 12-15 seconds.

And voila! There are sheep grazing in my local public park. And yes, there is a semblance of realism. The sheep in the shadows are properly shadowed, including the one in front center which is partially in sunlight and partially in shadow.

Also, the light on the ‘sheep’ is coming from the correct direction to match the lighting of the original photograph.

Photograph or Artistic Creation?

The question then becomes, “is this still a ‘photograph’ or has it become an artistic creation?”

Certainly, most photographers, except those engaged in purely journalistic photography, edit their photos. In addition to journalist-photographers, there is a group of photographers who do not believe in making any kind of post-production edits to their photographs. They prefer to challenge themselves to make the best photo they can using only camera settings and then accept the result SOOC – Straight Out Of the Camera.

But the majority of us use tools such as Lightroom or Photoshop to change the exposure and increase or decrease highlights. We may take measures to reduce photographic ‘noise’ induced by the camera in low light situations and we may sharpen the photo.

Likewise, we often crop the photo to help the viewer focus on a particular part of the image. We may even remove distracting elements which pull the viewer’s eye away from the subject of the photograph.

But – and this is key – photo editing is largely about 1) enhancing what is already in the photo and/or 2) removing distracting elements either through cropping or selective erasing or both. But photographic editing rarely includes adding elements to a photo².

So Does Generative Fill Have a Place in Photography?

Initially, following my introduction to Generative Fill as noted above, my answer was “probably not.” But then I watched a video where Scott Kelby did a great job of redirecting the focus of how Generative Fill (and the later addition of Generative Expand) could be used as an aid to heretofore acceptable methods of photographic editing.

In short, as photographers, we are using AI to do things to a photo which we have always had the ability to do in Photoshop, but with a much greater degree of speed and accuracy. So we’re not really doing anything that hasn’t been available before. We’re just harnessing the power of AI to do in seconds what would previously have taken many minutes or even hours. But the outcome is the same.

It is not my intent in this article to address the specifics of how to use the Generative Fill feature. Rather I’m only looking at the outcome vs. more ‘traditional’ photo editing techniques.

So let’s look at some examples.

Crowd Replacement

One of the goals of travel photography is to produce what is sometimes called a ‘timeless shot.’ This is particularly true of travel photos in Europe and Asia, where we may be photographing buildings or monuments that have existed for hundreds of years. The goal is to make a photo³ which could have been taken last week or 50 years ago – uncluttered by people or modern distractions.

The photo on the left below is one I took in 2016 of Il Duomo di Milano, the Milan Cathedral in Milan, Italy. The cathedral, the second largest church in Italy and fifth largest in the world, was constructed between 1386 and 1965 – nearly 600 years of construction.

Obviously, Il Duomo is a major tourist attraction. But to fully appreciate its beauty, it would be nice to have a photo uncluttered by the ever-present crowds of people in the courtyard. While it might be possible to arrive at Il Duomo very early in the morning and get a shot of, at least, smaller crowds, there was still the issue when I was there of ongoing renovation to the cathedral.

Previous to Generative Fill, it was completely possible to remove people and construction equipment from the shot using tools like the spot healing brush and the clone tool. Likewise, it was possible to reconstruct parts of the cathedral which might have been hidden by people or equipment. But this process – selecting a subject for removal and cleaning up artifacts – may have taken hours. However, it could be done – and has been done by many photographers.

The advent of Generative Fill meant that I could make a single selection of the crowd in front of the cathedral and then let artificial intelligence remove them while simultaneously creating a realistic representation of the empty courtyard. Sometimes it takes more than one try, but each attempt only takes 12-15 seconds – compared to tedious hours of selection and spot healing previously needed.

Likewise, removing the construction equipment and modern signage on the cathedral was simple with Generative Fill.

But, of course, there are exceptions. Even Generative Fill could not come up with a solution for the scaffolding on the right side of the cathedral. Nor could I successfully remove it in a reasonable amount of time with traditional methods. So in that case, the only solution for a ‘timeless shot’ was to crop off the side of the photo and thus remove the scaffolding entirely.

Generative Expand

A few weeks after Generative Fill was introduced in Photoshop (Beta), Adobe added an additional feature called Generative Expand. While it has long been possible to expand the borders of a photograph and somewhat accurately fill in the space with the content-aware feature of Photoshop, Generative Expand takes that ability to the next step.

The photo below is the Church of San Simeone Piccolo – the first easily identifiable building one sees when leaving the Venice train station in Italy. With this photo, I did all of the same types of edits that I did on the Il Duomo photo above. I removed visitors and modern artifacts, including a trash can and the modern power boat docked at the right. I chose to leave the gondolier since that’s a classic feature of Venice.

However, later in our trip, I discovered that I had not properly framed my photo and I had cut off part of the steeple of the church. By the time I discovered this, Venice was far behind us on our journey.

I most certainly could have found another photo of the church on the internet. Then using that photo, hopefully taken from much the same angle, I could have copied just the top of the steeple and then pasted it into my photo. Of course, it likely would have taken some time to size the steeple to match the size in my photo. Once that was done, I would also have to take steps to match the colors and, likely, match the light angle. So yes, it could be done the ‘old’ way.

But Adobe’s Generative Expand has access to millions of photos on the internet so it ‘knows’ what the steeple of San Simeone Piccolo looks like. Thus in 15 seconds, I was able to not only expand the sky – something easily done previously with content-aware fill – but more importantly, AI recreated the steeple and exactly matched the color, lighting, and point of view to my photo.

Not Just for ‘Timeless’ Photos

While we’ve seen how Generative Fill and Generative Expand can remove distractions from a photo to make it more ‘timeless’, that’s not the only usage. Sometimes we just want to remove distracting elements and enhance other elements of a photo that depicts a modern scene.

This was the case of the photo below. I visited the Ciao Ristorante⁴ in my home town of Louisville, KY. I was particularly impressed with the mix of industrial and antique looks in their furnishings. Their booths caught my eye, in part because they represented a more enclosed space than the usual restaurant booth. The mix of steel girders and antique barn wood also create an interesting atmosphere.

But in order to ‘make’ my photograph, I wanted to change some things. Not surprisingly, I wanted to remove the chairs from an adjoining table which protrude into the shot. I also wanted to remove the table number as well as the power cord for the lamp, which was just hanging down the wall.

I also wanted to slightly expand the posts on each side of the booth to more visually isolate it from adjoining booths without losing the overall ratio of the photo. I cropped the original photo to bring the edges in to conceal the adjoining booths, but I wasn’t happy with the narrow aspect of the resulting photo. Finally, there were two broken windows in the area above the booth and I wanted to ‘fix’ those.

For this photo, I used Generative Fill to remove the chairs and replicate the flooring. I also used it to remove the table number, although I could have just as easily done that using a selection tool and cloning in another portion of the table leg. I used Generative Expand to increase the size of the posts. Finally, while I could have used Generative Fill to replace the broken glass, I did that the ‘old fashioned’ way – making a selection then copying and pasting a piece of glass from an adjoining window.

The result is on the right, below. Is it a ‘journalistic’ representation of exactly how the scene looked? No. But is it an interesting photo of a rather unique setting in a local restaurant – free of distracting elements? I think so.

What Do You Think?

What is your opinion of the new Generative Fill feature? Is it something you’ve tried or think you might like to try? How do you feel about its application in photography?

Share your thoughts in the comments section below.

Footnotes

1. On September 14, 2023, the newest version of Photoshop – version 25.0 – was released and includes the Generative Fill and Expand options.
2. There are certainly some exceptions to this, notably where two or more photos are composited together to create a scene which didn’t – or even couldn’t – actually exist. But this type of photo editing is notable in that the surrealistic element of the addition is usually readily apparent.
3. See Ibarionex Perello’s book “Making Photographs” (https://www.amazon.com/Making-Photographs-Developing-Personal-Workflow/dp/1681983990) for a discussion of the differences between ‘taking photos’ and ‘making photos’.
4. Ciao Ristorante – 1201 Payne St – Louisville, KY 40204