What I've Been Thinking About Generative Art

Image generated by computer from a text prompt

The field of text-to-image generative art has been drawing investor dollars, and has been growing at an impressive pace, with lots of startups creating services using Stable Diffusion, Midjourney, Dall-E2 and ChatGPT. One notable investor newsletter called Antler has just released an introduction to the field and a map of what's out there.

My brief quote in the article is part of a longer Q and A that I did with the author. If you're interested in where my thoughts are at the moment, read on for the full interview, and I invite you to share your thoughts in the comments:

December 19, 2022

Thoughts about Generative AI or AI Art

Questions from Ollie Forsyth of Antler Investments

 

1. What impact does Generative AI have on you as a creator? 

Not much. I've tried Dall-E2, Stable Diffusion, and ChatGPT but the results for me always fall too far short of what I have in mind to create. I have very specific ideas in mind when I want to make a picture, and I get the best results and the most satisfaction from making my pictures with paint and brushes. I have always made my art with physical tools, and use digital tools for photography and video to document the process and the results.

That said, I'm watching what's happening with great interest. I'm truly inspired by the results other people are getting with the help of generative models. The people who consistently get the best results often have a computer coding background as well as an aesthetic sense. Is it “art?” Sure, why not? It definitely stimulates my imagination and the good stuff strikes me as original.

You often hear artists call AI image models as "tools," but AI is so much more than a tool. It's a creative partner, a synthetic genie, or an inspirational ally. It’s a weird feeling to acknowledge that a machine can be an able creative partner, with the human acting as a kind of midwife or helping.

But I'm 100% old school. For me, imagination is a basic human process, like eating or walking. I don't think I would feel fully human if I had to rely on such a system as a creative partner.

Image generated by computer from a text prompt

Is this a good thing or is it going to have a negative impact on creators?

There's a lot of good about generative AI in art. For one thing, there has been a huge surge in appreciation for surrealism and fantasy, because that's what it does best. I've seen a lot of images that are new and exciting. The potentials of the medium are almost limitless. But it’s also scary and threatening, especially to digital artists who are part of corporate entertainment pipelines, such as concept artists.

Many artists are concerned about Generative AI “scraping” copyrighted images in their training data. Others worry about it displacing artists. Some people are trying to stop it, or at least shape it.

I'm not worried about either of those issues because AI art for the most part alters and transforms its source material, just as humans do. An artist’s style can’t and shouldn’t be copyrightable.

I don’t feel threatened by AI Art. There's no way it can take away my livelihood because of where I'm positioned as a painter in gouache. Mostly what I do is share behind-the-scenes demos of plein-air sketching on YouTube. No way can AI ever replace that.

I've been in the art business for almost 45 years, through a lot of technological changes. My career has had plenty of highs and lows during that span and I've had to reinvent myself several times. I am concerned not only for emerging artists, but also most all digital artists (top of their game or not, it doesn't matter).

What I try to keep in mind is that AI represents both a threat and an opportunity. If we only focus only on the threat, it may kill our career. If we focus on the opportunity we might end up doing fine.

I would caution the alarmists to remember that applying some sort of digital rights management mechanism could have a chilling effect on the growth of this new art form, and end up helping the powerful entertainment corporations more than the little guys.

Looking a little further down the road into the future, I don't see any easy way around the pitfalls of potential misinformation, deep-faking, erosion of shared reality, and content moderation. Should we let people use these systems to create whatever they want? Should we ask the generative models to edit “dangerous” ideas at the level of generation or distribution? Who defines what is dangerous? Should the government define the limits of what we're allowed to imagine or should private companies do so? China is passing new laws requiring watermarking all AI works. Should other countries do that? What are the political ramifications? These are big societal questions.

The biggest negative impact from my point of view is an erosion of human hand-eye skills and a weakening of the artist's confidence in his or her own imagination. Just as the industrial revolution and the invention of the motorcar made us lazier and less physically active, the development of push-button creativity and synthetic writing partners will make us stupider, and dull our ability to dream. These are core human values, and the threat is very real.

Those concerns were at the forefront of concerns among my YouTube followers:

2. What are your most pressing concerns about the platforms?

By "platforms" I assume you mean generative AI models, such as Midjourney, Dall-E 2, and Stable Diffusion. The ease of creating images means that there has been such a cascade of computer generated images and videos that they've glutted many arenas of the art world: portfolio sites, art print markets, contests, and freelance illustrators. This leads to a cheapening of all the images, a dulling of appreciation, and a confusion about what's really created by humans. How do you run an art school when the students can solve any assignment with the push of a button?

People who worry about the threat of AI art focus on the production side, but we shouldn't forget about the distribution side. How will people consume these images, films, and writing? Remember that the prompt doesn't have to come only from the artist, creator or director. It can also come directly from the consumer, acting as individuals or as fan groups.

Imagine an AI video-generator with personal biofeedback input that creates hallucinatory music or video to suit your current mood. Or let's suppose you have online groups who love Japanese anime style. Their preferences can shape the generation of every video they see. They can watch custom editions of their favorite media or games output with their favorite anime style. Or they can watch shorter or longer versions of films, or films customized for whatever language they speak.

If the viewer and the generative model drive the entire creative process, there's really no need for an "artist" or "director" at all. The idea that a work of art or film exists fixed in a single form at the moment of creation and then finds its audience may be a quaint idea that many of us will outgrow as a culture.

The other big concern is that all these models are based on existing, previously manifested styles of art and photography rather than on the direct human response to nature itself. Therefore it is, by definition, "mannerist." In terms of Plato's Allegory of the Cave, we're talking about the realm of firelit shadows in the cave wall, not the light of the sun itself. Maybe someone will invent ambulatory robots exploring the real world, but without such empirical input, this will always be a profound limitation.

Image generated by computer from a text prompt

3. How will you be using Generative AI platforms? 

I won't condemn AI or vow never to use it, and I respect artists who choose to ally with computers to make art. This all didn't arrive out of the blue. Digital artists have been automating parts of their creative process for many years now, with 3D rendering, ray tracing, procedural effects, photobashing, etc. AI models have just moved the needle way beyond where it was.

Now, for anyone to succeed with generative AI, there's no halfway. They need to join the arms race, learn coding, train their models, learn about the secretive alchemy of prompt writing, and cultivate their awareness of past art styles.

It's a very different path from painting, drawing, and animating with physical materials. We all have to make our peace with the digital sphere. Even classical musicians playing original instruments use digital recording techniques and social media, and read their music off iPads in concerts.

4. Where is all this headed? Please provide examples.

AI art draws upon the collective unconscious of human expression to generate something new. Laws will be passed and new businesses will be formed that will shape the evolution. We will come to enjoy art forms that we can't currently imagine. Those forms may take market share away from traditional forms, just as the introduction of television took away from magazines and movies.

Because of the lack of friction to create these new art forms, there will be a lot of derivative junk out there. But let's assume we can develop algorithmic sorting techniques to allow the truly great stuff to surface. The art we'll see as a result of this technology will be surprising and fascinating. It will reflect many inputs: the prompter, the design of the generative model, the zeitgeist of the audience, and of the immensity of the dataset it draws from.

But as you think about generative computer models, don't forget that there's going to be a healthy backlash to the idea of handing off human creativity to computers. That cultural countermovement hasn't coalesced yet, but it will, and it will be powerful. What will this movement be called: the Human Agency? Hand-Eyes? H&H (Hearts and Hands)? Authentics? Analoggers? As David Sax has argued, the future is analog, or at least a healthy portion of it will be.

The invention of the internet a few decades ago was a boon for knitting and hand-lettering. In a similar way, the growth of AI and the decline of social media will be a powerful stimulus for such uber-traditional forms as face-to-face storytelling, on-location sketching, and recitation. Whatever it ends up being called, I plan to be part of that cultural countermovement.

--James Gurney

Revised slightly in 2024 to clarify a few points.

Will AI Eliminate 90% Of Animation Studio Jobs?

DreamWorks Animation co-founder Jeffrey Katzenberg made himself unpopular in the animation industry when he predicted a few days ago that AI would make feature film production ten times more efficient and reduce the number of artists by 90%. 

He said: “In the good old days, when I made an animated movie, it took 500 artists five years to make a world-class animated movie,” he said. “I think it won’t take 10% of that. Literally, I don’t think it will take 10% of that three years from now.”

Katzenberg, creator of the failed video app Quibi, is wrong on this prediction for at least four reasons:

1. AI will lead to a proliferation of second-guessing. Since everyone in the organization, including bean-counters and paper pushers, can generate alternate takes at the press of a button, it will be hard for any creative organization to commit to any set of decisions without facing the inevitable chorus of "Why don't we try this instead?" This effect will slow production, not speed it up.

2. Innovations won't come from the top down, but rather from the bottom up. AI techniques will surely lead to artistic innovations, new production methods, and even new art forms. But the animation production pipelines are so well established in studios like Disney, Pixar, and DreamWorks that they can't easily adapt to radically new toolsets. The filmmakers to watch are the little breakaway upstarts, the tiny mammals currently running around the ankles of the dinosaurs.

3. In practice they don't take you all the way to the finish line. Even though AI methods seem to magically produce immediate, sexy-looking results that would take teams of people weeks or months to produce, they still require the same kind of human expertise and effort to get really first rate results that the public will accept.

4. The lawyers of the big studios won't let it happen. They're already so nervous about copyright infringement lawsuits that they're trying to stop the internal use of AI in the production pipeline.

How Smart is Dall-E 2?

Prompt: “Polymer clay dragons eating pizza in a boat”

Computer-generated image (Dall-e 2 by OpenAI) 

For a several years now, computers have been able to generate images based on a natural-language prompt. 

The resulting images have suffered from problems of logic and global coherence.

For example, here's what you get if you give the computer the prompt “A rabbit detective sitting on a park bench and reading a newspaper in a Victorian setting.” (Latent Diffusion LAION-400M via @loretoparisi)

Where are his legs? His hands? Are those books or newspapers? Is that a coffee table in front of his bench? 

The image doesn't make sense, and we might conclude that the problem comes from the computer not having any experience of living in a body or dealing with the real world. No matter how big the data sets, or how many layers of processing you bring to the task, you can't get past that limitation. 

Or can you? 

Open AI is one of the pioneers of generating realistic images and art from descriptions in natural language. They recently unveiled new software called Dall-e 2, which has pushed the boundaries of what's possible with this technology.

Here's what Dall-E 2 does with the same prompt: “A rabbit detective sitting on a park bench and reading a newspaper in a Victorian setting.” 

The overall logic is much better. Now he has legs and is really sitting on that bench, even casting a shadow. But the image is still not perfect. What's the black loop in his left hand? And why doesn't he seem to be holding the newspaper with his right hand? 

Here's one more example of how the technology is improving, using the prompt “teddy bears working on new AI research on the moon in the 1980s” 

The first version using older tech (laion400m) looks like a paste-up of unrelated elements.

Here's what Dall-e 2 came up with: a pretty believable image with consistent lighting. 

Open AI released this YouTube video to introduce the sofware.

This technology scares some working artists and illustrators. @VividVoid says: "DALL-E is breaking my heart. AI art is about to lay utter waste to traditional visual art forms. This will be so much more destructive than what the Internet did to music. It will be a technological conquest of one of the great human avenues of spiritual transformation."

AI skeptic Gary Marcus doubts whether the technology will ever replace artists because it is just crunching big data sets. It's not learning from embodied experience, nor does it understand symbolic or semantic concepts the way a human does. Marcus says: "This whole thread is weaponized cherry-picked PR; the antithesis of science."

Soon after Dall-E2 was released, OpenAI gave me beta access to try it out. On this YouTube video, I share my first experiments with it. (Link to YouTube)

Read more

The Verge

Dall-e 2 at OpenAI

Podcast: Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI 

Caricatures by Artificial Intelligence

These caricatures were created by artificial intelligence, not by humans.

The system uses Generative Adversarial Networks (GAN), and doesn't require any input from humans to create the caricatures. Scientists developed this AI system to use a two-stage process, a combination of style transfer and geometric exaggeration of facial metrics.

The artistic styles are developed by training the network with large data sets of human-created caricatures from Toonpool and Pinterest. Ideally you'd want to give the computer examples of direct comparisons between photos of people and artistic interpretations from those specific photos, but there's just not a large enough set of those to use for training the computer.

Then they trained the system to distort the artistically-styled image to emphasize the factors that make the face differ from an average face. The scientists explain: "Shape exaggeration is not a distortion, which is complete denial of truth. The exaggerated shape should maintain the relative geometric location of facial components, and only emphasize the subject’s features, distinct from others. The final appearance should be faithful to visual styles of caricatures, and keep the identity with the input face, as addressed in other face generators." 

The system can also be used for animation. These are stills from a video where each frame is created by the computer.

Most of the reactions I've seen to this fall into two camps: "It's coming to take my job" or "It's not that good." To the first I'd say it depends on what your job is, but it might force many of us to change our jobs somewhat if we want to stay in business. For some artists who want to use these tools, more power to you, and these methods may lead to new art expressions that we've never seen before. 

To the second objection, I'd say that this is just a first baby step. How far the neural network strategy will take us is anyone's guess. Some say GANs are superficially impressive but that the method has hit a wall, and it can't begin to rival the complexity and sophistication of the human brain. 

As blog reader Eugene Arenhaus puts it: "It tells more about our preconceptions about what our brain can do, than about AI. Turns out certain functions require far less brain than previously thought - and not unique to humans, as a less snobbish look at animal intelligence had been showing in the past 15 years. But there is a lot of complexity in human cognition that these primitive NNs can achieve, although the people driving hype try hard to pretend otherwise and make wild promises."

For my part, I don't regard this as a direct threat to my livelihood, but more of a reminder that I'll have to evolve my art as these tools get more powerful—similar to the way artists had to evolve with the advent of photography. I also am fascinated to learn how these AI systems work, because those insights can help me understand better my own human thought process when I make a picture.

-----

YouTube video: Can an AI Learn to Draw Caricatures?
Scientific Website: CariGANs: Unpaired Photo-to-Caricature Translation

PDF of Science Paper

Your Face as an Old Master Portrait

A new website lets you upload a photo of yourself and it will modify it to look like an old master painting.

This is not an example of style transfer that we've seen before on the blog. The AI erases smiles and changes lines, colors, and shapes, and it decides on which period art style to use. 

With this system, according to their website, "anyone is able to use GAN (Generative Adversarial Network) models to generate a new painting, where facial lines are completely redesigned. The model decides for itself which style to use for the portrait. Details of the face and background contribute to direct the model towards a style. In style transfer, there is usually a strong alteration of colors, but the features of the photo remain unchanged. AI Portraits Ars creates new forms, beyond altering the style of an existing photo."

-----
See a gallery and try it out at AI Portraits
Article about the algorithm on Fast Company: AI Portraits
Thanks, Armand

Resources for Art Styles

One positive outcome of the growth of generative AI is an awakened interest in art styles: how to learn about them, recognize them and name them. 

For example, I had never heard of chainmail art, but here's an actual piece by Dave Austin. It's made by hand from metal wire, link by link.

Here are some examples of AI-generated images inspired by chainmail art, courtesy Twitter user Anonymouse, who has compiled helpful lists of styles of art and photography to use as inspiration.

Anonymouse has got about five collections so far on various Twitter posts. In his collection #2, for example, he's got ASCII Art, polar panorama, mosaic style, and Fresnel Effect.

Another rich vein of styles is Aesthetics Wiki, which defines and illustrates everything from Barbiecore to Dark Gatsby. This online community seems to be fueled as much by cosplay, anime art, and toys as it does by AI, but apparently the team at Midjourney took a lot of inspiration from these communities when they were developing their tools.

Every art school—and especially every design school—should offer classes that explore these art and photography styles, putting them in context without judgment. 

Have you found other resources for learning about art styles? Please share them in the comments.

A.I. Animates a Cartoony Drawing

 Do you have kids around? Here's a fun and easy animation project you can do with them. 

View this post on Instagram

A post shared by James Gurney (@jamesgurneyart)

1. Have them draw a humanoid character (a head, two arms and two feet). 

2. Go to this link "Animate a cartoony drawing". (It's an AI research project from Facebook / Meta.) 

3. Upload their drawing, following the prompts, and choose a style of action, such as walking, jumping, or dancing. (The artificial intelligence will do the pose and silhouette detection, joint placement, rigging, and animation.) 

4. You can share or download the animation, add music, etc. (I used music by Kevin Macleod from YouTube's free Creator Studio collection. The animation algorithm only lets you download once, so you can use QuickTime to record more.)

Meta's researchers say: "Someday, perhaps, an AI system could take a complex drawing and then instantly create a detailed animated cartoon using multiple fantastical characters interacting with one another and elements from the background. With AR glasses, those stories could even seem to come to life in the real world, dancing or talking with the child who drew it just moments earlier. The possibilities are as limitless as the human imagination."

Wall Street Journal's Hedcuts

Top row: hand-drawn portraits; Bottom row: computer-generated versions.

The Wall Street Journal has developed an artificial-intelligence system for creating its distinctive 'hedcut' portraits.

Human-created hedcut of Grumpy Cat, 2013, courtesy Wall Street Journal

Hedcuts have a wood-engraving or scratchboard look, made up of dots and dashes.

Left: human-created 'hedcut' of actress Chloë Grace Moretz
Right: AI-created hedcut, courtesy Wall Street Journal

The AI learned the style and produced adequate results in most cases. But there were difficulties. One challenge was "teaching the tool to render hair and clothes differently than skin, which was often a matter of whether to crosshatch versus stipple."

Error cases caused by AI working with too limited set of data,
courtesy Wall Street Journal

"The most harrowing issue of all was overfitting, which happens when a model fits a limited set of data too closely. In this case, that meant the machine became too satisfied with its artistic ability and began producing terrifying monstrosities like these."

----

Read more on Wall Street Journal: What’s in a Hedcut? Depends How It’s Made.

Looking Back on Illustration Magazine

Illustration Magazine has announced that it will end publication. The quarterly print magazine has spotlighted most of the great American illustrators from the 20th century in lavishly illustrated articles. The magazine was the brainchild of Dan Zimmer, who almost singlehandedly did the research, interviews, writing, editing, and publishing.

I asked him a few questions as we look back on his remarkable achievement.




Gurney: You're ending the 20+ year series of 84 issues on an amazing high note by releasing four issues all at once. How were you able to pull off four simultaneous issues?

I began looking at ending the magazine a few years ago, as the numbers were declining and I could see the end was nigh. The problem has always been my overhead of subscribers. I didn’t want to cheat people by suddenly going out of business or something. So… How do I fulfill my obligations to everyone in the most cost effective way? I didn’t want to do “double issues” or something. 

The only real solution was to produce a year’s worth of issues at one time in China, which would save a ton of money ($18,000 vs. printing in America), and then to ship them all at once, which would also save money on postage, boxes, etc. Anyone with a subscription beyond issue #84 was sent a refund check for their balance of issues. I think I wrote 105 checks.

 
As far as getting all of the issues together, I chose artists that were somewhat easy to cover, as I had the material readily at hand in my archives.

 
It was a lot of work compressed into a short period of time, but I tried to make the issues as solid as any of the others in the past. I didn’t want to “phone it in”. I think I succeeded.

The last four issues include features on John Held, Jr., 

Howard Chandler Christy, Robert Meyers, 

Bernard D'Andrea, Frank McCarthy, Paul Bransom, 

N.C. Wyeth, and Ward Brackett. 

Who were some of the key unsung heroes behind the scenes who helped you bring out the magazines?

I’ve had some great contributors over the years who have supplied me with wonderful articles, scans of artwork and tear sheets, etc.

The advertisers, of course, many of whom probably realized little return at all from their ads in the magazine but who continued to support me regardless. Grapefruit Moon, Fred Taraba, Kayo Books, Illustration House, Heritage, there are too many to list over the years.

David Saunders in particular (the son of famed pulp illustrator Norman Saunders) was one of the greatest supporters of the magazine, and contributed in many ways over the years and he became a great friend and confidante.

The late Charles Martignette was also a wonderful supporter, though his overwhelming blocks of ads in the magazine quickly became a bit of a problem. But it was hard to turn down the extra revenue!
As far as actually physically making, designing, retouching, packing, making the website, etc. I have always done all of this on my own. I would laugh when someone would refer to my “subscription department”.

Do you have any advice for small or individual publishers?

At one time publishing was a great way to make a living, in the 90s especially as desktop publishing turned everything upside down and one person could suddenly produce a magazine by themself. TODAY it’s all about the internet. An art magazine in print is kind of a crazy thing now, as everything is instantly accessible online. It’s an anachronism. I think my own magazine was overwhelmed by Facebook groups and websites, where the imagery is now readily available and best of all—FREE! I can’t compete with that.

Anyone who wants to be a publisher must wear a lot of hats. You will need to know how to do it all, from writing and editing to website design, social media maintenance, Photoshop, graphic design, the list goes on. If you aren’t doing it yourself, you’ll have to pay someone else to do it, and that gets expensive quick!

Print on demand enables you to do extremely tiny print runs, but the unit cost of the product is extremely high compared to “real” printing, so I don’t see where the money is at with such a venture. Each single copy ends up costing $25 or more. There’s no way to sell them wholesale at that rate, so distribution is not possible. It’s a fun hobby at that level, but you can’t do that to make a living.

Only get into publishing if you really love it, and you aren’t worried about making your money back.

Let's go back to the beginnings. How did you first become aware of the history of illustration? Were there any exhibitions, galleries, or personal encounters that grabbed you and got you into it? 

When I was a child, I was exposed to a Norman Rockwell art book and some framed prints. We also had a collection of Peanuts cartoons, MAD magazines, some 50s horror comics. I quickly got into comic books, bubblegum cards, there were so many cool visual things in the 1970s. (I was born in 1969.) My grandparents had some old Saturday Evening Post and Collier’s magazines, so I was able to see illustration art from bygone eras from my earliest days. And I loved it. So I started seeking it out wherever I could. Walt Reed’s book was in my local library, along with Andrew Loomis art instruction books, and so on. I learned to draw by copying Loomis’ work.

Star Wars prompted the publication of a lot of science fiction related picture books that reproduced old pulp covers, which were a revelation to me. What was a pulp magazine?? I had never seen one or heard of anything like that. Strangely I was a member of the Wizard of Oz fan club when I was 12 or something, and they published a directory of their members. Ray Bradbury was listed in there, with his actual home address! Crazy!

Around that time I had purchased a used and waterlogged copy of a book called The Pulps that had come out in 1970. The book had a small color section poorly reproducing incredible pulp covers that just blew my mind. (This was my very first exposure to artist H.J. Ward’s work.)

A guy named Sam Moskowitz contributed to the Pulps book, and I happened to see the same name in the Oz fan club member directory. Was it the same guy?? I wrote to him, and yes it was! I asked him about pulps, and how do I find out more about this stuff? He kindly wrote me back and turned me on to some fanzines and other information that was invaluable in those days.

I was in middle school in St. Louis, I knew NOTHING at that time about comic book conventions, or that a PULP CONVENTION was actually held in St. Louis regularly in the mid-70s. (I never knew about it, so I never went!) I learned much later that Moskowitz was a seminal figure in the history of science fiction. I was a kid, what did I know?

But his information opened up a lot of doors in the pre-internet days, so I was able to start buying pulp fanzines and learning about the great artists of the pulps. Around the same time I also discovered the book 40 Illustrators and How They Worked, which was a great introduction to a lot of great magazine illustrators of the past.

What books or magazines inspired your editorial approach to Illustration magazine?

 
I was heavily inspired by Communication Arts magazine, as far as the minimal “non-design” of my layouts. The emphasis being on the art, and not on any kind of flashy or distracting design. I didn’t want anything to ever look “campy” or “zany” as so many other books have looked that covered vintage or retro subjects. I wanted to treat the art as you would see it in a book on Money or Sargent. With respect. The boring layout was not intended to be a showcase of MY ability as a designer, I wanted the art to always be the main focus.

I was also inspired by Walt Reed’s auction catalogs, and Russ Cochran’s auction catalogs. The “Methods of the Masters” articles in Step-by-Step Graphics magazine were also a key inspiration. I would be so frustrated when they would skip the column from time to time, as that was my main interest in the magazine in the first place, and I thought how great it would be if there were a magazine devoted ENTIRELY to illustration from the past. I knew it would never be a mainstream kind of thing, but I knew it would make a great fanzine. I first started thinking about this in 1989 and throughout the 90s. As technology evolved and direct-to-plate desktop publishing became a reality, the costs of making a full color magazine by myself plummeted to the point where I started seriously considering it in 2000 or so.
What do you like or dislike in the coverage of illustration history by traditional art publishers or museum institutions?

Things have improved quite a bit over time. As I said, the “campy” type of coverage of the material is still an issue, but it seems like you’re seeing it taken more and more seriously now. In direct response to the MONEY that original illustration art is generating at auctions. The huge dollars always make a difference.

There's a debate in art history: Some make the case that the concept of a "golden age" is baloney. They argue it's an illusion powered by nostalgia. In other words, every age is a golden age, except our own. The other view is that there are creative periods of truly pioneering innovation and peak skills, and there other decadent periods of confusion and decline. Which of those two views of the golden age idea makes more sense to you and why?

I think the old illustrators, when they were referring to the Golden Age, were essentially referring to a time in the field when an illustrator could make enormous amounts of money and have significant cultural impact. That all faded as technology displaced illustrators as THE image and tastemakers of the day.
But as far as artistic opportunity, this may actually be one of the BEST ages yet. Artists have never had so much opportunity to have their work seen instantly all around the world. There are YouTube artists with millions of views, who sell their work to eager collectors. There are illustrators who have had $500,000 Kickstarter campaigns for their books. There are a lot of possibilities today, but you have to wear a lot of hats and master entrepreneurship like never before to be one of the stars who really makes it. Could THIS really be the new Golden Age? I think it might be!

So much of print culture disappeared after it came out, leading to a crisis of confidence for many illustrators about the value of their legacy. Were illustrators right to worry that they'd be forgotten, or do you see a trend for renewed recognition?

I think the internet has fostered a great deal of recognition and renewed interest in the illustrators of the past. The imagery and information is more accessible than ever. So work from old publications thrown out 70 years ago has a chance of being seen again, splashed across Facebook groups and Pinterest pages. The chance to be remembered is better NOW than ever before, no doubt about it.

Do you have any advice for older illustrators about how to preserve their legacy, and how to make life better for their descendants? Can you offer general rules or examples of who did it right or wrong?

Illustrators alive right now need to be very serious about taking great photos of their work and preserving their material in a permanent way.

I would say, build a great website. Share the work every day and get it out there. If you’re invisible you will certainly be forgotten.

Sign your work, probably TWICE! Never leave an original unsigned.
Making a hardcover book is probably the best way to build an enduring legacy. An artist is always taken more seriously when there’s a nice solid substantial book to point at, whether that book is a financial success or not! A book is a great catalog.

As you documented the lives of so many successful illustrators, did you discover common personality traits among them, or common work habits? Were there any that stood out as wildly different from the norm?

The common thread has always been the incredible work ethic. The most successful illustrators simply worked harder than everyone else. Norman Rockwell painted every day, even holidays. He was always at it, and he never cut any corners. The level of craftsmanship was astonishing. Any artist needs to realize that you have to put the time in. Writers have to write, artists have to make art. Constantly.

Were there any illustrators or illustrations you really wanted to cover, but couldn't because of copyright or other logistical reasons?

Gathering high quality material was always the greatest challenge. Photos of original art, and clear tear sheets.

 
People often asked me if I worried about running out of subjects, and I would laugh. The number of artists to cover is endless. I think I featured around 200 or so over 84 issues. My hard drive has file folders covering over 2000 different illustrators! I could have gone on and on for many years. It’s endless.

I think you mentioned you had an article in production on John Bauer. Were there other illustrators where you gathered some interesting elements, but for one reason or another just didn't get the articles finished?

In the case of Bauer, the magazine expired before the article was complete. There were three or four articles in the works that I couldn’t publish as the magazine was ending. I was continuously gathering material and scans for many many illustrators that I earmarked as favorites I wanted to cover. Sometimes it took many years to gather enough to fill 40 pages. I wanted to feature Mortimer Wilson, for example, but never got a good bio together. I wanted to do a book on Dell comic cover artist George Wilson, but I couldn’t find a photo of him or any link to his family. I had plenty of images, but not enough information.

What do you see coming for illustrators entering the business now? Will they all have to become animators, publicists, or coders? How will AI change things for artists?

Illustrators today have to be entrepreneurs. It’s more the lifestyle of a “fine artist” in that you have to generate your own projects, create your own worlds. You’re not working for clients as in the past. It’s possible that illustrators will still find clients in the old fashioned style of doing business, but the most successful people are working outside of that mold. I see illustrators creating their own “how to” books, video courses, YouTube channels, all sorts of things all at once. You have to be a master of social media now, and you have to be very diverse. You have to enjoy being in front of the camera and turn yourself into a celebrity of sorts. AI art is going to turn a lot of the business upside down, but there will always be room for real artists with vision creating unique things that an AI won’t be able to match. (Maybe!)

When it comes to AI art, I can only imagine that you've had a variety of reactions to the innovations in the last couple years. Have you been amazed, terrified, bored, disgusted, or some combination of those? Is there something people are missing when they talk about this new technology? Is there something about adapting to change that we can learn from the experience of previous illustrators?

The revolution in AI art has been incredible. As the founder of Nvidia said recently, the future won’t be in rendering, it will be in generating. You can make hyper real imagery without 3D modeling or rendering tools. I NEVER saw that one coming. What will it mean when anyone can “prompt” their own Hollywood style superhero movies from home? This reality is coming right around the bend. I can’t imagine how it will remake the entertainment business, the image making business.

In the end, though, someone with talent and taste is going to have to guide all of this in order to create something worth looking at. Who knows how far AI can go in terms of genuine “creativity” but I don’t think talented artists have to be worried. Things will change, but it’s a big world out there and there are always opportunities for the witty and clever to carve out a niche.


Illustrated Press

Using AI to Guess How a Painting Was Made

Scientists have developed machine-learning system that analyzes existing paintings and reconstructs how they might have been painted.

The authors say:

"The system, which the team calls “Timecraft,” was trained on more than 200 existing time-lapse videos that people posted online of both digital and watercolor paintings. From there, the team created a convolutional neural network that can look at a new painting that it’s never seen before, and figure out the most likely way it was created."

The study is a couple years old now, and with more extensive training data and more computer power, the results would now be a lot higher resolution and more accurate.

Link to YouTube

Science paper: "Using AI to recreate how artists painted their masterpieces"

Text-to-Image Synthesis

Novel photo-real images generated by an adversarial network of computers
based solely on a written prompt, without human intervention or photo cues.
Low resolution version on top row iterated to higher res on bottom row.
via Olivier Grisel on Twitter

In the ten years of this blog so far we've witnessed startling advances in the ability of computers to create and interpret images.

We've seen systems that can (links to previous posts):
• Re-render a photo in the style of any artist.
• Identify faces and objects, no matter the lighting, angle, or context.
• Render a photo in a painterly way that puts more detail in psychologically salient areas.
• Paint a generalized portrait that's typical of the style of Rembrandt.
• Generate captions for images describing at a higher level what's going on in a given photo.
• Analyze the abstract elements of a target image and then locate other abstractly similar images.

Despite these advances, most of us human picture-makers can still pride ourselves in our unique ability to create a photo-real image based purely on a written description.

Suppose, for example, you were asked to paint a picture of "a small bird with a pink breast and crown, and black primaries and secondaries." Could you do it? And could you render your picture so believably that someone else might mistake if for a real photo?

Computer generated images courtesy CreativeAI.net

Computers are figuring this out, and they're starting to get good at it. Scientists are approaching the problem of text-to-image synthesis by means of a deep-learning technique called "generative adversarial networks" or GANs for short.

This GAN strategy pits two separate computer networks against each other. The goal of the Generator one is to create images that fit the text prompt, and the goal of the Discriminator is to distinguish synthetic images from real ones.

As the Generator tries to create images to fool the Discriminator, it gets harder, because the Discriminator keeps learning, too. Exactly what the computer "knows" about the structure of form or the aptness of illustrative problem-solving is hard to say because it wasn't taught by a human; it figured it out on its own, in its own way.


Related video: Image Synthesis From Text With Deep Learning

The resulting images are not an average of existing photos. Rather they're completely novel creations.

Furthermore, GAN image synthesizers can be used to create not only real-world images, but also completely original surreal images based on prompts such as: “an anthropomorphic cuckoo clock is taking a morning walk to the pastry market.”



How good are these synthetic illustrations?

So far the images are small (about 64 x 64 pixels) and for the most part, they still won't fool any humans. But watch out: you're just seeing just baby steps.

GANs currently do pretty well generating plausible pictures of birds and flowers, but they have limited success with complex scenes involving human figures, or generalized text prompts such as "a picture of a very clean living room."

They're a bit garbled and incoherent at the moment, but they will develop rapidly. In a few years, advanced A.I. image-creating tools that can illustrate any text prompt in any style will be available cheaply to art buyers everywhere.  

Geek out: 
• A scholarly PDF: Generative Adversarial Text to Image Synthesis
• Related scientific paper about texture synthesis: Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
• More images at Creative AI

You're Not Allowed To Imagine Thaat

What are the implications of an AI model that refuses to cooperate? What happens when it tells me that I’m not supposed to imagine something? What if it becomes illegal to imagine something without using a safe-certified AI? More on Substack
 

A.I. Generates Oddly Evocative Abstractions

 

A high school student named Robbie Barat, working with machine learning software, has developed a series of images that is strangely provocative.

He fed the generative adversarial network with hundreds of nude painted portraits from a few hundred years ago. The "generator" AI would then create variations of that dataset to try to fool a "discriminator" network that was primed to distinguish real images from fabricated ones.

Barat told CNET: "'The generator tries to come up with paintings that fool the discriminator, and the discriminator tries to learn how to tell the difference between real paintings from the dataset and fake paintings the generator feeds it."

The way he configured the system, the images lack faces or immediately identifiable human parts, but they still have the feeling of fleshiness and weight.

The resulting images resemble something from modern art, and they remind us of how our visual brains seem to want to unconsciously resolve images into something basic that we can recognize.

A Good Explainer on AI Art

I'm honored that Vox media asked me to be part of this video about AI-generated artwork. 

If you don't want to watch the whole thing, I make a brief appearance at 10:25. 

Producer Joss Fong and her team came up with the brilliant explanation on what actually happens with a deep learning model. 

At 5:59 she explains multi-dimensionality with the example of yellow banana vs. red balloon. It's intriguing that we can't possibly know, in human terms, the criteria that the system is using to arrive at its results, or exactly what features it's extracting when a certain artist's name is used in the prompt.



There's a hidden bonus video that explores the reactions of various artists. 

Please add your comments:

• How do you feel this technology will affect the business and practice of art that you do?

• Do you want to use these tools?

• Will they change what you do or how you do it?  

I don't feel directly threatened by the tech, but I realize it will offer art buyers a cheap and fast method for generating editorial illustration, album cover art, and concept art. So it puts professional artists in those fields on notice and gives anyone the keys to becoming both an artist or an art director. 

As an art watcher, I have a kind of morbid curiosity to see where the technology is headed next, and I dread the onslaught of cheap surrealism that is already flooding social media. Another thing I've noticed is that there's a shelf life to each new set of tools, just as there is for each new type of VFX technique. Each new set of tools becomes old hat, as 

Image-to-Image Style Transfer

Suppose you made this rough sketch and wanted to finish it in a photo-real or painterly style.

Twitter user @TomLikesRobots is a digital artist who did just that, using the new image generation tool called Stable Diffusion (#stablediffusion), the open-source model which has a powerful feature called Image to Image (#img2img). 

He uploaded the sketch and told the system to render it as "A black and white photo of a young woman, studio lighting, realistic, Ilford HP5 400." The hair style is a little different, and the ear is weird, but the basic pose and lighting is pretty close to the sketch.

Tom says: "Overall, composition is controlled by the sketch and the details are controlled by the prompt. 

This time he told it he wanted it to make the sketch look like "A portrait of a young woman by Norman Rockwell." 

It looks like a painting, but not much like Rockwell—more like a Victorian artist like Charles S. Lidderdale. On close scrutiny it doesn't hold up too well: the blue ribbon and the costume don't make sense, and her left eye has too many eyelashes.

Here's the sketch rendered in the style of Gustav Klimt. It's got a lot of drawing problems that Klimt never would have allowed, but it is somewhat reminiscent of his style.

And here's the sketch as painted by Vincent Van Gogh. Again, we can pick it apart, but it's in the ballpark.

.

And finally Alfonse Mucha. He is one of the hardest to emulate. It used soft internal transitions and clear outline.

The system was also able to translate the sketch into the features of famous celebrities:

Selena Gomez

Scarlett Johansson

Nicole Kidman

Emma Stone

All of them have big problems with the hair, but they're recognizable, and let's admit the tech is in its infancy and will only get better.

If you're an artist, you might find all this a little scary, threatening, or astounding. I do too! I feel like we've been introduced to a magical genie who can bring whatever we wish into existence.

Some have responded by calling for laws to ban the technology from creating images in the styles of living, working artists. What if someone created a print for sale that was supposedly painted by you or me?

I think we have to be careful how we respond to this apparent threat. Let's keep in mind that the technology is transformative, meaning it doesn't copy/paste images. It creates something new. And artistic styles can't be—and shouldn't be—copyrightable. These tools are here to stay. They're only going to get more advanced, and they're open-source. Also the people who are developing it are artists, too. 

I'm wary of laws or AI bots that restrict the growth of this new art form, or that drive the prompts underground. Instead of involving politicians and lawyers and AI bots, we should encourage a culture of mutual respect and fair play. Perhaps we should encourage generative artists to share their prompts when they use a living artist's name, and never to mislead their audience into thinking the work was actually painted by that artist. Basically, people should give credit where credit is due.

How do you think we should regulate or guide this new art industry?