AI for Image Synthesis

How do we use AI to generate new images from scratch?

In my last article in this series on The AI Developer’s Toolkit, I introduced you to the three most popular AI tools for image analysis. These tools allowed us to extract useful information from digital images.

However, there are many cases where want to generate new images from scratch. This set of tasks is referred to as image synthesis.

In this article, I’ll introduce you to the three most popular AI tools for image synthesis.

Image Completion

Image completion allows us to fill in missing areas of an image with a best guess as to the missing content. It answers the question, “what should go in this missing part of the image?”

For example, we can use image completion to remove unwanted objects from our images. We provide the image-completion model with an image and a mask containing the content to be removed as input. Then, the model produces a new image with the missing area filled in as output.

Image completion is useful anytime you need to replace part of an image with a synthetic alternative. For example:

• digitally removing objects from images
• filling in the missing area outside of a photo’s borders
• restoring image quality to old and damaged photos

Image Generation

Image generation allows us to create an artificial image given a short text description. It answers the question “what would this thing look like?”

For example, we can use image generation to create visual content simply by describing the object that we want to see. We provide the model with a text description as input. Then the model produces a synthetic image matching that description as output.

Image generation is useful anytime you need to create new images from scratch. For example:

• creating content for presentation slides
• rapidly iterating on a new product design
• creating entirely new and unique works of art

Image Style Transfer

Image style transfer allows us to apply the stylistic characteristics of one image to another image. It allows us to restyle an image in a completely different visual style.

For example, we can use image style transfer to make our photos look like a painting by a famous artist. We provide the image-style-transfer model with a source image and a second image containing a target visual style as input. Then the model produces a third image containing the source content, but in the new visual style, as output.

Image style transfer is useful anytime you need to re-stylize an image. For example:

• applying artistic filters to your digital photos
• re-branding images with your company’s visual style
• creating new virtual reality and gaming experiences

Beyond the three image-synthesis tools that we’ve seen so far, there are also a variety of other tools available for image synthesis.
For example:

• Image colorization – which allows us to convert black-and-white images into full-color images
• Image super-resolution – which allows us to “zoom and enhance” the image quality of a low-resolution image (yes, this really exists)
• Depth estimation – which allows us to create a depth map from 2D image
• Face generation – which allows us to create entirely synthetic human faces given just a few attributes or physical characteristics
• Sketch generation – which can convert a sketch of a person into a synthetic face
• Image interpolation – which can fuse the attributes of two images into a single image

As we can see, image-synthesis tools allow us to transform existing images and create new images from scratch.

If you’d like to learn how to use all of the tools listed above, please watch my online course: The AI Developer’s Toolkit.

The future belongs who those who invest in AI today. Don’t get left behind!

Start Now!

[Image Source: Deep Dream Generator]