Each year at its MAX conference, Adobe hosts its Sneaks session that gives creatives a look into cutting-edge, innovative technologies that might become features in its products.
This year, Kevin Hart, alongside Adobe engineers and scientists gave a run-down of everything that the company might make a reality.
Project Clever Composites
This project looks to make compositing an object from one image into a background easier. For example suppose you missed the shot of you standing right in front of the Eiffel Tower in Paris. With Clever Composites, Adobe is trying to make it easy to add you into any background.
Project Clever Composites uses AI and automation to transform the compositing process into an essentially drag and drop action. With a few clicks, Adobe says you can achieve a realistic composite image. AI can quickly identify an object suitable for adding to a background image, automatically cut out the object, adjust its colour and size for consistency with the background, and then generate appropriate shadows by analysing the background lighting.
Project Instant Add
Everyone video editor has finished a project, made it into post-production and realised they forgot to add a logo or need to update some text.
Adobe says Project Instant Add uses AI and machine learning to simplify the processes of editing videos and adding VFX effects in post-production. The company says it will let anyone edit video content in the same way they edit images. Users need only to choose the element in the video to map text or graphics to and AI handles the rest by automatically mapping the graphics to the chosen element in the video.
Project Magnetic Type
This new tool lets anyone attach and instantly fuse any shape — even hand-drawn calligraphy — to a piece of live digital text.
Adobe says its AI tech allows fluid binding of shapes to text without distorting the original text’s aesthetics or its ability to be edited as text. The company says that Magnetic Type uses object detection models to enable extraction of swashes from any calligraphic image, which can be fused with any live type to give it a similar look.
Project Vector Edge
Vector Edge lets designers visualise, make edits, and collaborate on 2D designs within 3D environments during the design process. Using AI and vector graphic projection technologies, this project automatically projects 2D design assets onto surfaces in 3D environments, true-to-scale, and in high resolution, so designers and stakeholders can visualize how assets will look in the real world.
Project Motion Mix
Motion Mix is designed to give maximum viral impact to content creators. Users will be able to create high-quality looping human animations, complete with realistic human movements, all from a still image using AI-based motion generation and human rendering tech to create high-quality, realistic 3D movements for the subject of the video. Creators can also make changes to the video’s background, including adding a friend to the video who joins in the animation.
Project Blink
This project uses AI to help users find and pull highlights out from video content. Users can search for specific words, objects, sounds, or even types of activities that took place in the video, then select the portion of the video transcript they would like to use. Adobe’s AI then automatically transforms that section of the video into a new clip.
This project is currently in beta.
Project Artistic Scenes
Artistic Scenes lets users transform 3D scenes using styles from 2D artwork. This means you can create stylised 3D content easily, something Adobe says will be “essential” for the future of VR and AR.
Unlike traditional style techniques that only transform 2D images, Adobe’s approach transforms the entire 3D scene. This makes it possible to transform a 3D scene into any artistic style. For example, users could render a playground as a watercolour painting including realistic style details such as brushstrokes, creating a fully immersive, artistic 3D experience.
Project All of Me
Everyone knows how to crop an image to make it smaller but un-cropping to a larger size is seriously hard.
Adobe reckons that Project All of Me will be able to accurately generate un-cropped components of photos and edit out unwanted distractions using AI. In fact, the software will even be able to provide recommendations and modify the photo subject to better fit the new un-cropped elements.
Project Beyond the Seen
Beyond the Seen plans to let users create 360-degree experiences from flat 2D images.
Depth estimation methods allow users to extend a panoramic image, creating realistic and fully immersive 3D environments. Since AI is able to generate content behind, above, below, and to the sides of the image, users can easily and quickly add objects, including AI-generated reflections — and more.
Project Made in the Shade
Finally, Project Made in the Shade uses AI to make adding shadows easy. Adobe says its AI tech can recognise depth, lighting, and other aspects of a scene, enabling users to move a person or object within a photo while casting realistic shadows.
