AI-Generated Video - AI Research From The Lab - Red Marble AI

AI-Generated Video – AI Research From The Lab

By Diesel Laws

October 2023

AI-Generated Video: Has AI-Generated Video arrived?

Overview

In our latest round of experiments, we set out with two main objectives. First, we aimed to determine the feasibility of Al generating a 20-second video that merges our unique content with generic data. Second, we sought to gauge the current state of quality of Al-generated videos, specifically assessing its capability to achieve our first goal.

Through these experiments, we gained significant insight into the practical possibilities and limits amid the rapid evolution of Al-generated video.

Key Concepts

Initially, our task involved blending and rendering a custom object, integrating the Red Marble Connectome with a hot air balloon. Using this integration, we transitioned to the video generation phase, utilising the image to produce a video.

This stepwise approach helps in methodically navigating through the process, for precise and enhanced visual representations in the generated video.

Deep Dive

Due to the absence of the Connectome in the base research data, image supply is crucial, enabling a controlled outcome but limiting organic experimentation.

There are a limited number of video generation Al services that utilise images as input to generate videos. Among these, RunwayML emerges as the frontrunner. Other services like Pika and Genmo also offer video generation from image and/or text description inputs, albeit with lower levels of effectiveness and quality.

Setup & Testing

We explored how PlaygroundAl and RunwayML handle custom object blending and rendering. A notable demonstration was done using RunwayML, where a “custom model” was trained to render a flying Connectome hot air balloon, showing a real-world application of custom object rendering.

Additionally, we used RunwayML to creates a video within a time frame of 4 seconds (extendable to 16 seconds). This indicates a certain level of control over the video generation process, making it a structured yet limited approach.

Hitting The Limits

ControlNet struggled with text instructions for object removal or replacement in images. Pika’s video length was unfortunately capped at 3 seconds. Genmo’s video quality was subpar compared to RunwayML and Pika, often significantly altering objects and backgrounds.

Meanwhile, RunwayML had the ability to extend videos by 4 seconds each time, but this caused the source object to change in unexpected ways as the video progressed, morphing undesirably with each extension.

What’s Next? 

Al image generation can create captivating images with rich details, but video generation is still in its early stage. We’re looking forward to exploring more, diving into publications from RunwayML, Meta, and Google. We will also explore open-source tools and other platforms to see what’s on the horizon.

What’s the verdict?

Right now, Al-generated video isn’t yet a controllable medium, but rather an intriguing form of abstract art, captivating but unpredictable.

The tech landscape is teeming with image-focused Al providers, leaving video somewhat in the shadows.

If you’re looking to incorporate your own creative spin or use your own imagery or non-trained data, be prepared for some challenges.

Thanks for checking out our business articles. If you want to learn more, feel free to reach out to Red Marble AI. You can click on the "Let's Talk" button on our website or email Diesel, our AI expert at d.laws@redmarble.ai.

We appreciate your interest and look forward to sharing more with you!

Let’s Talk

Keep reading

An Update on AI Agents - AI Research From The Lab - Red Marble AI
Research Briefs
OpenAI for Docket Recognition - AI Research From The Lab - Red Marble AI
Research Briefs
Fine-Tuning GPT-3.5 Turbo - AI Research From The Lab - Red Marble AI
Research Briefs
12 steps to responsible ai
AI Governance
Audiocraft AI Music Generation - AI Research From The Lab - Red Marble AI
Research Briefs
GPT4all - AI Research From The Lab - Red Marble AI
Research Briefs
Emerging LLMs - AI Research From The Lab - Red Marble AI
Research Briefs
AI-Powered Autonomous Agents - AI Research From The Lab - Red Marble AI
Research Briefs
AI Regulatory Update
AI Governance
AI Regulatory Update
AI Governance
AI Regulatory Update
AI Governance
AI Regulatory Update
AI Governance
AI Regulatory Update
AI Governance
descrimination in ai
AI Governance
The Quiet AI revolution in Heavy Industries -Red Marble AI
AI Strategy
Red Marble Construction Language Research project
AI in Construction
The AI Revolution is here - Red Marble AI whitepaper
AI in Business
AI
AI in Construction
AI Strategy
AI in Business
AI in Business
AI
AI
AI
AI Strategy
AI
Experiments with Red Marble AI
AI Strategy
AI in Business
AI
AI
AI in Business