Real-Time Foundations, Scene-Aware Control, and LLM-Guided Rewards

News

NVIDIA Unveils 12B-Parameter Text-to-Image Model with Sub-2-Second Generation

NVIDIA's new text-to-image foundation model (12B parameters) generates photorealistic images in under 2 seconds on their latest hardware. This is a significant performance leap that makes real-time, high-quality generation more accessible for applications like gaming and design.

Adapting ControlNet for Consistent Spatial Control in Video Generation

Researchers demonstrate a way to apply ControlNet-like spatial control directly to video generation models, enabling more consistent pose and depth-guided scene creation across frames. This addresses a major pain point in controllable video synthesis.

Tools

CogVideoX Releases Scene-Aware Control for Longer Video Generation

An upgrade to the open-source CogVideoX model improves temporal consistency and adds a new 'scene-aware' control mode for generating longer, more coherent video sequences from text prompts.

Open-Source Toolkit for Custom High-Resolution Text-to-Image Fine-Tuning

A new toolkit simplifies creating custom, high-resolution text-to-image models by automating data curation and fine-tuning workflows. This lowers the barrier for artists and small teams to develop specialized generators.

Analysis

Using LLMs as Reward Models to Improve Text-to-Video Instruction Following

This paper proposes a method to use large language models as reward models for video generation, fine-tuning diffusion models to better follow complex, multi-step instructions. It's a promising step toward more controllable and instruction-aligned video generation.