This AI model lets you generate videos using only your photos


Screenshot by Lance Whitney/ZDNET

As one of the latest developments in the AI landscape, AI-powered video generators have been growing in popularity. These services work similarly to other generative AI tools: Describe your desired content in a prompt, and the bot responds, hopefully delivering what you want.

Launched in June, Luma Dream Machine is an AI video generator with two capabilities: text-to-video and image-to-video. Luma only creates short videos — a few seconds in length — but you can extend a video another five seconds by adding more details to the prompt.

Also: Want to clone yourself? Make a personal AI avatar – here’s how

What’s more, Luma is now running Dream Machine 1.5, its latest model, which is designed to better understand your prompts and generate more realistic videos more quickly.

Here’s how to use Luma Dream Machine’s AI to create a short video.

How to use Luma Dream Machine

To start, head to the Dream Machine website. Click the “Try now” button and you’ll be prompted to sign in with a Google account.

Luma offers a variety of plans based on how many videos you need to generate per month and how you plan to use them. The free plan offers you 30 videos each month with no commercial use allowed, while the Lite plan costs $10 per month for 40 videos. 

The most expensive option is the Premier plan priced at $500 per month, which comes with 2,400 videos each month. 

Sign into the website

Screenshot by Lance Whitney/ZDNET

Text-to-video prompt

Let’s start with Luma’s text-to-video generator. In the prompt field, type a description of the video you want. Be specific and include any important details. 

You can also describe the mood or atmosphere you want to create. Make sure the option for “Enhance prompt” is checked, as that helps Dream Machine turn simple descriptions into more detailed scenarios. 

Also: The best AI image generators of 2024: Tested and reviewed

If you want the video to keep looping, check the option for Loop. After you’ve typed the description, click Create.

Type your description

Screenshot by Lance Whitney/ZDNET

Your video will queue up. Depending on the complexity of the scene, you may have to wait several minutes or longer for the video to appear. Hover your mouse over the video to play it.

Review the video

Screenshot by Lance Whitney/ZDNET

You can now rate the video by giving it a thumbs up or thumbs down. I asked Dream Machine to create a video of a knight in shining armor battling a dragon breathing fire with both of them in front of a castle. The video looked good, but the fire came out of nowhere instead of the dragon’s mouth. For that reason, I gave it a thumbs down.

Respond to the video

Screenshot by Lance Whitney/ZDNET

If the video doesn’t meet your requirements, you can always try again. Revise your original prompt to correct whatever element didn’t come out right. In my case, I rewrote the prompt to specifically indicate that the dragon should be shooting fire out of its mouth. 

Also: Meet Stability AI’s Stable Video 4D, a nuanced take on AI video generation

With the revised prompt, Dream Machine did a better job. Each video you request appears on the main screen so that you can play or tweak any of them.

Try again

Screenshot by Lance Whitney/ZDNET

Next, you can extend the video by five seconds by adding other details to the prompt. To do this, click the Extend button and then elaborate on your prompt with additional instructions. When done, click the up arrow to submit the new details. 

You can then preview the video to see if the new information shows up.

Extend the video

Screenshot by Lance Whitney/ZDNET

Image-to-video prompt

Now that we’ve tried Luma’s text-to-video feature, let’s move on to its image-to-video generator.

This capability lets you upload images that Luma will use to create a video instead of writing a prompt from scratch. You can then submit your request with or without a description. 

Two images work best here — the first acts as the starting frame and the second as the ending frame. Dream Machine’s job is to create footage in between those two frames. The images must be in JPG, JPEG, or PNG format and ideally have a 16:9 aspect ratio, but Dream Machine can work with other formats as well.

Also: TikTok’s parent company launches AI text-to-video generation tool

To try this, click the image icon in the prompt. Select the two images you wish to use. You can switch the position of the two images by clicking the curved arrow between them or swap out one of the images by clicking the X. 

Let’s try submitting the request without any prompt to see what Dream Machine cooks up on its own.

Use images to create a video without a prompt

Screenshot by Lance Whitney/ZDNET

Hover over the video to see how it plays. You can extend the video by five seconds by clicking the Extend button and adding another element to the prompt.

In my case, Dream Machine extended a video I had requested based on two photos of the Statue of Liberty. The extra few seconds added a nice touch by lingering on a close up of the statue’s iconic torch.

View the video

Screenshot by Lance Whitney/ZDNET

To generate a more specific video, add your two images and then type a description of what you want.

Use images to create a video with a prompt

Screenshot by Lance Whitney/ZDNET

Hover over the resulting video to see if it matches your description. If not, you can try again or extend the video by adding more details to the prompt. 

I added two portrait images — one of Herbert Hoover and another of Franklin D. Roosevelt. I asked Dream Machine to create a video changing Hoover into FDR, and it pulled off an effective morph of the 31st president turning into the 32nd.

Also: Midjourney’s AI-image generator website is now free for everyone

I also tried the same two images but without a description. In this case, the results were not quite as satisfying, as Dream Machine failed to add the cool morphing. Even when you take the image route, adding a prompt can help steer the results to better meet your expectations.

View the result

Screenshot by Lance Whitney/ZDNET

Once you’re satisfied with the video, click the Download button to download it as an MP4 file. You can then play it in your default media player or tweak it in your favorite video editor.

Download the video

Screenshot by Lance Whitney/ZDNET

Also: How to use ChatGPT to write your resume

Overall, I was pleased with the performance of Dream Machine. I did have to tweak and finesse the prompts to get my desired results, but that’s true of any generative AI tool. In some cases, the videos took a long time to generate, so you’ll have to be patient. 

To learn more about Dream Machine, check out Luma’s FAQ page.





Source link