One photo in, a moving video out. It is the most seductive promise in the category, and unlike most such promises it mostly works — just not uniformly, and not in the way the demos imply.
This is what is actually happening under the hood, what breaks, and how to get a clip you would run.
What the model is actually doing
Your photo becomes the first frame. The model then generates every subsequent frame, one conditioned on the last, trying to produce something that looks like a plausible continuation of what it was given.
That framing explains almost every behaviour you will observe:
- The first frame is the one thing you fully control. Everything after it is a prediction. A bad first frame cannot be rescued by a better prompt.
- Error accumulates. Frame 2 is a small guess away from frame 1. Frame 120 is 120 guesses away. This is why clips degrade toward the end and why eight seconds is harder than four.
- The model has no idea what your product is. It sees shapes, colours and textures. It does not know the logo is a logo, or that the text has to stay spelled correctly.
- Motion is invented, not discovered. There is no hidden information in a still about how the object moves. The model is making it up, constrained by what usually happens in videos like this one.
Everything below follows from those four facts.
The four ways it breaks
1. Product drift
The packaging slowly becomes a different packaging. The logo softens into a smear. Text that was legible in frame 1 is nonsense by frame 60. Colour shifts half a shade.
This is the most common failure and the most damaging, because it is invisible while you are watching for motion and obvious once you look at the product.
Mitigations: keep the clip short. Keep the product large and centred — small objects drift faster. Avoid asking for motion that would turn the product, because a rotation forces the model to invent the sides it has never seen.
2. Hands
The hardest problem in generative video, unsolved at the time of writing. Fingers multiply, merge, bend the wrong way, pass through the object.
Mitigation: do not ask for hands. If a hand must appear, have it already in the first frame and ask for almost no movement from it. A hand resting still is achievable. A hand picking something up is a coin flip at best.
3. Physics that nearly works
Liquid that pours slightly wrong. Fabric that moves like a solid. A dropped item that falls at the wrong speed. Each individually subtle, collectively giving the clip a dreamlike quality viewers notice without being able to name.
Mitigation: prefer camera motion over subject motion. A slow dolly across a static product is simple geometry and models handle it well. A product being poured is physics and they do not.
4. The morph
The clip simply becomes something else. The bottle grows a second cap. The background reorganises itself. Usually a sign the prompt asked for more change than the model could deliver coherently.
Mitigation: ask for less. The single most reliable way to improve a generation is to reduce how much you asked it to change.
Writing a prompt that works
The intuitive prompt is a description of a scene. The effective prompt is a description of a camera.
Weak: "A woman holds the serum bottle and smiles while applying it to her face."
Four separate hard problems — a hand, a face, an interaction, a sequence.
Strong: "Slow dolly in on the serum bottle on a marble surface. Soft morning light from the left. Shallow depth of field, background falls out of focus. Product stays centred and still."
One easy problem: move the camera.
A usable prompt generally names:
- Camera move — dolly in, pan left, slow orbit, static with drift
- Speed — slow, gradual, subtle
- Light — direction and quality
- Depth — shallow or deep focus
- What must not change — "product stays centred, label faces camera"
That last one does real work. Stating what should stay fixed constrains the model's freedom in exactly the place where its freedom hurts you.
What makes a good first frame
Because the first frame is the only part you control, it deserves most of your attention.
- Product large in frame. Small products drift more, because they occupy fewer pixels for the model to anchor on.
- Clean separation from the background. A product that blends into its surroundings will be partly re-invented.
- Sharp focus on the label. A soft label becomes a wrong label.
- Already composed. If the first frame is not a good photo, no amount of motion makes it a good video. Fix the still first.
- Right aspect ratio. Generate at 9:16 if the ad is vertical. Cropping afterwards wastes a third of what you paid for.
A workflow that does not waste money
The expensive mistake is generating video repeatedly until something works. Video costs several times what an image costs, so iterating there is the slowest and priciest path to a result.
The cheap-first sequence:
- Get the still right. Generate or shoot it. Iterate here, where each attempt costs very little.
- Check the product at full size. Zoom in on the label. If it has drifted at this stage, it will drift further in motion.
- Generate four seconds with a conservative camera move.
- Watch for drift, not for beauty. Scrub to the last frame and compare the product against the first. That is the test.
- Only then extend or get more ambitious.
Tools that let you hand a generated image straight into image-to-video without downloading and re-uploading make this loop fast enough to actually follow. Tools that do not will quietly push you into iterating on video, which is where budgets go.
When image-to-video is the wrong tool
Be honest about these:
- A step-by-step demonstration. Multiple interactions in sequence. Film it, or assemble it from stills with text.
- Anything over ten seconds. Generate several short clips and cut them together instead.
- Precise on-screen text. Add it in an editor afterwards. Models cannot spell reliably.
- An exact brand-guideline match. If legal has to approve the shade of the logo, a generative model is the wrong instrument.
Realistic expectations
A good image-to-video result is a four to six second clip with modest, believable motion that makes a still photo feel alive. That is genuinely valuable: it is the difference between a static listing image and something that holds attention in a feed, for a fraction of the cost of filming.
It is not a replacement for a shoot, and the tools that imply otherwise are selling the demo rather than the daily experience.
Trying it
The Playground does image-to-video directly: upload a still, describe the motion, pick a length and resolution. A generated image can be sent through to the motion step without a download in between, which is the loop described above.
If you want the whole ad rather than one clip — script, several shots, audio — the product-to-video path starts from a product photo or listing URL and carries the product reference through every shot.
New accounts get credits to generate a set of first frames without entering a card, which is enough to find out whether your particular product survives the process. That is the only test that matters, and it is specific to your packaging.
The short version
The first frame is the only thing you control, so spend your effort there. Ask the camera to move, not the product. Keep it short. Check the label at the end of the clip, not the beginning. And iterate on images, because video is where the money goes.