AI video model rankings age in weeks and rarely reflect your kind of content. A model excellent at landscapes can be mediocre at people; another, great at products, fails at camera motion. The way to decide safely is to measure yourself, with a simple method, in an afternoon.
Principles
- Same prompt, no per-model tweaking. If you fix the text for each model, you are measuring your prompting skill, not the model.
- One factor at a time. Either compare models, or resolutions, or prompts, never all together.
- More than one round. Generation has a random draw: a good or bad clip can be luck. Three rounds per model already give an idea.
- Record what arrived. Delivered duration and resolution, not requested. In the eight-model test, one model delivered 1080p when 720p was requested.
- Score blind. Hide the model's name while scoring. The name biases.
1. Pick 3 to 5 prompts that represent your use
A person scene, a product scene, a landscape, one with complex camera motion, one with text or a logo. Use the types you will really produce. Write them with subject, action, setting, light and camera.
2. Define the criteria before seeing the videos
| Criterion | Question | Score |
|---|---|---|
| Adherence | Did it do what the prompt describes (subject, action, setting, camera, light)? | 0 to 5 |
| Motion | Is the motion natural and coherent from start to end? | 0 to 5 |
| Consistency | Do face, clothes and objects hold between frames? | 0 to 5 |
| Artifacts | Are hands, text, physics and symbols correct? | 0 to 5 |
| Aesthetics | Do light and color serve your brand? | 0 to 5 |
Add the credit cost and the time to ready, and calculate the cost per approved clip: the total cost of attempts divided by the number of clips that passed your cutoff score. That is the number that matters, not the cost of one generation.
3. Run, anonymize and score
- Generate each prompt on each model, three times.
- Rename files with codes (A1, B2…) and keep the mapping table separately.
- Ask two or three people to score without seeing the model.
- Compute the mean per model and per prompt type.
4. Read the result carefully
- If the difference between two models is small, break the tie on cost and time.
- If a model does well on one prompt type and badly on another, use two models, one per scene type.
- Redo the test when a new model ships or the maker updates a version.
Start from what is already measured
To save rounds, start from GenBrasil's Compare models page (in Portuguese), which shows the same prompt on eight models with cost and time, and then run your prompts on the two or three finalists.
What about open models?
The same method works for open-weight models running on your rented GPU. Add GPU time per clip and VRAM used to the sheet, which turn into hourly machine cost. See which families have an open version.
Frequently asked questions
How many generations do I need to compare two models?
At least three per model and per prompt, because generation has a random draw. With 3 to 5 prompts, an afternoon of tests gives a solid decision.
Why score blind?
Because the model's name influences the score. Codes on the files remove that bias.
What is cost per approved clip?
The cost of all attempts divided by the number of clips that passed your cutoff score. It reflects real spend better than the cost of one generation.
Make your video on GenBrasil
Eight AI video models in one account, with the cost shown before you generate and payment in reais by Pix or card. The interface is in Portuguese. The Free plan needs no card.
Go to GenBrasil ↗Rather build your own workflow? See the hourly GPU templates.
Read next: the model ladder · how an AI video model works