1. What the check found
Between 24 September and 12 October 2026 Herofab generated 311 product images from 212 product photos, across 14 tools. 303 of them were read by the automated check that runs before every delivery. It flagged 63, or 20.8%; the other 240 (79.2%) passed.
A flagged image is not delivered. The job is generated once more, and if the second image is flagged as well, nothing is delivered and the customer is not charged.
The most common fault was an invented detail: a print, label, pocket or seam that the product photo does not show. It appeared in 29 images, 46.0% of the flagged ones. One image can have more than one fault, so the rows add up to more than 63.
| Fault | What it means | Images | Share of the 303 checked | Share of the 63 flagged |
|---|---|---|---|---|
| Invented detail | A print, logo, label, pocket or trim that the product photo does not show | 29 | 9.6% | 46.0% |
| Unnatural look | A plastic or glassy surface, a visible mannequin, an obvious edit seam | 17 | 5.6% | 27.0% |
| Different product | The generated product is not the one in the photo | 12 | 4.0% | 19.0% |
| Collar or neckline changed | A shirt collar became a crew neck, a hood appeared, and similar | 11 | 3.6% | 17.5% |
| Inside label showing | The neck or waistband label is drawn on the outside | 10 | 3.3% | 15.9% |
| Anatomy | Extra, missing or fused body parts, or an object that floats | 8 | 2.6% | 12.7% |
| Wrong number of people | Two people where one was asked for, or nobody where a model was expected | 7 | 2.3% | 11.1% |
| Wrong text or logo | Letters or a logo on the product changed | 5 | 1.7% | 7.9% |
| Product cut off or badly framed | The product is cut by the image edge, off-centre or too small | 2 | 0.7% | 3.2% |
| Collage instead of one photo | Several panels or an inset instead of a single photograph | 2 | 0.7% | 3.2% |
| Missing print | The front print of the garment is gone | 1 | 0.3% | 1.6% |
| Watermark or overlay | A watermark, web address or caption on the image | 1 | 0.3% | 1.6% |
2. Product, text and collar
Product match: in 238 of the 303 images (78.5%) the product matched the photo fully, in 52 (17.2%) it showed small differences, and in 13 (4.3%) it was a different product.
Text and logos: 90 images had readable text on the product. It matched in 53 (58.9%), showed small differences in 32 (35.6%) and was wrong in 5 (5.6%).
Collars: 154 images showed a collar or neckline that could be compared with the photo. It had changed in 12 (7.8%).
These counts are the judge's raw answers. An answer becomes a flag only in a tool where that question is part of the check: Seamless pattern is not checked for product match, and Detail shot is not checked for the collar. That is why the product and collar counts here are one higher than their rows in the table above.
3. By tool
A percentage is given only where at least 30 images were checked; smaller groups say little. Background removal and upscaling have no model judge and are not among the 303.
| Tool | Images checked | Flagged | Share | Most common faults |
|---|---|---|---|---|
| Virtual try-on | 115 | 26 | 22.6% | invented detail (18), inside label showing (10), different product (7) |
| Canvas | 38 | 8 | 21.1% | invented detail (4), collar or neckline changed (3), different product (1) |
| Create model | 33 | 2 | 6.1% | wrong number of people (2), collage instead of one photo (2), watermark or overlay (1) |
| Product studio | 30 | 6 | 20.0% | anatomy (3), invented detail (3), different product (1) |
| Ghost mannequin | 18 | 9 | not given | unnatural look (7), product cut off or badly framed (2), different product (2) |
| Colour variants | 17 | 2 | not given | invented detail (2), different product (1) |
| Change pose | 15 | 1 | not given | invented detail (1) |
| Seamless pattern | 13 | 0 | not given | none |
| Extend frame | 8 | 5 | not given | unnatural look (4), anatomy (1) |
| Retouch | 8 | 4 | not given | wrong text or logo (3), unnatural look (2) |
| Detail shot | 5 | 0 | not given | none |
| Fabric studio | 3 | 0 | not given | none |
Virtual try-on by product type
Within Virtual try-on, tops were the largest group: 58 images, 12 flagged (20.7%). Trousers and other bottoms were the weakest, with 8 of 11 flagged, almost always for an invented detail or an inside label drawn on the outside. Jewelry had 3 of 20 flagged, footwear 1 of 7, bags 0 of 4 and eyewear 0 of 4.
| Product type | Images checked | Flagged | Share |
|---|---|---|---|
| Tops | 58 | 12 | 20.7% |
| Jewelry | 20 | 3 | not given |
| Trousers and other bottoms | 11 | 8 | not given |
| Dresses and other one-pieces | 11 | 2 | not given |
| Footwear | 7 | 1 | not given |
| Bags | 4 | 0 | not given |
| Eyewear | 4 | 0 | not given |
4. What an automated check misses
The check is mostly right when it flags. Of 18 flagged images compared with the product photo by eye, 16 showed the fault and 2 were unclear. None was plainly wrong.
It does not see cropping. Of 19 scene images checked by eye, 9 cut the product off at the edge of the frame: a sleeve, a trouser leg, the sides of a jacket. The automated check called the framing of all 9 acceptable.
The practical reading for anyone publishing AI product images: a vision model can be asked whether the product changed, and it answers well. Whether the whole product is inside the frame still needs a person, or a rule in the generation prompt.
5. How this was measured
The images are real generations from Herofab's own test cases, made while the product was being built. Several product photos were generated more than once, with different settings.
The check is the automated judge that runs in the product before an image is delivered: a vision model reads the product photo and the generated image and answers a fixed list of questions, such as whether there is a watermark, how many people are shown, whether the product matches and whether the collar changed. Virtual try-on of clothing gets a second judge for print placement and inside labels.
Five ad images were left out, because their headline is typeset after the check and the files measured here already carried it. A generation took 27.4 seconds at the median, and 90% finished within 41.8 seconds.
6. Limits
This is Herofab's test set, not customer work and not an industry average. It includes deliberately hard inputs and images made with earlier versions of the generation prompts.
A flag is the judge's reading, not a human one. The judge misses faults, cropping above all, and the sample checked by eye was small: 18 flagged images.
Small groups say little. Eight images of one tool cannot carry a percentage, so none is given for them.
Checklist
- Details that are not on the product: compare prints, labels, pockets, seams and trims with the photo.
- The product itself: colour, material, sleeve length and overall shape.
- Collar and neckline, especially when the photo shows the back of the garment.
- Labels and print side: an inside label drawn on the outside, a back print on the front.
- Hands and bodies: count the fingers and check which arm a hand belongs to.
- Text and logos: read every letter at full size.
- The frame edges: no sleeve, leg, strap or corner cut off. The automated check did not catch this.
- People and panels: one person, one photograph.
Updated 12 October 2026
Free to reuse under CC BY 4.0. Credit: Herofab (herofab.app).