Same insurance pivot table as the last video, same generic "*" query, no schema, but this time run through Sparrow's Advanced mode, backed by Gemma 4 31B Dense, instead of Standard mode Ministral 14B.
Result: Gemma 4 31B, despite being over twice the size, was less accurate on this pivot table than Ministral 14B was. It hallucinated on column values — assigning null to cells that had real values, then shifting the actual value into the next column instead. A column-alignment failure Ministral 14B didn't have.
The takeaway: model size isn't a proxy for fit. Structured extraction on dense tabular layouts seems to favor whatever model was better tuned for structure-following, not the model with more parameters. Worth testing both tiers against your actual documents before assuming "Advanced" means "more accurate."
No comments:
Post a Comment