Model cards stop predicting phone performance: Liquid AI open-sources Pipette across 1,000 device configurations — MarkTechPost
Liquid AI released Pipette under Apache 2.0, measuring model, quantization, runtime and device as one unit. The launch dataset covers more than 1,000 configurations across 30-plus models, with Artificial Analysis verifying the methodology.
Positive-sum angle: A public results dataset hands every small team the hardware evidence that only large OEMs used to hold. Chip vendors gain a neutral scoreboard, app builders stop guessing at decode throughput, and each submitted run makes the next procurement decision cheaper for everyone.
What's the impact: SEA product teams shipping on mid-tier Android, still the default handset from Jakarta to Manila, should benchmark the exact quantization and runtime before committing a sprint. Two same-size models retained 78.4% and 33.8% of decode throughput on identical hardware.