Judging
How we judge
Recognition only means something if the judging behind it is credible. This is exactly how entries are reviewed, scored and selected — and where the line is drawn.
How JIMSO Judges
JIMSO exists to recognise exceptional digital and software products. That recognition only means something if the judging behind it is credible. This page explains exactly how entries are reviewed, scored and selected, and where we draw the line.
The short version: independent practitioners score every entry against a fixed rubric, at least two of them per entry, with conflicts declared and blocked, claims checked against evidence, and a high bar for finalists and winners. When the quality isn't there, we give no award.
An independent, practitioner-led jury
Entries are assessed by an independent jury of people who build, ship and evaluate digital products for a living: product leaders, designers, engineers, founders and operators. Judges are chosen for relevant domain experience, not title or profile.
Judging is separate from anything commercial. Buying an entry, sponsoring, or advertising with JIMSO has no bearing on how an entry scores. There is no paid placement, no upgraded review, and no way to purchase a better result. Fees cover the cost of running a rigorous review; they do not buy one.
Judges work to a shared rubric and calibration guidance so scores mean the same thing across reviewers and categories.
Conflict-of-interest controls
Before reviewing, every judge declares relationships that could compromise their objectivity: current or former employers, clients, investments, advisory roles, close personal ties and competing products.
Those declarations drive assignment. Known conflicts are blocked automatically, so a judge is never routed an entry connected to them. Where a conflict surfaces mid-review that the system didn't catch, the judge is required to recuse, and the entry is reassigned.
Judges do not contact entrants directly, and entrants cannot contact judges to plead a case. Any questions about an entry are handled through the JIMSO editorial team, so every submission is treated on the same terms. Deliberations and individual scores are kept confidential.
At least two reviewers per entry
No result rests on a single opinion. Every eligible entry receives at least two independent first-round reviews, carried out separately so one judge's view doesn't anchor another's.
Entries that clear the first round as finalist candidates go to a third, lead review. The lead reads the earlier scores and rationales, reconciles material disagreement, and confirms whether the entry genuinely meets the finalist bar. This is where close calls are resolved.
Scoring methodology
Each judge scores an entry on six weighted criteria, marking each from 1 to 10 and writing a short rationale for the mark. The weights turn those marks into a single score out of 100:
| Criterion | Weight |
|---|---|
| Problem significance and market relevance | 15 |
| Product quality and user experience | 20 |
| Innovation and differentiation | 20 |
| Evidence of impact and traction | 25 |
| Execution and scalability | 15 |
| Submission clarity and integrity | 5 |
The heaviest weight sits on evidence of impact and traction, because what a product has actually achieved matters more than how it presents. A rationale is mandatory for every criterion: a number with no reasoning behind it doesn't count, and unexplained scores are sent back.
Evidence requirements
Claims have to be supportable. Judges score what an entry can demonstrate, not what it asserts.
Strong submissions back their case with verifiable material: working links and demos, usage and performance metrics, adoption and retention figures, revenue or growth indicators, customer references and case studies, and third-party validation. Vague or unsupported claims are discounted. Where numbers are central to a claim, judges expect enough context to read them, such as the period covered and the basis for comparison.
Overstated or misleading claims don't just fail to help; they weigh against an entry under submission clarity and integrity.
Finalist thresholds and selectivity
The bar is fixed and applied consistently:
- Finalist: a weighted score of 75 or above.
- Seasonal winner: 82 or above.
- Annual winner: 85 or above.
Within any category, JIMSO recognises up to five finalists and a single winner. These are ceilings, not quotas. A category might produce five finalists, one, or none, depending entirely on what the entries earn.
A low finalist rate is deliberate. The thresholds are set so that reaching them says something real, and we would rather name fewer finalists than dilute what the recognition stands for.
The right to give no award
If no entry in a category meets the threshold, or the evidence behind the leading entries doesn't hold up, JIMSO names no winner, and no finalists, in that category.
This is a feature of the process, not a failure of it. A JIMSO award is a statement that the work cleared a demanding bar. We will not award one to fill a slot, and an empty category is simply an honest reflection that, this cycle, the bar wasn't met.
Annual judging
Seasonal results feed into a separate annual assessment; they don't roll up automatically. Each year, JIMSO re-evaluates the pool of seasonal finalists alongside annual-only qualifiers, judged fresh against the same rubric and the higher annual thresholds.
From that assessment, JIMSO names annual category winners: the standout products of the year in their fields. Across all categories, the strongest overall entry may be recognised as Product of the Year. As with every other stage, if the quality isn't there, the honour isn't given.