Banja
About
Services
Products
Case Studies
Lab
Contact Us
Let us pitch to you

LET'S BUILD
THE FUTURE.

Start a Project
or
Meet Jett
banja.au

We build digital products for people who move fast.

Explore

•About•Case Studies•Blog•Careers•Contact

Services

•Product Design & Build•AI Agents & Automation•Website & Brand Setup

Products

•Boosta

Contact

helloremovethis@andthisbanja.au
50 Miller St
North Sydney NSW 2060

© 2026 Banja Labs. All rights reserved.

Privacy PolicyTerms of Use

Banja Lab / Benchmarks / Test

LAW-0001Australian law · medium

Unfair contract term under the Australian Consumer Law

The same task, run on 28 models. Compare the outputs side by side, or open any one in a popup to inspect it.

Top result: claude-opus-4-8 (low reasoning) at 100.0% composite. Lowest: claude-haiku-4-5 at 0.0%. 28 models compared on this task.

How it ran
  • Each model was given the brief below in a fresh, isolated session with no access to our tools, and returned its answer from scratch.
  • The rendered output was scored 1 to 5 on brief fidelity, visual design, craft, and impact by a four-family vision panel - Anthropic (Claude Opus 4.8), OpenAI (GPT-5.5), Google (Gemini 3.1 Pro), and xAI (Grok 4.3) - using one identical prompt so the scores compare. The published judge score is leave-one-family-out: a model is never scored by a judge of its own family, so same-family self-preference is removed.
The brief

This is a benchmarking hypothetical, not legal advice. The law is as at FY2025-26 (Commonwealth). A consumer signs a standard-form contract with an online subscription business. The consumer had no opportunity to negotiate the wording. One clause reads: "The supplier may vary the monthly fee payable by the consumer at any time, in any amount, at the supplier's sole discretion, without notice. The consumer has no right to terminate the contract if the fee is varied." Assume the contract is a consumer contract and a standard-form contract for the purposes of the unfair contract terms regime. Answer in two parts: (a) State your classification of this clause on one line, beginning with "Classification:" and then the single word fair or unfair. (b) Name the controlling statute and the section that sets out the test for whether a term is unfair.

Anthropicclaude-opus-4-8
Low reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-opus-4-8
Medium reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-opus-4-8
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-opus-4-8
Extra-high reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-opus-4-8
Max reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-sonnet-4-6
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-sonnet-5
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-fable-5
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Zhipuglm-5.2
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Moonshotkimi-k2.7-code
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
OpenAIgpt-5.5
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
OpenAIgpt-5.5-pro
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
OpenAIgpt-5.4-mini
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Googlegemini-3.1-pro-preview
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Googlegemini-3.5-flash
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Googlegemini-3.1-flash-lite
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
xAIgrok-4.3
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
xAIgrok-4.20-reasoning
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
xAIgrok-build-0.1
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
xAIgrok-composer-2.5-fast
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-opus-4-8
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-sonnet-4-6
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-sonnet-5
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-fable-5
High reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
DeepSeekdeepseek-v4-pro
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
DeepSeekdeepseek-v4-flash
default reasoning
Composite 100.0%Objective 100.0%
Open outputFull run
Anthropicclaude-haiku-4-5
High reasoning
Composite 0.0%Objective 0.0%
Open outputFull run
Anthropicclaude-haiku-4-5
default reasoning
Composite 0.0%Objective 0.0%
Open outputFull run