Gary Marcus

Rank 14 of 47
|
Score 83
4/23/2025, 9:19:37 PM
X
In reply to:
Professor Bad Trip 🔻
@_profbadtrip
·
271d
@GaryMarcus @jacobandreas It's a new test set, ARG-AGI 2.
Gary Marcus
@GaryMarcus
·
271d
Hold up – did the @jacobandreas lab score better on ARC in November than o3-medium did this week?

Why is that not even being discussed?

(arXiv, November 11).
Mike Knoop
@mikeknoop
·
272d
ARC-AGI o3 retest results are in!

Takeaway: o3 (medium) is the industry leading AI reasoning system by large margin. 2X score and 1/20 cost compared to next leading chain-of-thought system as measured by ARC v1 semiprivate set scoring 57% for $1.5/task.

On v2, o3 (medium)

The statement is part of a technical discussion about AI test results, specifically comparing performance metrics of different AI models on the ARC-AGI test. The tone is inquisitive and seeks clarification from experts, indicating a collaborative intent to understand the results better.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is focused on understanding technical details. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by seeking clarification and engaging with experts. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by asking for help and clarification without personal attacks. [+1]