Gary Marcus

Rank 16 of 47

Score 76

@VraserX link?

the main issue here is that the results on math don’t seem to generalize well to many domains, suggesting data contamination and/or heavy math-specific data augmentation (which is in easier in math than other domains because of verifiability etc).

by the way, computers

6/9/2025, 2:16:20 PM

In reply to:

VraserX e/acc

@VraserX

🚨 At a secret math summit, top researchers tried to outsmart AI with never-before-seen math problems, unpublished, never on the web, not in any training set.

Result?

OpenAI’s o4-mini solved most of them faster than the mathematicians themselves.

One researcher spent months

The statement critiques the generalizability of AI's performance in math to other domains, suggesting potential data contamination or domain-specific augmentation. It raises concerns about the validity of AI's capabilities across different fields.

Principle 1:
I will strive to do no harm with my words and actions.
The statement does not cause harm and raises a valid concern about AI's generalizability. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
It promotes understanding by questioning the limitations of AI, encouraging further investigation. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
Engages in constructive criticism by addressing potential issues without personal attacks. [+1]