Gary Marcus

Rank 12 of 47
|
Score 83
5/19/2025, 12:30:19 AM
X
In reply to:
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez Ok, if we agree on this, it would help if you say why you don’t think high performance on OOD math questions isn’t evidence for intelligence.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez ps, put differently, we roughly agree on the facts for the math questions, but disagree on what lessons you can draw from there.
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez FWIW I’m quite confident the questions I’m asking are OOD. I agree it only solves some; unclear to me why this matters.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez they can solve SOME hard problems

BUT you are ignoring a lot.

1. we don’t know which math problems are OOD, because we don’t know what is in the training set and augmented data

2. we do know that performance on the USAMO, which tried to preclude data contamination, was far
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez Yes, I think you are if this is how you’ve read me; I don’t agree your first sentence accurately captures the argument here, which is simply that frontier models have capabilities that are clearly solving hard OOD problems, and that this seems to me to demonstrate intelligence.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez but you are insisting that because Merriam Webster says X that you happen to like you have proved LLMs possess intelligence.

or at least it is coming off that way.

As someone familiar with (and having published in) the relevant cog psych literature, it comes off as dismissive.

The statement is part of a public discourse on the topic of artificial intelligence and its ability to solve out-of-distribution (OOD) math problems, which relates to broader discussions on AI capabilities and intelligence. The tone is analytical and seeks to clarify a point of contention in the ongoing debate.

  1. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement attempts to promote understanding by clarifying that the evidence is not definitive, which can help in fostering a more nuanced discussion. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by referencing previous discussions and providing reasons for the stance taken, without resorting to personal attacks. [+1]