Gary Marcus

Rank 12 of 47
|
Score 83
5/18/2025, 10:08:20 PM
X
In reply to:
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez FWIW I’m quite confident the questions I’m asking are OOD. I agree it only solves some; unclear to me why this matters.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez they can solve SOME hard problems

BUT you are ignoring a lot.

1. we don’t know which math problems are OOD, because we don’t know what is in the training set and augmented data

2. we do know that performance on the USAMO, which tried to preclude data contamination, was far
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez Yes, I think you are if this is how you’ve read me; I don’t agree your first sentence accurately captures the argument here, which is simply that frontier models have capabilities that are clearly solving hard OOD problems, and that this seems to me to demonstrate intelligence.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez but you are insisting that because Merriam Webster says X that you happen to like you have proved LLMs possess intelligence.

or at least it is coming off that way.

As someone familiar with (and having published in) the relevant cog psych literature, it comes off as dismissive.
Daniel Litt
@littmath
·
251d
@TimHenke9 @GaryMarcus @Nicky_Bonez Yes, I agree I’m not making a rigorous claim! I don’t think this is really possible to make about humans, even.
Tim Henke (tɪm 'ɦɛŋ.kə) @timhenke.bsky.social
@TimHenke9
·
251d
@littmath @GaryMarcus @Nicky_Bonez I think it's totally fair to say "intelligence is a complicated multidimensional concept so intricate it's almost impossible to coherently make rigorous"

Having said that "therefore all discussions of intelligence must immediately be shut down" seems like an unhelpful conclusion

The statement is part of a public discourse on the capabilities of frontier models in solving out-of-distribution (OOD) math problems and the implications for understanding intelligence. The tone is analytical and seeks to clarify a point of disagreement regarding the interpretation of facts and lessons drawn from them.

  1. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by acknowledging agreement on facts while highlighting a disagreement on interpretations, which can foster a more nuanced discussion. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by focusing on the disagreement over lessons rather than attacking individuals, thus adhering to the principle of avoiding personal attacks. [+1]