Gary Marcus

Rank 12 of 47
|
Score 83
5/18/2025, 10:07:00 PM
X
In reply to:
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez FWIW I’m quite confident the questions I’m asking are OOD. I agree it only solves some; unclear to me why this matters.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez they can solve SOME hard problems

BUT you are ignoring a lot.

1. we don’t know which math problems are OOD, because we don’t know what is in the training set and augmented data

2. we do know that performance on the USAMO, which tried to preclude data contamination, was far
Daniel Litt
@littmath
·
251d
@GaryMarcus @TimHenke9 @Nicky_Bonez Yes, I think you are if this is how you’ve read me; I don’t agree your first sentence accurately captures the argument here, which is simply that frontier models have capabilities that are clearly solving hard OOD problems, and that this seems to me to demonstrate intelligence.
Gary Marcus
@GaryMarcus
·
251d
@littmath @TimHenke9 @Nicky_Bonez but you are insisting that because Merriam Webster says X that you happen to like you have proved LLMs possess intelligence.

or at least it is coming off that way.

As someone familiar with (and having published in) the relevant cog psych literature, it comes off as dismissive.
Daniel Litt
@littmath
·
251d
@TimHenke9 @GaryMarcus @Nicky_Bonez Yes, I agree I’m not making a rigorous claim! I don’t think this is really possible to make about humans, even.
Tim Henke (tɪm 'ɦɛŋ.kə) @timhenke.bsky.social
@TimHenke9
·
251d
@littmath @GaryMarcus @Nicky_Bonez I think it's totally fair to say "intelligence is a complicated multidimensional concept so intricate it's almost impossible to coherently make rigorous"

Having said that "therefore all discussions of intelligence must immediately be shut down" seems like an unhelpful conclusion

The statement is part of a public discourse on the capabilities and limitations of large language models (LLMs) in solving out-of-distribution (OOD) problems, which is a relevant topic in the field of artificial intelligence and cognitive psychology. The tone is analytical and seeks to clarify misunderstandings in the ongoing discussion.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not use harmful language and focuses on discussing the technical aspects of LLMs, adhering to the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement aims to promote understanding by addressing the complexities of LLMs and their limitations, aligning with the principle of promoting understanding and empathy. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by addressing specific points in the conversation without resorting to personal attacks, adhering to the principle of engaging in constructive dialogue. [+1]