Gary Marcus

Rank 13 of 47
|
Score 83
5/17/2025, 11:35:33 PM
X
In reply to:
Colin Fraser
@colin_fraser
·
239d
@GaryMarcus @littmath @MelMitchell1 @rao2z On one hand the problems it solves correctly are ood. On the other hand so are the ones it doesn’t.
Colin Fraser
@colin_fraser
·
239d
@GaryMarcus @littmath @MelMitchell1 @rao2z Pretty reliable at adding pairs of numbers under 40 digits or so. Pretty unreliable above. This is evidence for both sides of this argument depending on how you frame it.
Gary Marcus
@GaryMarcus
·
239d
@littmath @colin_fraser @MelMitchell1 @rao2z What is it that you think they can solve very reliably?

I didn’t say all and would appreciate your not putting words in my mouth.
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z Let me put it this way—they can very reliably solve some out-of-distribution hard problems that humans can’t reliably solve. Being able to solve *all* problems is not a pre-condition for intelligence; otherwise people aren’t intelligent.
Gary Marcus
@GaryMarcus
·
239d
@littmath @colin_fraser @MelMitchell1 @rao2z Some, but not reliably. They can’t even solve large multiplication problems reliably.

LLMs exceed their predecessors but demonstrably still have not solved this problem, which is most clear when the training set is known, and when direct data augmentation is not possible.

Some
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z They’re clearly generalizing out of distribution. E.g. they can solve some novel math problems.

The statement is part of a technical discussion on the capabilities of AI models in solving mathematical problems, specifically focusing on their limitations with out-of-distribution (OOD) tasks and complex functions. The tone is analytical and contributes to a broader debate on AI capabilities.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and maintains a neutral tone, focusing on technical aspects rather than personal attacks. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by discussing specific limitations of AI models, which can foster a more informed dialogue about their capabilities. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by addressing the technical limitations of AI models without resorting to personal attacks. [+1]