Gary Marcus

Rank 13 of 47

Score 83

@littmath @colin_fraser @MelMitchell1 @rao2z Some, but not reliably. They can’t even solve large multiplication problems reliably.

LLMs exceed their predecessors but demonstrably still have not solved this problem, which is most clear when the training set is not known, and when direct data augmentation is not possible.

5/17/2025, 11:21:30 PM

In reply to:

Daniel Litt

@littmath

237d

@GaryMarcus @colin_fraser @MelMitchell1 @rao2z They’re clearly generalizing out of distribution. E.g. they can solve some novel math problems.

Gary Marcus

@GaryMarcus

237d

@littmath New situations is clearly the weakness in these approaches, and has been forever.

@colin_fraser has documented this endlessly, informally; I gave the intuitions behind that in 1998. The recent Apple reasoning paper, the studies of @MelMitchell1 and @rao2z etc reinforce it.

The

Daniel Litt

@littmath

237d

@GaryMarcus These seem fine.

Gary Marcus

@GaryMarcus

237d

@littmath Under some definitions of intelligence, maybe. What’s yours?

Daniel Litt

@littmath

237d

I don’t know how you can use frontier models and hold onto beliefs like this. Not sure about “conscious,” but for any reasonable meaning of “intelligent,” LLMs are already intelligent.

The statement critiques the capabilities of large language models (LLMs) in solving mathematical problems, particularly in novel situations. It acknowledges improvements over predecessors but highlights ongoing limitations, especially when training data is unknown or direct data augmentation is not possible.

Principle 1:
I will strive to do no harm with my words and actions.
The statement does not cause harm and focuses on technical critique.
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
It promotes understanding by discussing specific limitations of LLMs. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
Engages in constructive criticism without personal attacks. [+1]