Gary Marcus

Rank 11 of 47
|
Score 111
6/8/2025, 1:28:01 PM
X
In reply to:
Antonio Valerio Miceli Barone
@AVMiceliBarone
·
41d
@GaryMarcus @shinboson @rao2z I'm all for investigating the limitations of LLMs, but this paper is just showing performance limits, which aren't even worse compared to typical human performance.
Antonio Valerio Miceli Barone
@AVMiceliBarone
·
41d
@GaryMarcus @shinboson @rao2z The solution of 10-disk Tower of Hanoi is 1023 moves long. How many humans can solve the problem in their head and give you the correct sequence, without using any physical prop to track the state?
Gary Marcus
@GaryMarcus
·
41d
@shinboson It’s not a trivial test; it’s failure modes are not in fact same as people (e.g. a bright teenager can easily learn to do 8 discs), and if you can trviially implement with a scratchpad, you should do so and demonstrate it. you should also read the related work by @rao2z that i
𝞍 Shin Megami Boson 𝞍
@shinboson
·
41d
@GaryMarcus 1) what this paper evaluates is very specifically whether LLMs are able to perfectly *execute* a formal algorithm without any sort of scratchpad or tooling like symbolic reasoning

2) this paper determines that LLMs are bad at this, which is a trivial result and also so are we
Gary Marcus
@GaryMarcus
·
42d
See you later, LLM’s?

Weekend long read on how the new Apple paper spells trouble, though not quite the end:

[The link you need to type is in bottom left of the image, because Elon]

The statement and conversation engage in a discussion about the limitations and capabilities of large language models (LLMs) in comparison to human reasoning. The tone is analytical and critical, focusing on the performance of LLMs in executing formal algorithms without external aids. The conversation includes references to a paper and a Substack article, indicating a deeper exploration of the topic.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The discussion aims to critically evaluate LLMs without resorting to harmful language, maintaining a focus on technical limitations. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The conversation promotes understanding by discussing the nuances of LLM capabilities and limitations, encouraging a deeper look into the subject. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The dialogue is constructive, with participants engaging in a reasoned debate about the performance of LLMs compared to humans. [+1]