Gary Marcus

Rank 15 of 47

Score 76

@davidad That is *exactly* what I said in The Algebraic Mind (2001) and Rethinking Eliminative Connectionism (1998).

Astonishing how little progress has been made on such a central problem in a quarter century.

Cc @garybasin

6/17/2024, 1:40:38 PM

In reply to:

davidad 🎇

@davidad

510d

When @GaryMarcus and others (including myself) say that LLMs do not “reason,” we mean something quite specific, but it’s hard to put one’s finger on it, until now. Specifically, Transformers do not
generalize
algebraic structures
out of distribution.

Gary Basin 🍍

@garybasin

510d

Why deep learning is ngmi in one graph

The statement engages in a substantive discussion about the limitations of current deep learning models, specifically transformers, in generalizing algebraic structures out of distribution. It references previous works and expresses concern over the lack of progress in this area over the past quarter-century. The tone is critical but professional, aiming to highlight a significant issue in the field of AI research.

Principle 1:
I will strive to do no harm with my words and actions.
The statement does not cause harm and is focused on a technical critique rather than personal attacks, adhering to the principle of doing no harm with words and actions. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
The statement promotes understanding by referencing past works and highlighting a central problem in AI research, thus fostering a deeper discussion on the topic. [+2]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages in constructive criticism by pointing out the lack of progress in a specific area of AI research without resorting to personal attacks or ad hominem arguments. [+2]