Gary Marcus

Rank 12 of 47
|
Score 83
1/2/2025, 4:55:33 AM
X
In reply to:
Yagami Ackerman
@YagamiLovesDL
·
397d
@GaryMarcus Does this still holds "We found no evidence of formal reasoning in language models. Their behavior is better explained by sophisticated pattern matching" for o1 and other inference time models ??
Gary Marcus
@GaryMarcus
·
397d
Tells you a lot. Fits closely with Apple’s reasoning paper, and the work on distribution shift I did in the late 1990s, etc.

Same old problem, 27 years on.
elvis
@omarsar0
·
397d
o1-preview shows a whopping 30% reduction in accuracy when Putnam math problems are slightly variated.

Not sure about the reliability of the results but I think it's a good paper worth checking out to understand LLM robustness on complex math problems better.

A hot research…

The conversation involves a discussion on the robustness of language models, referencing a study on accuracy drops in mathematical problem-solving. It touches on the limitations of AI in formal reasoning, a topic of public interest in AI ethics and development.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and contributes to a technical discussion. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The conversation promotes understanding of AI limitations, fostering informed discourse. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The dialogue is constructive, focusing on data and research without personal attacks. [+1]