Gary Marcus

Rank 13 of 47
|
Score 83
5/17/2025, 11:48:00 PM
X
In reply to:
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z I replied about USAMO here. I just don’t understand your argument; why is poor performance on USAMO indicative of non-intelligence if they perform well on other OOD tasks?
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z They’re hard proof questions, which are indeed difficult for frontier models. There’s nothing to explain.
Gary Marcus
@GaryMarcus
·
239d
@littmath @colin_fraser @MelMitchell1 @rao2z Dude. Aside from that being ad hominem and a bit insulting, it fails to address the USAMO result and the work by multiple authors. as such it is a pretty weak reply.

be better.
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z I’m in a couple of things you’ve written, but the only things reported therein are my negative comments on AI capabilities. I also found your reporting of FrontierMath to be incomplete based on my own experience with it. These experiences have affected how I read your writing.
Gary Marcus
@GaryMarcus
·
239d
@littmath @colin_fraser @MelMitchell1 @rao2z I pointed to many works: you named none, and refuted zero specific arguments.

THAT is unpersuasive.
Daniel Litt
@littmath
·
239d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z I have read it and found it unpersuasive.
Gary Marcus
@GaryMarcus
·
239d
@littmath @colin_fraser @MelMitchell1 @rao2z I think that they can do a limited form of analogical like reasoning in some cases but that their comprehension remains superficial, and that there are still vast numbers of basic problems they fail if trivial things like variable names are changed.

I don’t get the sense that

The statement is part of a public discourse on the performance of AI models on specific mathematical tasks, particularly the USAMO, and the implications for AI's capabilities. The discussion involves multiple participants debating the significance of AI performance on out-of-distribution (OOD) tasks and data contamination issues.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm directly, but it does not actively strive to avoid harm either. It is neutral in this regard.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement attempts to clarify a point about data contamination and its relevance to OOD performance, which can promote understanding. However, it lacks empathy or compassion in its delivery. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in dialogue by addressing a specific point about data contamination and OOD performance. It does not engage in personal attacks or ad hominem arguments, maintaining a focus on the issue at hand. [+1]