Gary Marcus

Rank 13 of 47
|
Score 83
5/18/2025, 12:19:46 AM
X
In reply to:
Colin Fraser
@colin_fraser
·
241d
@GaryMarcus @littmath @MelMitchell1 @rao2z This is why I find the arithmetic problems thing informative. I know exactly how many 40 digit sums there are, and I know that the vast vast vast majority of them must not be in the training set. So if it can do them pretty reliably, it *must* be generalizing.
Gary Marcus
@GaryMarcus
·
241d
@littmath @colin_fraser @MelMitchell1 @rao2z I want to see it in domains where you can’t brute force with a ton of verifiable data augmentation.

Or in those augmentation domains, if we can actually see the training set and hence discuss in an informed way what OOD even is.

You can’t know for sure what is OOD if you don’t
Daniel Litt
@littmath
·
241d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z What would demonstrate reliable OOD for you? A top score on the next USAMO?
Gary Marcus
@GaryMarcus
·
241d
@littmath @colin_fraser @MelMitchell1 @rao2z My point there is that there is not reliable OOD.

There is not.
Daniel Litt
@littmath
·
241d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z Ah, sorry I misunderstood. FWIW I think the article you linked is a reasonable example of my complaint—you link to a lot of papers (e.g. the 20x20 multiplication task) which show a mix of successes and failures in OOD tasks and pull out only the failures.
Gary Marcus
@GaryMarcus
·
241d
@littmath @colin_fraser @MelMitchell1 @rao2z NOOOO!

I did not ask you why MY writing is unpersuasive.

I asked you to look at large body of work and you blew it off and personalized it.

The statement is part of a technical discussion on the limitations of AI in performing arithmetic tasks, specifically focusing on out-of-distribution (OOD) generalization. It engages with public issues related to AI capabilities and their implications.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm; it is a technical observation about AI limitations. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by discussing the challenges of AI generalization. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the technical aspects of the discussion. [+1]