Gary Marcus

Rank 13 of 47
|
Score 83
5/18/2025, 12:03:08 AM
X
In reply to:
Daniel Litt
@littmath
·
240d
@colin_fraser @GaryMarcus @MelMitchell1 @rao2z But I have seen Gemini in particular excel at problems with some casework etc.
Daniel Litt
@littmath
·
240d
@colin_fraser @GaryMarcus @MelMitchell1 @rao2z I don’t really have a good model for what they’ll be good and bad at. Certainly if you ask them to do a huge number of arithmetic operations in a row you’re probably in trouble.
Colin Fraser
@colin_fraser
·
240d
@littmath @GaryMarcus @MelMitchell1 @rao2z I think we’ve talked about this before and you disagreed with me but I’ll ask again; I’ve found the hallmark of problems that they do badly at is they involve a lot of “busywork”—trying many cases, finding the correct path out of many possible options, etc. Do you find this?
Daniel Litt
@littmath
·
240d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z Let me put it this way—they can very reliably solve some out-of-distribution hard problems that humans can’t reliably solve. Being able to solve *all* problems is not a pre-condition for intelligence; otherwise people aren’t intelligent.
Gary Marcus
@GaryMarcus
·
240d
@littmath @colin_fraser @MelMitchell1 @rao2z Some, but not reliably. They can’t even solve large multiplication problems reliably.

LLMs exceed their predecessors but demonstrably still have not solved this problem, which is most clear when the training set is known, and when direct data augmentation is not possible.

Some
Daniel Litt
@littmath
·
240d
@GaryMarcus @colin_fraser @MelMitchell1 @rao2z They’re clearly generalizing out of distribution. E.g. they can solve some novel math problems.

The statement is part of a technical discussion about the capabilities of AI models, specifically in solving mathematical problems. It seeks clarification on the term 'casework' within this context, indicating an engagement with the topic of AI performance and limitations.

  1. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement seeks to promote understanding by asking for clarification, which can lead to a more informed discussion. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by asking a question rather than making assumptions or personal attacks. [+1]