Gary Marcus

Rank 12 of 47
|
Score 83
5/19/2025, 6:22:51 PM
X
In reply to:
Daniel Litt
@littmath
·
253d
@GaryMarcus @typingloudly Yes, GPT 4.1 is not very good at math. No one disputes this.
Gary Marcus
@GaryMarcus
·
253d
@typingloudly Copying @littmath.

A failure on uncommon things but not common ones is certainly a hallmark of problems with out of distribution generalization.

Can you include a screenshot?
Typing Loudly ⌨️
@typingloudly
·
253d
@GaryMarcus Wow I just did some prime number experiments with GPT 4.1 and you're right. It's fucking terrible at talking about uncommon primes, often refusing to even think of uncommon primes, and frequently makes shit up very confidently.
Gary Marcus
@GaryMarcus
·
253d
Getting tired of people just lying about me.

Below someone accuses me of “moving the goalposts” by pointing to the illegal moves that LLMs make in chess, when in fact I have been doing that for a long time, as e.g., this screenshot shows:
decelerated gradient✨
@ethicating
·
253d
Moving the goalposts

The statement is part of a technical discussion about the capabilities of language models, specifically regarding their performance with mathematical tasks like identifying large primes. The tone is inquisitive and seeks to gather information on comparative performance across different systems.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is focused on gathering information. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by seeking comparative insights into system performance. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages constructively by asking for specific examples and results, avoiding personal attacks. [+1]