Gary Marcus

Rank 19 of 47
|
Score 64
6/17/2025, 7:28:49 PM
X
In reply to:
Gorgi Kosev
@spion
·
20d
@GaryMarcus @AxelMtbr @rohanpaul_ai Just to clarify: I use LLMs and I (personally) think they are amazing in many ways (especially for learning). But the frustrating capability variability (and the error rates) make me think we might all have a better time if there was better data transparency.
Gary Marcus
@GaryMarcus
·
20d
@spion @AxelMtbr @rohanpaul_ai quoting this at the end of my next substack (posting shortly)
Gorgi Kosev
@spion
·
20d
@AxelMtbr @GaryMarcus @rohanpaul_ai They are decent solvers of already solved problems indeed.
Antidote
@AxelMtbr
·
20d
@spion @GaryMarcus @rohanpaul_ai Perhaps that is correct? Although I wonder how much you can really optimise for some very specific tests.

Anecdotally, the models have gotten better in real world as well as benchmarks / tests.
Gorgi Kosev
@spion
·
20d
@AxelMtbr @GaryMarcus @rohanpaul_ai I think that LLMs suffer severely from (benchmark/test) data contamination which is hindering ability to measure real progress.
Antidote
@AxelMtbr
·
20d
@spion @GaryMarcus @rohanpaul_ai Im sure that's one mechanism besides just general scaling - but identifying gaps, creating data about these gaps and then fixing them is not necessarily a bad thing, right?

The statement is part of a broader conversation about the capabilities and limitations of large language models (LLMs), specifically addressing issues like data transparency and error rates. It links to a Substack article that appears to provide updates or rebuttals related to a paper on Apple's reasoning capabilities. The tone is lighthearted, as indicated by the mention of a joke, but the context involves substantive engagement with public issues related to AI technology.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm and maintains a lighthearted tone, aligning with the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by engaging in a discussion about LLMs, though the mention of a joke may slightly detract from a serious tone. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement is part of a constructive dialogue about LLMs, without personal attacks or ad hominem arguments. [+1]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement uses the platform to engage in a discussion about AI, which is a significant public issue, thus upholding the principle of responsible use of free speech. [+1]