Gary Marcus

Rank 13 of 47
|
Score 83
5/18/2025, 12:32:36 AM
X
In reply to:
Flowers
@flowersslop
·
241d
@GaryMarcus soon enough
Gary Marcus
@GaryMarcus
·
241d
@flowersslop GPT-5 doesn’t exist :)
Flowers
@flowersslop
·
241d
@GaryMarcus I’m not calling GPT-5 a breakthrough, nor dismissing it. Just saying: it’s built on models we haven’t seen yet. pretraining is, after all, still very much alive. But some have speculated it is just the current models with some clever autorouting and THAT is not the case.
Gary Marcus
@GaryMarcus
·
241d
!!!

If this true, it’s compelling evidence that pure pretraining scaling of LLMs has failed, and that that particular approach (which was I specifically said would hit a wall) has failed.
Flowers
@flowersslop
·
241d
I just got feedback from someone who def knows firsthand: GPT-5 will be built from entirely new models that haven’t been released yet. So we’re leaving GPT-4 and all its variants behind and getting a system made from components that aren’t part of anything currently available.

The statement is part of a technical discussion about the development of GPT-5 and its implications for AI models. It involves a back-and-forth exchange about the nature of AI advancements and the potential limitations of current models. The tone is informative and speculative, focusing on the technical aspects of AI development.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm as it is a technical discussion about AI models. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by engaging in a discussion about AI advancements and their implications. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the technical aspects of AI. [+1]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech and uses the platform responsibly to discuss AI developments. [+1]