Gary Marcus

Rank 20 of 47
|
Score 39
2/18/2025, 8:22:04 AM
X
In reply to:
Ethan Mollick
@emollick
·
271d
@GaryMarcus There are so many things that might be wrong with the rushed post-training, etc. that I have no idea what the ceiling might be, but they got a top-performing non-reasoner by scaling up pre-training, which suggests there is some juice still in pre-training, though at great cost.
Gary Marcus
@GaryMarcus
·
271d
@emollick Putting it slightly differently, how much scaling of pretraining were you and are you anticipating would get us to a model people would be comfortable calling GPT-5? I don’t think Grok 3 is at that level, despite being 15x Grok 2.
Gary Marcus
@GaryMarcus
·
271d
@emollick Are you saying that the new data fit eg the original Chinchilla scaling laws with original exponents?

Or something much weaker, like “returns may be diminishing but there are some continued improvements with 15xing compute, at least on cherry picked benchmarks?”

Could you…
Ethan Mollick
@emollick
·
271d
Based on the early stats, looks like Grok 3 base is going to be a very solid frontier model (leads Chatbot Arena), suggesting pre-training scaling law continues with linear improvements to 10x compute

No Reasoner, yet (one is coming?) so GPQA scores are still below o3-mini (77%)

The statement critiques the effectiveness of scaling in AI models, highlighting ongoing qualitative issues like hallucinations and errors. It suggests skepticism about the benefits of increased scaling, despite improvements in benchmarks.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm but raises concerns about AI model limitations.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    It promotes understanding by discussing technical challenges in AI development. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism without personal attacks. [+1]