Gary Marcus

Rank 16 of 47

Score 64

@woody_says @emollick Ethan lost some credibility with me with this tweet. Clear evidence of diminishing return on pure pretraining, which was what my original deep learning is hitting a wall paper was about.

2/18/2025, 7:57:02 AM

In reply to:

Woody Says Hello

@woody_says

272d

@emollick Same results with more compute

Ethan Mollick

@emollick

272d

So no wall so far (not that we were expecting an overall wall given the rise of test-time compute), and the moats appear to be the usual: speed of execution, good partnerships and ecosystems, CapEx (I suspect the labs also believe speed to AGI and flywheels from AI development)

The statement is part of a technical discussion on the limitations and advancements in deep learning, specifically addressing the concept of diminishing returns on pretraining models. It critiques a previous statement by expressing a loss of credibility in the original poster's argument, suggesting a disagreement on the interpretation of data or research findings.

Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages in constructive criticism by addressing a specific point of disagreement regarding the interpretation of deep learning advancements. It does not resort to personal attacks or ad hominem arguments, maintaining a focus on the technical aspects of the discussion. [+1]