Gary Marcus

Rank 14 of 47
|
Score 83
3/6/2025, 7:02:17 PM
X
In reply to:
Masen Dean
@masenmakes
·
301d
@GaryMarcus @emollick @_alexreisner Gary, last time you replied to me, you said you didn't use LLMs much due to them not being useful

If you're going to use vibe tests as an argument just after also discounting benchmarks, I hope you at least touch the technology you grandstand about

I request this in good faith
Gary Marcus
@GaryMarcus
·
302d
@emollick @_alexreisner and then there are many of the issues I raise here, and the fact that Karpathy found a lot of people didn’t even prefer GPT 4.5 to a prior model:
Gary Marcus
@GaryMarcus
·
302d
Ouch. Ezra Klein left out a LOT of important things yesterday, in his podcast with Biden’s AI czar Ben Buchanan. Here’s a partial list, excerpted from my new essay responding with what I think Klein got wrong.

• Reports from The Wall Street Journal and Fortune that Project…
Gary Marcus
@GaryMarcus
·
302d
@emollick @_alexreisner https://www.theatlantic.com/tech…
Gary Marcus
@GaryMarcus
·
302d
@emollick You are looking at a single measure with (just) 198 questions and a serious possibility of data contamination, not to mention unknown data augmentation techniques. I think your conclusion is premature.

Have you read e.g., the article today in the Atlantic by @_alexreisner?
Ethan Mollick
@emollick
·
302d
So it looks like the First Scaling Law (the bigger the model the “smarter”) still holds- order of magnitude increases in compute lead to linear improvements in ability

GPT-3.5 Turbo scored 30% on GPQA, GPT-4 Turbo got 47%, now GPT-4.5 got 70%

And Reasoners add a new Scaling Law

The statement engages in a discussion about the utility and evaluation of AI models, specifically Grok 3 Beta, and addresses concerns about the author's personal experience with the technology. It acknowledges that while the tool may not be useful to the author, it could be beneficial to others, thus maintaining a balanced perspective.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement strives to do no harm by clarifying that the author's experience is subjective and not a universal judgment. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    It promotes understanding by acknowledging the potential usefulness of the technology to others, despite personal dissatisfaction. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by addressing a previous comment and providing a reasoned response without personal attacks. [+1]