Gary Marcus

Rank 13 of 47
|
Score 83
4/30/2025, 1:12:56 AM
X
In reply to:
MindMirror
@TheMysteryDrop
·
257d
@GaryMarcus @samuelaxon How good do you think the average SWE is at this benchmark? AI doesn't need to hit 100% to soak up >50%.

Note that Zuck isn't claiming 100% of software will be AI generated - just the majority - and that there will be a role for top SWEs to cover the holes for the time being.
Gary Marcus
@GaryMarcus
·
257d
@TheMysteryDrop @samuelaxon driverless cars were (I imagine) greater than 80% correct a decade ago; the last bit is always hard.
MindMirror
@TheMysteryDrop
·
257d
@GaryMarcus @samuelaxon Given that (1) Sonnet 3.7 is at 48% success on this metric today, (2) and tool use is shown to provide a critical boost, and (3) new models like O3 are great at tool use, why would you be bearish on the capabilities in 12-18 months?
Gary Marcus
@GaryMarcus
·
257d
@TheMysteryDrop The debugging part of this is seriously implausible, see eg https://arstechnica.com/ai/2025/… by @samuelaxon, discussing a recent Microsoft study
MindMirror
@TheMysteryDrop
·
257d
@GaryMarcus Google is >25% today. Why is 50%+ within 12-18 months a stretch?
Gary Marcus
@GaryMarcus
·
257d
I call bullshit. But nobody will ever hold Zuck (or Dwarkesh) responsible, if this is turns out to be wrong like so many previous overhyped predictions.

The statement engages in a discussion about the future capabilities of AI in debugging compared to human engineers. It expresses skepticism about AI surpassing top engineers in a short timeframe, contributing to a broader debate on AI's role in software development.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive debate. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    It promotes understanding by questioning the rapid advancement claims, encouraging a realistic view of AI capabilities. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism without personal attacks, focusing on the plausibility of technological predictions. [+1]