Gary Marcus

Rank 14 of 47

Score 83

@davidad 1. The y-axis is very weird; not using time per se, but how that maps onto tasks. I am sure e.g., you can find short tasks that people can do involving scene comprehension that machines still don’t do remotely well. @anh_ng8 has many.

2. Accepting the y axis for the sake of

4/24/2025, 1:15:31 PM

In reply to:

davidad 🎇

@davidad

270d

@GaryMarcus I’m sure you won’t approve of it, but I don’t mind

davidad 🎇

@davidad

306d

My takes:
1. I’d bet that synthetic-data self-improvement (e.g. RL on CoT with verifiable reward) is inducing a new growth regime (pictured).
2. 50% success rate corresponds to something like pass@2. With ungameable reward, pass@k becomes an issue of cost; pass@8 may be worth it.

Gary Marcus

@GaryMarcus

271d

@davidad Mind showing your work?

davidad 🎇

@davidad

272d

For those who are not familiar with exponentials, this may be the clearest way of understanding what people are implying when we say “AI timelines have gotten shorter recently”: the expected rate of progress has changed from the orange line to the red line.

AI Digest

@AiDigest_

272d

Here's what that faster trend looks like extrapolated out.

The statement engages in a technical discussion about AI progress, specifically addressing the representation of data on a graph and the capabilities of AI in scene comprehension tasks. It critiques the choice of the y-axis in the graph, suggesting that it may not accurately reflect the tasks AI can perform. This contributes to public discourse by questioning the metrics used to evaluate AI progress.

Principle 1:
I will strive to do no harm with my words and actions.
The statement does not cause harm and focuses on a technical critique, aligning with the principle of doing no harm. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
By questioning the graph's representation, it promotes understanding and encourages a more accurate depiction of AI capabilities. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages in constructive criticism without personal attacks, adhering to the principle of constructive dialogue. [+1]