Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 4:06:57 PM
X
In reply to:
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose The training on the hundreds of examples is test-time training, which as noted by Chollet et al., is “expect[ed]” to “be the primary technique for LLM-based AI systems to improve
performance on tasks outside of what they were pretrained on.”

Difficult to find an analogous…
Gary Marcus
@GaryMarcus
·
390d
@RishabJainK @kevinroose i don’t follow what you are saying, maybe you can rephrase.
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose This is not a critique of “pretraining,” it’s a critique of TTT. You are conflating the two
Gary Marcus
@GaryMarcus
·
390d
@RishabJainK @kevinroose 03 had far more pretraining than any human, and should not have required more. please see the below for many people making this point
Gary Marcus
@GaryMarcus
·
390d
𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴—𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀

@KevinRoose, of Hard Fork and NYT, was so impressed with OpenAI’s rollout that he joked “of course they…
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose Humans don’t solve ARC-AGI tasks zero-shot either—we “pretrain” through life. Toddlers do puzzles, learn patterns, and generalize problem-solving. If anything, this just suggests we need more evals and benchmarks to test against to see if o3 generalizes well.
Gary Marcus
@GaryMarcus
·
390d
𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴—𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀

@KevinRoose, of Hard Fork and NYT, was so impressed with OpenAI’s rollout that he joked “of course they…

The statement is part of a technical discussion on AI training methods, specifically addressing the comparison between human learning and machine learning. It raises a question about the fairness of comparing human trials to machine trials, suggesting a need for further investigation. The tone is analytical and seeks to clarify a point in an ongoing debate.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is focused on a technical discussion, maintaining a neutral and respectful tone. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by clarifying a point about AI training, encouraging further exploration of the topic. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by addressing a question in the debate without resorting to personal attacks. [+1]