Gary Marcus

Rank 13 of 47
|
Score 83
5/18/2025, 3:36:20 PM
X
In reply to:
Lawrence Chan
@justanotherlaw
·
250d
@GaryMarcus Yeah, I couldn't find a reference for this for newer models (everything with public documentation is from 2024 or before), so maybe my impression is just wrong here
Gary Marcus
@GaryMarcus
·
250d
@justanotherlaw i don’t think that is true of some of the newer models
Lawrence Chan
@justanotherlaw
·
250d
@GaryMarcus My impression is this is a result of visual capabilities being much less common during training than text-based reasoning. For many models visual tokens are added only very late in pre-training or have them grafted on only after training is complete.
Lawrence Chan
@justanotherlaw
·
250d
@GaryMarcus Your point is directionally correct though -- models have a lot easier time with the text grid than with the original screenshots; in general, they do a lot better with text reasoning than they do with images. There's a reason most of the remaining hard academic benchmarks for
Lawrence Chan
@justanotherlaw
·
250d
@GaryMarcus Reading o3's CoT (summary) is pretty illustrative -- to solve the easier one, it just looks at the board a few times; to solve the harder one, it wrote a script to turn the board into a text grid. I'll post the share link but the app tells me it doesn't support sharing convos
Lawrence Chan
@justanotherlaw
·
250d
@GaryMarcus o3 seems to be able to do both your original question and the harder one, though the harder one takes much longer (5-8 minutes it seems). Example solve for the harder one.

The conversation involves a technical discussion about the capabilities of language models, particularly in handling visual and text-based tasks. It touches on the integration of image and text understanding in AI models, referencing specific technologies like transformers and multimodal models. The tone is analytical and informative, aiming to clarify misconceptions and share insights about AI model training and capabilities.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement strives to do no harm, focusing on sharing factual information about AI models. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The discussion promotes understanding by explaining complex AI concepts in a clear manner. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    Engages in constructive dialogue, addressing potential misconceptions without personal attacks. [+1]