Gary Marcus

Rank 13 of 47
|
Score 115
7/22/2025, 3:43:24 AM
X
In reply to:
Boaz Barak
@boazbaraktcs
·
53d
@GaryMarcus The basic equation of deep learning of spending compute to obtain better performance still holds. How that compute is allocated between different steps of training and how exactly it used at inference (e.g. invoking a big model once vs a small model many times) is of second order
Gary Marcus
@GaryMarcus
·
53d
@boazbaraktcs 1. the returns that i think are diminishing are on base models with pure pretraining scaling, which is what I warned about in 2022
2. i think the gains are increasingly coming from the use of (neuro)symbolic tools.
3. i agree that o3 is better that GPT-4.
Boaz Barak
@boazbaraktcs
·
53d
@GaryMarcus No intention to insult. Seeing the progress from GPT-4 to o3, I think the returns are anything but diminishing but to each their own.
Gary Marcus
@GaryMarcus
·
53d
Even in my meme below I didn’t plot zero progress. I have often talked about diminishing returns. I have said publicly e.g., that o1 and o3 and the new model are impressive. Never once did I say that the field has made no progress post GPT-4. (I did say that certain problems
Boaz Barak
@boazbaraktcs
·
53d
Yes we really have made no progress

The statement is a lighthearted acknowledgment of a previous message about convergence in a technical discussion. It reflects a positive and humorous tone, indicating a resolution or agreement in the conversation.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and maintains a friendly tone. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding and camaraderie through humor. [+1]