Gary Marcus

Rank 14 of 47
|
Score 83
4/18/2024, 8:17:18 PM
X
In reply to:
Matthew Barnett
@MatthewJBar
·
622d
@GaryMarcus Gemini provides some evidence, but it didn't use much more compute than GPT-4 according to the estimate from Epoch. Differences in architectures, training methods, and so on can wash out small differences in training compute. The real test hasn't arrived yet.
Gary Marcus
@GaryMarcus
·
622d
@MatthewJBar My x axis would be tome, but even on yours why wouldn’t Gemini Ultra count?
Matthew Barnett
@MatthewJBar
·
622d
I think the plateau hypothesis should be operationalized by looking at returns to compute scaling. As far as we know, no model so far released was trained on more than 10^26 FLOP, so we don't have much evidence for or against this theory yet.
Gary Marcus
@GaryMarcus
·
622d
Each massive new model that is NOT better than GPT-4 is a bit of evidence for the hypothesis of a plateau or diminishing returns.

Each massive new model that is NOT better than GPT-4 is a bit of evidence that we are NOT still on an exponential rise.

Everyone understands that,…

The statement '@MatthewJBar So much for smaller more efficient models?' is a response in a technical discussion about the efficiency and computational requirements of machine learning models, specifically in the context of comparing newer models like Gemini Ultra to GPT-4. The conversation involves a debate on whether newer models are truly advancing in efficiency or just increasing in size without significant improvements.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm but rather contributes to a technical discussion, which is inherently neutral and aimed at clarifying or questioning technological advancements. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects the privacy and dignity of others as it does not involve any personal attacks or sensitive information about individuals. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The comment is not directly aimed at promoting understanding, empathy, or compassion, as it is a technical critique rather than a comment on social issues.
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in a form of constructive criticism by questioning the efficiency of newer models, which is a valid point in a technical debate. [+1]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    The statement is straightforward and does not involve any acknowledgment of mistakes as it is part of an ongoing discussion and critique.
  6. Principle 6:
    I will use my influence for the betterment of society.
    The use of the statement in the discussion could be seen as using influence to question and possibly improve understanding and approaches in the field of machine learning, which could be beneficial for societal technological advancement. [+1]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech by contributing to a public technical discussion without misuse of the platform. [+1]