Gary Marcus

Rank 11 of 47
|
Score 83
4/12/2024, 2:28:28 AM
X
In reply to:
Brydon Eastman
@brhydon
·
661d
@GaryMarcus The carrying capacity analogy is formal/mathematical, not in terms of interpretation. (When the range of a quantity is limited by some fundamental limit, exp growth becomes sigmoidal)

Fully agree that there is a _lot_ more space in the universe for intelligence.
Gary Marcus
@GaryMarcus
·
661d
@brhydon Also the analogy to carrying capacity is a bit tricky here. E.g., I don’t think we are close to maximum intelligence that the universe could support.

Maybe we are close to what current data given current methods can support, but that’s kind of my point.
Gary Marcus
@GaryMarcus
·
661d
@brhydon Also a fair point, and why we need more difficult measures.

On the other hand, I think it is problem with the measures, not that we have saturated on general intelligence.

In any case if the field doesn’t improve reliability, perhaps not fully captured in this measures,…
Brydon Eastman
@brhydon
·
661d
When I used to teach DEs, this phenomenon was a favourite example of mine.

N'(t) = rN
vs
N'(t) = rN (1 - N/K)

What happens to exponential growth when there's a carrying capacity? It becomes a sigmoid.

Fit a sigmoid to these numbers and tell me what you see.
Gary Marcus
@GaryMarcus
·
661d
What happens when you plot GPT-2, 3, 4, and Turbo side-by-side?

Below I have plotted one common measure, MMLU, where there are easy to find data going back to GPT-2. (There may be others with data going back that far; this is just a first quick attempt.)

What I see is an…

The statement is a technical contribution to a discussion on AI growth limits, engaging with the topic in a respectful and constructive manner.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is focused on technical analysis. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects the dignity of others and avoids harmful language. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding of a complex topic. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement is part of a constructive dialogue without personal attacks. [+1]
  5. Principle 6:
    I will use my influence for the betterment of society.
    The statement contributes to societal betterment by informing the AI development discourse. [+1]
  6. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech and is shared responsibly and with integrity. [+1]