Gary Marcus

Rank 23 of 47
|
Score 29
5/2/2024, 2:56:09 PM
X
In reply to:
(╯°□°）╯︵ ┻━┻
@Laserfish2
·
502d
@RomeoLupascu @JimDMiller @GaryMarcus The error rate for human doctors is approximately 1 in 5.
Romeo Lupascu
@RomeoLupascu
·
502d
@JimDMiller @GaryMarcus race to the bottom for 100Billon? also people tend to believe machines better then humans hence we must expect way way worse outcomes from machine errors then humans, we can't put the LLM in prison can we? :) .. so ... yeah for me is 1/1000000
James Miller
@JimDMiller
·
502d
@RomeoLupascu @GaryMarcus How do I find a human doctor who has an error rate of under 1/1000000?
Romeo Lupascu
@RomeoLupascu
·
502d
@GaryMarcus I do not understand how we can show a model that is correct only 80% of the time and then push it in production to handle sensitive decisions like health, financial, legal. The necessary error rate must go under 1/1000000 before we can take any of these models seriously.
Gary Marcus
@GaryMarcus
·
503d
Scaling “laws” were never laws.

They were trends. Trends whose time may be up.

13.5 months and $50B post GPT-4, GPT-4 remains on top.
Noam Brown
@polynoamial
·
503d
Many have pointed out that LLM benchmarks are broken and gamed. Happy to see my former resident @hughbzhang, @summeryue0, and the great @scale_AI folks do something about it! They made a private version of GSM8k and evaled GPT-4, Claude, Mixtral, Phi, etc: https://arxiv.org/pdf/2405.00332

The statement in question is part of a broader conversation about the reliability and error rates of human doctors versus machine learning models in making decisions. The specific reply highlights the need to consider different metrics when comparing error rates, suggesting that the context or specific task (like taking blood pressure readings) significantly influences these rates. This contributes to a nuanced understanding of the performance and reliability of both human and machine decision-makers in healthcare and other sensitive areas.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement aims to clarify and add depth to the discussion, which helps in reducing misunderstandings or harm that might arise from oversimplified comparisons. [+2]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    It respects the dignity of professionals (doctors) by providing a more accurate representation of their error rates in specific tasks, countering potentially misleading generalizations. [+2]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The reply fosters understanding by pointing out the importance of context in evaluating error rates, thus promoting a more informed discussion. [+2]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    It engages constructively by adding a critical perspective to the ongoing dialogue, which helps in refining the arguments presented. [+2]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    The statement corrects a potential oversight in the discussion by specifying that error rates can vary significantly depending on the task, thus acknowledging the complexity of the issue. [+2]
  6. Principle 6:
    I will use my influence for the betterment of society.
    By contributing a thoughtful insight, the statement uses its influence to improve the quality of public discourse around the topic of machine versus human error rates in sensitive decision-making. [+2]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech responsibly by contributing substantively to the discussion without resorting to misinformation or harmful rhetoric. [+2]