Gary Marcus

Rank 14 of 47
|
Score 83
3/28/2024, 7:53:31 PM
X
In reply to:
Yonathan Arbel
@ProfArbel
·
640d
@GaryMarcus @emollick No argument on not good enough for journalism, but I don't read the paper as only useful if it can solve this high-end problem
Gary Marcus
@GaryMarcus
·
640d
@ProfArbel @emollick In this one it is all but irrelevant. No serious periodical would outsource its factchecking to underpaid crowdsourcers.

If I have read correctly, this paper is deeply misleading.
Yonathan Arbel
@ProfArbel
·
640d
@GaryMarcus @emollick It's always good to keep in mind the distinction between better than average humans and better than best of humans. Still, for many real-world apps, better than avg human is meaningful
Gary Marcus
@GaryMarcus
·
640d
@emollick On a quick read I can’t figure out much about the human subjects, but it looks like superhuman means better than an underpaid crowd worker, rather a true human fact checker? That makes the characterization misleading. (Like saying that 1985 chess software was superhuman).…
Ethan Mollick
@emollick
·
640d
AIs have a bad reputation for truth, so three important findings in this paper:
1) "LLM agents can achieve superhuman rating performance" on fact checking when given access to Google!
2) Bigger models are more factual
3) LLMs are 20x cheaper than humans
https://arxiv.org/pdf/2403.18802…

The statement is part of an academic debate on the appropriate terminology and interpretation of research findings related to AI performance in fact-checking. The tone is critical and suggests a need for precision in language to accurately reflect the capabilities of AI compared to human experts. The intent is to advocate for clarity and accuracy in the paper's claims.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement aims to correct potential overstatements in the paper, thus aligning with the principle of doing no harm with words and actions. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects the dignity of human workers by advocating for a more accurate comparison with AI performance. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The critique is intended to foster a more empathetic understanding of the capabilities of average workers versus experts. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism of the paper's terminology and findings. [+1]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    The statement itself does not acknowledge mistakes but calls for the paper's authors to do so.
  6. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses influence to advocate for more accurate and responsible research communication. [+1]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech by engaging in a critical review of the paper's claims responsibly. [+1]