Gary Marcus

Rank 16 of 47
|
Score 81
9/14/2024, 12:29:38 AM
X
In reply to:
Martin Rodgers
@FaustinoBerlin
·
349d
@colin_fraser It's simple. LLMs parse text statistically. So if I prompt "Summarise Marx's thinking in a dialectical model with Hayek as counterpoint" it will produce some damn impressive output, because it is plagiarising. I'm here in my chess game best next move ? It's beat
@GaryMarcus
Colin Fraser
@colin_fraser
·
349d
Anyway, turned into a bit of a rant at the end there, but that's my updated review. I think it's cool. I would definitely not recommend putting it in charge of anything important. I'd like a few more prompts per week but not so much that I'd pay for them.

6/10
Colin Fraser
@colin_fraser
·
349d
If you give it a list of things to do with complex interdependencies will it be able to reliably keep track of them all and execute them in the right order? AIME benchmark suggests yes, goats boats suggests no. Who's right? No one knows.
Colin Fraser
@colin_fraser
·
349d
Yes, it scores 14/15 on the AIME, but are the problems you want to put it to work on more like advanced plane geometry, or more like getting goats across a river? I think if you think about it, you might realize that many problems are actually a bit like goat boat problems.
Colin Fraser
@colin_fraser
·
349d
It kind of suggests that the benchmarks are just not that informative. The benchmarks tell you about whether it's good at the problems on the benchmarks. But your problems aren't the problems on the benchmarks.
Colin Fraser
@colin_fraser
·
349d
The only reason I think you would want to ignore this question is if you were invested—financially, emotionally, whatever—in the wide adoption of this technology. The fact that it can't get the goats across the river raises some questions about whether this AI is right for you.

The statement in question is part of a broader conversation about the capabilities and limitations of language models (LLMs) and artificial intelligence (AI). It engages with public issues related to technology, AI ethics, and their applications, thus constituting public discourse. The tone is somewhat critical and dismissive, particularly towards Tyler Cowen, suggesting a lack of understanding on his part.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm directly, but the dismissive tone towards Tyler Cowen could be seen as undermining constructive dialogue.
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not engage in cyberbullying, harassment, or hate speech, but it does not fully respect the dignity of Tyler Cowen by implying he doesn't understand the topic. [-1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not promote understanding, empathy, or compassion. Instead, it dismisses another person's understanding without providing a constructive explanation. [-1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement does not engage in constructive criticism or dialogue. It makes a personal attack by implying Tyler Cowen's lack of understanding without addressing the substance of his views. [-2]
  5. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principle of free speech but does not use the platform responsibly or with integrity, as it dismisses another person's understanding without constructive engagement. [-1]