Gary Marcus

Rank 13 of 47
|
Score 83
8/6/2024, 1:59:50 PM
X
In reply to:
David Manheim
@davidmanheim
·
524d
@AleksanderMolak @GaryMarcus The second is whether LLMs have correct implicit models of causal relationships at the textual level, and can they correctly predict causal outcomes in cases not in their training data - and the answer is absolutely yes, albeit only sometimes.
David Manheim
@davidmanheim
·
524d
@AleksanderMolak @GaryMarcus OK, there are two different claims that are relevant here. The first is whether deep learning via SGD can directly learn causal features from observational data, and the answer to that is no.
Aleksander Molak (CausalPython.io)
@AleksanderMolak
·
524d
@davidmanheim @GaryMarcus ...a correct formal structure.

At the same time I've seen many examples of them failing to learn it, even under conditions like setting initial weights to the correct solution, transformers unlearn it & overwrite it with a statistical shortcut.

2/2
Aleksander Molak (CausalPython.io)
@AleksanderMolak
·
524d
@davidmanheim @GaryMarcus This is a different claim than "we can infer causal structure from obseravtions only with sophisticated enough model".

But even with the internet scale data containing a mix of obs. & (passive) interventional data I haven't seen any proof of these models learning...

1/2
David Manheim
@davidmanheim
·
525d
@AleksanderMolak @GaryMarcus Yes - PCH usually holds in data-limited settings without explicit past interventions. But general cases we're talking about allows for insanely more data on both interventions and overall than is needed to infer structure and causality.
Aleksander Molak (CausalPython.io)
@AleksanderMolak
·
525d
@davidmanheim @GaryMarcus I don't see how this could be true in general (excl. special cases), it seems to contradict the PCH theorems.

Neither interventional nor counterfactual queries can be answered (in general) from observations only.

Would you disagree with this?

2/2

The statement in question is part of a broader discussion on the capabilities and limitations of large language models (LLMs) in understanding and predicting causal relationships. This conversation involves multiple participants and addresses a substantive issue in the field of artificial intelligence and machine learning, specifically the ability of LLMs to infer causal structures from data. Therefore, it constitutes public discourse.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement uses a metaphor comparing LLMs to a broken clock, which is a mild form of criticism but does not appear to cause harm.
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not engage in cyberbullying, harassment, or hate speech. It respects the privacy and dignity of others.
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not explicitly promote understanding, empathy, or compassion, but it does contribute to a technical discussion.
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by questioning the reliability of LLMs without resorting to personal attacks or ad hominem arguments. [+1]
  5. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech and uses the platform responsibly to discuss a technical issue with integrity. [+1]