Gary Marcus

Rank 12 of 47
|
Score 83
5/19/2025, 9:50:20 PM
X
In reply to:
Christopher Chabris
@cfchabris
·
255d
@GaryMarcus What if someone tried to super-train an LLM using just chess text databses, PGN, FEN, text of every chess book and article ever written, etc. I suspect that a structureless LLM still would not learn an adequate logical model of chess despite feeding it all extant chess text.
Gary Marcus
@GaryMarcus
·
255d
@cfchabris And the FEN doesn’t line up, for good measure
Christopher Chabris
@cfchabris
·
255d
It's even worse: the position in the description is not legal, as it cannot be Black's move if White is in check and there is no previous move (not even Rh1) that would transform a legal position into this one. Great example of the Gell-Mann Amnesia concept applied to LLMs.
Gary Marcus
@GaryMarcus
·
257d
𝗔𝗯𝘀𝗼𝗹𝘂𝘁𝗲 𝗰𝗵𝗲𝘀𝘀 𝗴𝗶𝗯𝗯𝗲𝗿𝗶𝘀𝗵 𝘁𝗵𝗮𝘁 𝗶𝗹𝗹𝘂𝘀𝘁𝗿𝗮𝘁𝗲𝘀 𝗵𝗼𝘄 𝗳𝗮𝗿 𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝗳𝗿𝗼𝗺 𝗯𝗲𝗶𝗻𝗴 𝗔𝗚𝗜

I asked a slightly unusual question (“Create a chess position in which white has 3 queens but will lose on the next move”) and the system

The statement is part of a technical discussion about the limitations of large language models (LLMs) in understanding and generating valid chess positions. It references a study and confirms findings related to the performance of LLMs in specific scenarios. The tone is analytical and focused on evaluating AI capabilities.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is focused on technical evaluation. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    It promotes understanding of AI limitations in a specific context. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    Engages in constructive dialogue by referencing research and confirming findings. [+1]