Gary Marcus

Rank 32 of 47

Score -1

Reposted by Gary Marcus

This adds to a growing body of work showing that LLMs do not (yet) have robust world models. They can seem like they do, but the models are fragile and easily broken, like if you add detours to the route the LLM plans, it makes mistakes. Will full multimodality change that?

Keyon Vafa

@keyonV

149d

New paper: How can you tell if a transformer has the right world model?

We trained a transformer to predict directions for NYC taxi rides. The model was good. It could find shortest paths between new points

But had it built a map of NYC? We reconstructed its map and found this:

6/20/2024, 9:33:33 PM

The statement engages in public discourse by discussing the limitations and potential future developments of large language models (LLMs) in the context of artificial intelligence research. It critically examines the current state of LLMs' world models and raises a question about the impact of full multimodality on these models. The tone is analytical and inquisitive, aiming to contribute to the ongoing debate about the capabilities and future of LLMs.

Principle 1:
I will strive to do no harm with my words and actions.
The statement strives to do no harm and is focused on a technical discussion about LLMs, thus adhering to this principle. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
The statement promotes understanding and empathy by providing a thoughtful analysis of the current limitations of LLMs and considering future improvements. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages in constructive criticism and dialogue by questioning the robustness of LLMs' world models and considering the potential impact of full multimodality, without resorting to personal attacks or ad hominem arguments. [+1]