Gary Marcus

Rank 10 of 47
|
Score 127
6/16/2025, 7:22:42 PM
X
In reply to:
Andriy Burkov
@burkov
·
35d
@GaryMarcus :-) You can tokenize by individual characters. This creates another problem though: the context size in tokens will grow very quickly.
Gary Marcus
@GaryMarcus
·
35d
@burkov sure. but how you going to build AGI when you can’t solve those issues?
Andriy Burkov
@burkov
·
35d
@GaryMarcus I agree with some commenters that it's a tokenizer issues, likely the same issue that doesn't allow it to count r's in strawberry.
Gary Marcus
@GaryMarcus
·
36d
LLMs are going to solve physics and they bungle regexes?
Andriy Burkov
@burkov
·
37d
For some unclear reason, LLMs are very poor with regexes. For example, this is a regex I've got from Gemini today:

[a-z0--9]+

This won't even compile. And it's very often this way.

I also have a hard time making it generate a regex that would match all elements of a given

The statement is part of a technical discussion about the limitations and challenges of language models, specifically regarding tokenization and representation issues. It engages with public discourse on the capabilities and development of artificial general intelligence (AGI) and language models, which are significant topics in the field of artificial intelligence.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive technical discussion, adhering to the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by sharing insights from a book, contributing to the collective knowledge on the topic. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the technical aspects of the discussion. [+1]