Gary Marcus

Rank 15 of 47
|
Score 91
1/20/2025, 12:08:47 AM
X
In reply to:
Tim Rocktäschel
@_rockt
·
2d
Recurring question: So what? It can't be used to train Deep NNs, let alone LLMs.

That's true. ES doesn't work well in high dimensional spaces. As it maintains a population, memory requirements (if you are using ES to train Deep NNs) are prohibitive and sample efficiency is not…
Tim Rocktäschel
@_rockt
·
3d
Highly non-convex toy optimization problem.

SGD vs Adam vs Evolutionary Strategies (ES).

Let's gooooooo... https://x.com/_rockt/status/1880…

The statement is a response in a technical discussion about optimization strategies in machine learning, specifically comparing Stochastic Gradient Descent (SGD), Adam, and Evolutionary Strategies (ES). The reply highlights the uniqueness of evolution in creating general intelligence, which is a tangential point to the original technical discussion.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is a neutral observation about evolution and intelligence.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement could promote understanding by introducing a broader perspective on intelligence, but it does not explicitly foster empathy or compassion.