Gary Marcus

Rank 15 of 47
|
Score 81
4/7/2024, 5:34:54 PM
X
In reply to:
Danrae Pray
@DanraeP
·
533d
@GaryMarcus @Simeon_Cps @StatsLime @WSJ I think his definition of data augmentation is just distinguishing between multiple 'perspectives' or 'views' of the same data vs using simulations to generate data.

They really are very similar, though.
Gary Marcus
@GaryMarcus
·
533d
@DanraeP @Simeon_Cps @StatsLime @WSJ Renders from UE5 seems to me to qualify both as data augmentation and synthetic data.

I am lost here, if there is a real distinction.
Danrae Pray
@DanraeP
·
533d
@Simeon_Cps @GaryMarcus @StatsLime @WSJ No, synthetic data doesn't contain any new information and majority of it is data augmentation.

This is why half the Sora videos look like real-time renders from UE5 with a first-person character controller or camera-boom for character movement.
Siméon
@Simeon_Cps
·
533d
@GaryMarcus @DanraeP @StatsLime @WSJ That's also why the human metaphor I made makes sense. Humans say bs stuff all the time but fortunately we teach kids some highly curated content.

For that to work we just need to be able to generate sometimes some good content and filter the rest out.
Gary Marcus
@GaryMarcus
·
533d
@Simeon_Cps @DanraeP @StatsLime @WSJ @Simeon_Cps I don’t that I fully follow your distinction. People often use the terms more or less interchangeably, don’t they? How are you distinguishing?
Siméon
@Simeon_Cps
·
533d
@DanraeP @StatsLime @GaryMarcus @WSJ That's not what people are doing with synthetic data and LLMs. People are using models to generate maths proofs than they then check with theorem provers that you can then throw in the dataset if it's correct. It's not "rearrangement".

Don't conflate synthetic data and data…

The statement is part of a technical discussion on data augmentation and synthetic data in machine learning, which does not constitute public discourse as it does not address broader societal issues or policies.