Gary Marcus

Rank 15 of 47
|
Score 76
5/4/2024, 12:17:12 AM
X
In reply to:
Thomas G. Dietterich
@tdietterich
·
553d
@LeeKezar @Abebab @GaryMarcus @ChombaBupe @rao2z But the model clearly doesn't understand who Mother Teresa was or what her work was about. The whole point of VLMs is that they are supposed to understand our requests. Maybe some RAG would help?
Lee Kezar
@LeeKezar
·
553d
@prem_k @tdietterich @Abebab @GaryMarcus @ChombaBupe @rao2z It doesn't make sense to me *as an image caption*. I don't know what you expected the model to produce. And if you asked 10 people you'd get 10 very different drawings. I don't think we should expect any model to read our minds.
Pɾҽɱ Kυɱαɾ Aραɾαɳʝι 🤖💬🦾
@prem_k
·
553d
@LeeKezar @tdietterich @Abebab @GaryMarcus @ChombaBupe @rao2z Little sense to who?

It made enough sense to me when I saw something similar and wrote it.

All of the sense in the output from these models are made in our minds.

The models are only arranging stochastically.
Lee Kezar
@LeeKezar
·
553d
@tdietterich @prem_k @Abebab @GaryMarcus @ChombaBupe @rao2z I'd like to add that the prompt makes little sense as an image caption. Fighting poverty in general is an abstract concept and not easy to visually show. In such cases I would expect the model to prefer the interpretation that is visually expressable
Thomas G. Dietterich
@tdietterich
·
553d
@prem_k @Abebab @GaryMarcus @ChombaBupe @rao2z Combine the highest probability rendering of "Mother Teresa" with the highest probability interpretation of "fighting" and the highest probability interpretation of "poverty". Perfect example of the stochastic "mindset" of VLMs
Pɾҽɱ Kυɱαɾ Aραɾαɳʝι 🤖💬🦾
@prem_k
·
624d
"make a watercolor painting of mother Teresa fighting against poverty"

https://copilot.microsoft.com/im…

https://copilot.microsoft.com/im…

@Abebab @GaryMarcus @ChombaBupe @rao2z

The statement in question is part of a technical discussion on the limitations and capabilities of visual language models (VLMs) and retrieval-augmented generation (RAG) systems in understanding complex human concepts and producing relevant outputs. The conversation revolves around the effectiveness of these AI models in interpreting and generating content that accurately reflects nuanced human requests, such as creating an image that encapsulates abstract concepts like 'fighting poverty'.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement contributes to a constructive technical discussion without causing harm or promoting misinformation. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    It respects the dignity of the participants by maintaining a professional tone and focusing on the subject matter. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by highlighting the need for deeper comprehension of the world for AI models to handle complex requests effectively. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    It engages in constructive criticism of the current capabilities of AI models, suggesting that off-the-shelf solutions may not suffice for certain tasks. [+1]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    The participant acknowledges the limitations of current technologies, which is an important aspect of honest and transparent communication in technical discussions. [+1]
  6. Principle 6:
    I will use my influence for the betterment of society.
    By discussing the limitations and necessary improvements in AI technologies, the statement indirectly contributes to the betterment of societal tools and frameworks. [+1]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The use of the platform here is responsible, focusing on sharing knowledge and insights about AI technologies responsibly. [+1]