Venice Uncensored AI Engine
(media.scored.co)
New here?
Create an account to submit posts, participate in discussions and chat with people.
You are viewing a single comment's thread. View all
Toast message
The LLM inevitably has to speculate what the user wants and has to provide an answer that is generic. I just tested it, so let me show you the results from Gemini (fast mode). I used 2 similar questions, one with no context, one with more context:
---
What is the Dunning-Kruger effect?
> The Dunning-Kruger effect is a cognitive bias where individuals with low ability or knowledge in a specific area overestimate their own competence. This occurs because the lack of expertise required to perform a task also deprives them of the "metacognitive" ability to recognize their own errors. Conversely, highly competent individuals often suffer from "impostor syndrome," assuming that because a task is easy for them, it must be equally easy for others. Essentially, it is the inability to accurately judge one's own level of skill.
---
I recently heard criticism about the Dunning-Kruger effect. What I heard is that it got popular because it used fallacious methods, and is essentially a non-reproducible experiment. For example there is the popular graph with the "mount stupid", but it is said that it is a statistical artifact resulting from an a/b formula, leading the lower end to have much more extreme values than the upper end. Meaning someone who was tested thought he'd get a 20 instead got a 10, has a value of 2.0 on that graph, meaning 100% too confident, whereas someone who scores 80 but thought himself to be the world-best (aka score 100) would be a 1.25 on that graph, meaning 25% too confident. Which is clearly absurd, because the former is just someone with little knowledge giving a timid estimation in the lower range, while the latter is someone being completely delusional. Also the basis of the Dunning-Kruger effect was that it was originally based on psychology students who had to estimate the scores of their tests. To assume "confidence" and "competence" would be also misleading. "Confidence" indicates not only a prediction, but also the level of conviction of it being true - which is clearly not measured. "Competence" is also a subjective criterion, as it's a test environment that can be measured, like a test. In real-life there are many factors to competency which cannot be measured, like how well an applicant for a position can sell himself and communicate. There are studies which show that higher attractiveness and higher height have positive influence for that. I've seen two graphs that were produced by the Dunning-Kruger effect, which were two linear graphs. It raises the question if it wasn't the popularity of the graph with "mount stupid" that made it popular, even though it was misleading. And given its popularity all sorts of ancient wisdom has been attributed to it, even to simple ideas like the fact that one is not able to evaluate his own competence well when his competence is still low (like a beginner).
Given this context, how would you describe what the Dunning-Kruger effect is?
> Response: Modern critiques describe the Dunning-Kruger effect less as a psychological phenomenon and more as a statistical artifact known as "regression toward the mean." Critics argue that the original 1999 study’s findings result from the mathematical nature of plotting actual vs. estimated scores; people at the bottom have more room to overestimate, while those at the top have more room to underestimate.
> The "Mount Stupid" curve you mentioned is largely a digital-age fabrication—the original data actually showed two nearly linear, upward-sloping lines. Mathematically, if you subtract a random variable (score) from a fixed estimate, you naturally produce the "overestimation" observed in low performers. Ultimately, the effect may simply be a byproduct of human noise in self-assessment rather than a specific cognitive deficit in the "incompetent."
---
Also, I asked it about how it responded, and what people would think:
> The contrast between your two prompts highlights a fundamental trait of LLMs: we are designed to be helpful, which often means mirroring the consensus or "common knowledge" found in our training data when a prompt is brief.
> *User Conclusions based on the Two Responses* - From the First Response: A reader would likely conclude that the effect is an established psychological fact. Because the prompt was "What is it?", the model provided the standard textbook definition. This reinforces the status quo because there was no "friction" in the prompt to trigger a deeper dive into the methodology.
> From the Second Response: A reader would conclude that the effect is scientifically shaky and often misused. By introducing the "Mount Stupid" myth and the statistical artifact argument, you shifted the model from "definition mode" to "critical analysis mode."
> *The "Gap-Filling" Trap* - You're exactly right about speculating to fill gaps. In the first prompt, I "filled the gap" by assuming you wanted the most popular interpretation. In the second, I "filled the gap" by assuming you were looking for a technical validation of the modern critique.