AI 模型置信陷阱

Q: 引言

简述文章的主要问题和背景。

Q: 网站隐私政策

概述网站隐私政策的重要性。

Q: 结论

总结文章的主要观点和建议。

Towards Data Science

Towards Data Science2026年5月26日

AI 模型置信陷阱

2.0Score

TL;DR · AI 摘要

文章主要介绍了一个关于 AI 模型置信陷阱的问题，但主要内容是网站的隐私政策和 cookie 设置，缺乏深度和实用性。

核心要点

AI 模型置信陷阱是指对 AI 模型过于自信，忽视其局限性和潜在风险。
网站的隐私政策和 cookie 设置对于用户隐私保护非常重要。
了解和管理 cookie 对于提高网站性能和用户体验至关重要。

结构提纲

按章节快速跳转。

§引言
简述文章的主要问题和背景。
·AI 模型置信陷阱
解释什么是 AI 模型置信陷阱及其影响。
·网站隐私政策
概述网站隐私政策的重要性。
·Cookie 设置
详细说明网站的 cookie 设置及其作用。
·结论
总结文章的主要观点和建议。

思维导图

用一张图看清主题之间的关系。

查看大纲文本（无障碍 / 无 JS 友好）

AI 模型置信陷阱
- 定义与影响
- 网站隐私政策
  - 重要性
- Cookie 设置
  - 作用与管理

金句 / Highlights

值得收藏与分享的关键句。

AI 模型置信陷阱是指对 AI 模型过于自信，忽视其局限性和潜在风险。
— 第 2 段
⬇︎ 下载 PNG 𝕏 分享到 X
了解和管理 cookie 对于提高网站性能和用户体验至关重要。
— 第 4 段
⬇︎ 下载 PNG 𝕏 分享到 X

#AI#隐私#Cookies

打开原文

The AI Model Confidence Trap | Towards Data Science

We value your privacy

We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies.

Customise Reject All Accept All

Customise Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ...Show more

Necessary Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Cookie BCTempID
Duration 10 minutes
Description No description available.

Cookie __cf_bm
Duration 1 hour
Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management.

Cookie AWSALBCORS
Duration 7 days
Description Amazon Web Services set this cookie for load balancing.

Cookie _cfuvid
Duration session
Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services

Cookie li_gc
Duration 6 months
Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.

Cookie __hssrc
Duration session
Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.

Cookie __hssc
Duration 1 hour
Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.

Cookie wpEmojiSettingsSupports
Duration session
Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.

Cookie BCSessionID
Duration 1 year 1 month 4 days
Description Blueconic sets this cookie as a unique identifier for the BlueConic profile.

Cookie _octo
Duration 1 year
Description No description available.

Cookie logged_in
Duration 1 year
Description No description available.

Cookie __Secure-YEC
Duration past
Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video

Cookie __eoi
Duration 6 months
Description Description is currently not available.

Cookie AWSALBTGCORS
Duration 7 days
Description No description available.

Cookie login-status-p
Duration past
Description Description is currently not available.

Cookie AWSALBTG
Duration 7 days
Description No description available.

Cookie csrf_token
Duration session
Description No description available.

Cookie token_v2
Duration 1 day
Description Description is currently not available.

Cookie D
Duration 1 year
Description Description is currently not available.

Cookie PHPSESSID
Duration session
Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.

Cookie VISITOR_PRIVACY_METADATA
Duration 6 months
Description YouTube sets this cookie to store the user's cookie consent state for the current domain.

Cookie cookietest
Duration session
Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.

Cookie __Host-airtable-session
Duration 1 year
Description This cookie is used to enable us to integrate the services of Airtable.

Cookie __Host-airtable-session.sig
Duration 1 year
Description This cookie is used to enable us to integrate the services of Airtable.

Cookie m
Duration 1 year 1 month 4 days
Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.

Cookie BIGipServer*
Duration session
Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user.

Cookie __cfruid
Duration session
Description Cloudflare sets this cookie to identify trusted web traffic.

Cookie _GRECAPTCHA
Duration 6 months
Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.

Cookie __Secure-YNID
Duration 6 months
Description Google cookie used to protect user security and prevent fraud, especially during the login process.

Cookie cookieyes-consent
Duration 1 year
Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.

Functional

[x]

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Cookie lidc
Duration 1 day
Description LinkedIn sets the lidc cookie to facilitate data center selection.

Cookie brw
Duration 1 year
Description No description available.

Cookie brwConsent
Duration 5 minutes
Description Description is currently not available.

Cookie WMF-Uniq
Duration 1 year
Description Description is currently not available.

Cookie loom_anon_comment
Duration 1 year
Description No description available.

Cookie loom_referral_video
Duration session
Description Description is currently not available.

Cookie VISITOR_INFO1_LIVE
Duration 6 months
Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.

Cookie yt-remote-connected-devices
Duration Never Expires
Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.

Cookie ytidb::LAST_RESULT_ENTRY_KEY
Duration Never Expires
Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.

Cookie yt-remote-device-id
Duration Never Expires
Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.

Cookie yt-remote-session-name
Duration session
Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.

Cookie yt-remote-fast-check-period
Duration session
Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.

Cookie yt-remote-session-app
Duration session
Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.

Cookie yt-remote-cast-available
Duration session
Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.

Cookie yt-remote-cast-installed
Duration session
Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.

Cookie cp_session
Duration 3 months
Description Codepen sets this cookie for Help systems found in the website.

Cookie loid
Duration 1 year 1 month 4 days
Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.

Analytics

[x]

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Cookie __hstc
Duration 6 months
Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).

Cookie hubspotutk
Duration 6 months
Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.

Cookie _ga
Duration 1 year 1 month 4 days
Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.

Cookie _ga_*
Duration 1 year 1 month 4 days
Description Google Analytics sets this cookie to store and count page views.

Cookie __Host-psifi.analyticsTrace
Duration 6 hours
Description Description is currently not available.

Cookie __Host-psifi.analyticsTraceV2
Duration 6 hours
Description Description is currently not available.

Cookie _gh_sess
Duration session
Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.

Cookie YSC
Duration session
Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.

Cookie ajs_anonymous_id
Duration 1 year
Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.

Cookie vuid
Duration 1 year 1 month 4 days
Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website.

Performance

[x]

Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Cookie AWSALB
Duration 7 days
Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.

Cookie acq
Duration past
Description Description is currently not available.

Cookie acq.sig
Duration past
Description Description is currently not available.

Cookie ptc
Duration 2 years
Description No description available.

[x]

Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Cookie muc_ads
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website.

Cookie guest_id_marketing
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor.

Cookie guest_id_ads
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor.

Cookie personalization_id
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.

Cookie guest_id
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.

Cookie bcookie
Duration 1 year
Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.

Cookie __Secure-ROLLOUT_TOKEN
Duration 6 months
Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.

Cookie yt.innertube::nextId
Duration Never Expires
Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.

Cookie yt.innertube::requests
Duration Never Expires
Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.

Cookie session_tracker
Duration session
Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.

Cookie edgebucket
Duration session
Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.

Cookie did
Duration 1 year
Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.

Uncategorised

Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet.

No cookies to display.

Reject All Save My Preferences Accept All

Skip to content

Publish AI, ML & data-science insights to a global community of data professionals.

Sign in

Submit an Article

* *

Write For TDS

Toggle Mobile Navigation

Toggle Search

Search

Machine Learning

The AI Model Confidence Trap

Why your AI model can be wrong with 99% confidence

Sara A. Metwalli

May 26, 2026

7 min read

Share

Image by Houssam benamara from Pexels

Last year, I was feeling a bit whimsical on a Saturday and decided to ask ChatGPT a fairly simple question: “_Who won the Nobel Prize in Physics in 2025?_”

ChatGPT responded immediately: “_The 2025 Nobel Prize in Physics was awarded to…_” It even provided names, research areas, and an explanation of the specific research that earned them the Nobel Prize!

There was just one problem—a very small one, actually. The Nobel Prize had not yet been announced. Yet, the model did not hesitate; it did not pause; it surely did not say, “_I don’t have enough information_” or, even better, “_the Nobel Prize winner for 2025 has not been announced yet!_”

Instead, it confidently walked into the room, sat down, and delivered fiction with the energy of someone defending a PhD dissertation. As a person who once defended a PhD, I wish I had ChatGPT’s confidence when it makes stuff up!

As humans, we tend to do something interesting with confidence; we associate it with correctness, but that’s not always the case. If someone says, “I _think_ the answer might be 42” and another person says, “The answer is _absolutely_ 42,” most of us instinctively trust the second person more, even when both are equally likely to be wrong. For us, confidence sometimes acts as a useful signal of correctness. For AI systems, however, confidence can be a surprisingly unreliable narrator.

In this article, we will explore why.

Confidence Feels Like Probability

Let’s say we asked an LLM to predict what animal it is in a given picture. It says:

markup

Cat: 0.97
Dog: 0.02
Bird: 0.01

Most will interpret that as: ”_The model is 97% sure this is a cat_.”

That is a reasonable assumption. Unfortunately, that is often not what those numbers mean. We need to remember that many AI models use a function called Softmax to generate predictions.

The Softmax function converts raw outputs (called logits) into values that sum to one and resemble probabilities. The important thing to notice here is the exponential term, which can cause small differences to suddenly become very large.

Image by the author

So basically, the model isn’t saying, “I have overwhelming evidence that this is a cat.” It may simply be saying: “Among these options, cat happened to win by a small margin.” Those are very different statements with completely different meanings.

Humans and AI Handle Uncertainty Differently

Though it might be uncomfortable to sit with, humans are surprisingly good at expressing and dealing with uncertainty.

We constantly hear: “_I might be wrong…_”, “_I’m pretty sure…”_, “_Maybe…_”, or “_I think…_”. Our confidence tends to exist on a spectrum. AI systems, however, often behave like that one person in a group project who confidently explains something they learned three minutes ago (I am sure we all had that classmate…).

So, when chatting with an LLM, both telling it “I think Paris is the capital of France,” and it responding “Paris is the capital of France with 99.8% probability,” gives the same energy as telling it “I think Atlantis is fictional,” and it responding “Atlantis is located approximately 400 miles west of Portugal with 98.7% confidence.”

Although the two cases have very different outcomes, the LLM treats them equally.

The Confident Fool Problem

This creates what I think of as the confident fool problem. Where a system can be spectacularly wrong while sounding spectacularly certain. And unfortunately, confidence often increases exactly when we would prefer more caution.

This becomes especially noticeable when LLMs encounter situations outside their training distribution.

Suppose we train an image classifier to identify cats and dogs. But then we decided to give it a picture of a toaster! Ideally, the model should say, “I have absolutely no idea what this is.” What would be the response of most people when shown something they have never seen before? Instead of saying that, the model might respond:

markup

Dog: 98%
Cat: 2%

Now, unless your toaster is poodle-shaped, that answer is clearly false!

Why does this happen? The answer is simpler than most people think. Simply, it happens because the model was never trained to say: “None of the above.” So, when it encounters something unfamiliar, it chooses the highest available score among the options.

It is like forcing someone to answer “_What fruit is this?_” while pointing at a bicycle. Eventually, they will choose a fruit just to resolve the situation and say, “_Banana?_”

Let’s simulate a model that is overconfident.

Image by the author

If the model reports “90% confidence”, we would hope it is correct roughly 90% of the time. Instead, many systems look more like “90% confidence, 65% accuracy.” This gap between confidence and accuracy is why the way we choose to train these LLMs matters a lot.

Teaching Models to Be More Honest

Okay, we know why models tend to be so confidently wrong, but how can we overcome that to have better models with higher accuracy, or accuracy that matches their confidence? This is where calibration comes into play.

Calibration does not necessarily improve predictions. Instead, it improves honesty! So, if a model says 90% after calibration, it should mean: “Historically, predictions at this confidence level were correct about 90% of the time.”

Methods such as:

Platt Scaling
Temperature Scaling
Isotonic Regression

attempt to align predicted confidence with observed outcomes.

Let’s see what this looks like:

Image by the author

Why This Matters

It is easy to laugh when an AI thinks a toaster is a dog. Because that is, arguably, very funny. However, many less funny situations exist. Not just less funny, but critical, and maybe even life-threatening. Using LLMs in medical diagnosis systems, autonomous vehicles, fraud detection, and financial forecasting requires high accuracy.

If a model tells a doctor: “_Cancer probability: 99%_” or “_Cancer probability: 62%_,” the doctor’s response will vary significantly!

If confidence scores are poorly calibrated, people may trust predictions that do not deserve trust. And humans are particularly vulnerable here because confidence feels persuasive. Even when we know better.

As models continue moving into real-world workflows, we may need to stop asking: “_How accurate is the model?_” and start asking: “When the model says 90%, does it actually mean 90%?” Because there is a difference between a smart model and a trustworthy model.

Humans are not perfect at uncertainty, either. We become overconfident all the time. We think we can finish a project in two days. We think we can assemble furniture without reading the instructions. We think we only need one trip from the car to bring in groceries. Even when history suggests otherwise.

Maybe AI is simply inheriting some of our bad habits? The difference is that when humans are confidently wrong, usually only a few people suffer. When AI is confidently wrong, the mistake can scale to millions, and confidence at scale is a very different problem.

Final Thoughts

For years, we have measured AI progress by asking increasingly impressive questions:

_Can it write code? Can it generate art? Can it pass exams? Can it reason?_

Those questions are useful, but they can sometimes distract us from a more important one:

Can we trust it?

A model producing the right answer once is exciting. A model that produces the right answer repeatedly while knowing when it might be wrong is something entirely different. Reliability rarely creates flashy headlines.

Confidence itself is not the problem. The problem begins when confidence becomes a performance rather than a meaningful measure of certainty. As AI systems continue moving into healthcare, education, finance, research, and decision-making pipelines, we may need to stop treating confidence scores as truth meters and start treating them as estimates that require validation.

Because a model sounding certain is easy, where a model knowing when not to be certain may be one of the hardest problems we still have left to solve.