Can You Trust AI Stock Analysis?

Giancarlo Benincasa is the founder of qualiscore.ai, based in Graz, Austria.

You can trust an AI stock analysis as far as you can check it: each number should lead to a source you can open, for the right period, read on a known date.

What AI does well in stock research

A language model is good at reading, as long as it has the right document. Given one, it can find a figure or summarise a risk section quickly.

In FinanceBench, a 2023 benchmark built from questions about 10-Ks, 10-Qs, 8-Ks and earnings reports, GPT-4-Turbo answered 85% of a 150-question sample correctly when it was handed the correct page, and 79% when the full document sat in its context window [1]. With a shared vector-store retrieval setup, the same model got 19% right, which is where the paper's "81%" of wrong or refused answers comes from.

Tools help the same way. In a 2023 study by Kang and Liu, accuracy on historical stock-price questions rose to between 91% and 100% once models could call a price-lookup tool [2].

How AI stock analysis goes wrong

Asked from memory, models invent numbers. In a study accepted at the 2025 Conference on Language Modeling, Shah and colleagues asked seven models, including GPT-4o and GPT-4.5 preview, for a company's revenue in a given year, with no retrieval [3]. Llama-3-70B-Chat was right for 54.17% of companies for 2017 and 6.32% for 1995. The authors found that "LLMs are more likely to hallucinate for larger companies, especially for data from more recent years."

Wrong-period errors are harder to spot because the number itself can be real. Fin-RATE, a benchmark on US SEC filings accepted at KDD 2026, tested 17 models, including GPT-5 with web search [4]. Accuracy dropped by 18.60% when tasks moved from one filing to tracking one company over time, and by 14.35% when comparing companies. The authors associate the drop with "comparison hallucinations, temporal and entity mismatches." In one case, a model recalled two figures correctly, 70% and 65%, and swapped the years they belonged to.

Prices are a weak spot without a tool. In Kang and Liu's 2023 tests, Llama 2 7B models asked for historical prices without a tool had a mean absolute error of about $6,358 and 0% accuracy [2]. GPT-3.5-turbo and GPT-4 declined all such questions. ESMA, the EU securities regulator, warns that AI advice "can be based on outdated, incorrect or incomplete information" [5].

Models also invent sources. In a 2024 study, Erdem, Hassett and Egriboyun asked three chatbots for references on finance topics and checked 150 per model [6]. Depending on the chatbot, between 20.0% and 76.7% of the citations did not exist, and rates tended to be higher for more recent topics. In the 2025 Finance Agent Benchmark, one model, LLaMA 4 Maverick, tried to extract data from documents that did not exist [7].

A confident tone makes all of this harder to catch. FinanceBench's authors describe "superficially coherent and seemingly well-justified answers" [1]. ESMA warns that AI tools "can provide highly convincing and apparently professional advice on investments" [5]. A joint investor alert from the SEC, NASAA and FINRA adds: "Even when based on accurate input, information resulting from AI can be faulty, or even completely made up" [8].

Recent models still miss a large share of real analyst work. In the Finance Agent Benchmark paper (2025), the best model at launch, OpenAI's o3 with search and EDGAR tools, scored 46.8% on 537 expert-written questions that can be checked against SEC filings [7].

How to verify a stock report in five steps

Run each claim your view depends on through these five steps.

1. Claim. Write down the exact statement: the figure, its unit, the period and what it measures, such as revenue for fiscal 2025 in US dollars.
2. Source. Open the cited page. Confirm it exists, that it comes from who it says, and that the number appears on it. A reference you cannot open is unverified.
3. Period. Match the fiscal year or quarter on the source to the one in the claim, and check whether it is a quarter, a full year or a trailing twelve months. This catches the year swaps Fin-RATE documented [4].
4. Calculation. Redo any ratio or growth rate from its inputs. In a 10-K, Item 8 holds the financial statements with the audit report, and the SEC notes they "are accompanied by notes that explain the information presented" [9]. Item 7, MD&A, "allows company management to tell its story in its own words," so read its framing as the company's view.
5. Date. Note when the source was published and when the analysis read it. Prices, estimates and guidance all age.

A claim that fails a step stays unverified until you find a source that holds up.

What a Certainty score should and should not mean

We score the evidence and its direction separately. Certainty (1-10) is how much is known, independent of which way it points. It comes only from how complete the evidence is, how well categories agree, and how much is unresolved. It is never moved by the direction of the evidence. It is not confidence, conviction, or the probability of being right.

A Certainty of 8 says the evidence is fairly complete and consistent. It says nothing about whether the stock will rise or whether the analysis is right.

Bullishness (1-10) carries the direction. It measures how positively the evidence points for a stock relative to what the market already assumes about it. It is a claim about the security at its current price, not about the business alone. The same company is a different Bullishness at 40 and at 80. It is not a rating, recommendation, signal, price target or forecast.

A high Certainty with a middling Bullishness describes well-documented evidence that the price largely reflects. A low Certainty means the evidence is thin, conflicting or unresolved, so your own checks matter more.

ESMA tells investors to "be wary of websites and apps that assert their ability to predict future securities prices with high accuracy" [5]. We publish no accuracy figures, track record or returns, and neither score is a prediction.

How qualiscore documents its sources

qualiscore.ai covers the S&P 500, the S&P MidCap 400, the German DAX and MDAX, and the Austrian ATX. The roughly 90 companies in the weekly run are re-scored every Monday. Other covered companies are analysed when a Premium member requests one (Premium opens at launch). Summaries are public at /stocks/<ticker>, and the full analysis needs a free account.

Each analysis is written by a large language model with live web search. The model searches the web on every run, and we record what it read. Claims in the analysis carry numbered citations. Phase two of the process may only cite pages that phase one actually retrieved; it cannot type a URL. For US companies, key fundamentals come from the company's SEC filings, written in by code rather than by the model. For non-US companies, the balance-sheet figures behind debt to equity come from the company's own published reports. Every analysis page has a "Report an error" link.

Here is how a cited claim works. In our Costco analysis of November 2, 2026, the statement that fiscal 2026 revenue grew 10.14% to $303.15 billion carried a citation to Costco's annual report on Form 10-K at sec.gov. Its income statement shows total revenue of $303,154 million for the 52 weeks ended August 30, 2026, against $275,235 million a year earlier [10]. That is the 10.14%, for the right period, from a filing you can open.

Certainty works the same way. When no consensus forecast exists, the analysis says so, and Certainty is lower: 5.5 for CA Immobilien Anlagen, an Austrian commercial real estate company in the ATX, on November 2, 2026, when its analysis found no consensus earnings or revenue forecasts. Missing forecasts mean less complete evidence, and that is what Certainty measures.

To test this, open any company on the stocks page and follow one citation number to its source with the five steps above. The methodology page defines both scores.

How we keep our own analyses checkable

Live search and numbered citations remove several of the failures above at the source. For everything else, we make each claim easy to check.

Freshness is visible. The roughly 90 companies in the weekly run are re-scored every Monday, and every analysis shows the date it was produced, so you can set it against the company's latest filing.

Periods and figures can be traced. A model reading a real filing can attach a figure to the wrong fiscal year, the failure Fin-RATE documented [4], so every cited figure in our analyses leads back to the page it came from. Open it and check the period and the unit.

Citations are real. Every citation points to a page the model actually read, which makes each claim quick to confirm: open the source and find the number.

Thin evidence shows up in the score. When little is published about a company, Certainty reflects it, so you know where your own checking matters most. If you find a mistake, the "Report an error" link on every analysis page goes straight to us.

FAQ

Can AI hallucinate financial numbers?

Yes. Studies from 2023 to 2026 recorded invented revenue figures, real figures assigned to the wrong year and calculation errors [1][3][4]. The SEC, NASAA and FINRA warn that AI output "can be faulty, or even completely made up" [8].

Is an AI stock rating investment advice?

Our scores are research input, and nothing on qualiscore.ai is investment advice. Bullishness is not a rating, recommendation, signal, price target or forecast. ESMA notes that publicly available AI tools "do not hold any obligation to act in your best interest" and tells investors not to rely solely on them for investment decisions [5].

How do I check a financial claim?

Pin down the exact claim, open the cited source, confirm the period, redo any calculation and note the dates. For US companies, the 10-K's Item 8 holds the financial statements and their notes [9].

How often should an AI analysis be updated?

At least as often as the company publishes new results or material news, because an analysis is only as current as its newest source. The roughly 90 companies in qualiscore's weekly run are re-scored every Monday.

This article is for information only and is not investment advice.

Sources

1. Islam, Kannappan, Kiela, Qian, Scherrer, Vidgen, "FinanceBench: A New Benchmark for Financial Question Answering", arXiv:2311.11944. https://arxiv.org/abs/2311.11944. Published 2023-11-20, accessed 2026-11-08.
2. Haoqiang Kang, Xiao-Yang Liu, "Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination", arXiv:2311.15548. https://arxiv.org/abs/2311.15548. Published 2023-11-27, accessed 2026-11-08.
3. Agam Shah, Liqin Ye, Sebastian Jaskowski, Wei Xu, Sudheer Chava, "Beyond the Reported Cutoff: Where Large Language Models Fall Short on Financial Knowledge", arXiv:2504.00042, accepted at CoLM 2025. https://arxiv.org/abs/2504.00042. Published 2025-03-30 (v2 2025-07-28), accessed 2026-11-08.
4. Yidong Jiang, Junrong Chen, Eftychia Makri, Jialin Chen, Peiwen Li, Ali Maatouk, Leandros Tassiulas, Eliot Brenner, Bing Xiang, Rex Ying, "Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings", arXiv:2602.07294, KDD '26. https://arxiv.org/abs/2602.07294. Published 2026-02-07 (v4 2026-06-10), accessed 2026-11-08.
5. European Securities and Markets Authority (ESMA), "Using Artificial Intelligence for Investing: What you should consider". https://www.esma.europa.eu/sites/default/files/2025-03/ESMA_Warning_on_the_use_of_AI_-_EN.pdf. Published 2025-03-25, accessed 2026-11-08.
6. Orhan Erdem, Kristi Hassett, Feyzullah Egriboyun, "Evaluating the Accuracy of Chatbots in Financial Literature", arXiv:2411.07031. https://arxiv.org/abs/2411.07031. Published 2024-11-11, accessed 2026-11-08.
7. Antoine Bigeard, Langston Nashold, Rayan Krishnan, Shirley Wu (Vals AI and Stanford), "Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks", arXiv:2508.00828. https://arxiv.org/abs/2508.00828. Published 2025, accessed 2026-11-08.
8. SEC Office of Investor Education and Advocacy, NASAA and FINRA, "Artificial Intelligence (AI) and Investment Fraud" (Investor Alert), FINRA. https://www.finra.org/investors/insights/artificial-intelligence-and-investment-fraud. Published 2024-01-25, accessed 2026-11-08.
9. U.S. Securities and Exchange Commission, "How to Read a 10-K". https://www.sec.gov/answers/reada10k.htm. Last modified 2011-07-01, accessed 2026-11-08.
10. Costco Wholesale Corporation, Annual Report on Form 10-K for the fiscal year ended August 30, 2026, Consolidated Statements of Income. https://www.sec.gov/Archives/edgar/data/909832/000090983226000093/cost-20260830.htm. Accessed 2026-11-08.