Free checkcnbc.com
Can Google and AI find cnbc.com?
61 / 100Score out of 100: the fewer and smaller the problems below, the higher. Checked 2026-09-29 12:30 UTC from the outside, the way a crawler sees it.
Fix these first
The problems that cost you the most visibility, worst first. Each one says why it matters and how to fix it.
1. 4 AI answer engines are blocked
robots.txt keeps out ChatGPT search, ChatGPT reading a link, Perplexity, Gemini (Google AI). They cannot read or cite your pages when people ask them.
Fix: If you want to be recommended in AI answers, allow OAI-SearchBot, ChatGPT-User, PerplexityBot, Google-Extended in robots.txt. Blocking training bots such as GPTBot is fine; these are the search ones.
2. No main heading (H1) on your home page
Search engines and AI use the main heading to understand what the page is about.
Fix: Add one H1 that names what you offer.
3. No llms.txt file
An optional file that gives AI tools a plain summary of your site. Google says it does not affect search; coding assistants and agents do read it. Nice to have, not a ranking factor.
Fix: Add /llms.txt with one paragraph about you and links to your key pages.
Who is allowed to read your site
Read from your robots.txt. Blocked means that search engine or AI cannot read your pages, so it cannot show or cite them.
- Google Search Googlebotallowed
- Bing (also feeds ChatGPT search and Copilot) Bingbotallowed
- ChatGPT search OAI-SearchBotblocked
- ChatGPT reading a link ChatGPT-Userblocked
- Perplexity PerplexityBotblocked
- Claude search Claude-SearchBotallowed
- Gemini (Google AI) Google-Extendedblocked
- OpenAI model training GPTBot training onlyblocked (your choice)
- Anthropic model training ClaudeBot training onlyblocked (your choice)
From your robots.txt.
Already good
Nothing to do here.
- Google and Bing are allowed to read your site
- Sitemap found with 50,000 pages
- Served over https
- Home page title is a good length
- Structured data found (CollectionPage)
Get an email if cnbc.com breaks for Google or AI
We re-check it every week and write only when something new goes wrong: a crawler gets blocked, a page is marked noindex, the sitemap breaks. Free. No account. One click to stop.
This is what a crawler sees. Now see what people do.
Connect Google Search Console and SearchLink tells you which pages lost clicks and why, checks every week, and emails you the 3 things to fix. Free for up to 3 sites.