AI2DEV / www.ai2dev.com
Audit AI crawlability, GEO files, indexation, and site-level signals.
Audited website
https://www.ai2dev.com
Last analyzed 10/24/2026, 4:03:03 PM
Technical health score
64
out of 100
Improvements recommended
Across 11 scored checks
Audit result
4
Healthy technical signals
Audit result
6
Improvements recommended
Audit result
1
Visibility blockers found
Action plan
Start here for the changes most likely to improve discovery and eligibility.
1 of 1 sampled pages have missing or conflicting canonicals.
Recommended next step
Add one self-referencing canonical to indexable pages and ensure it matches your HTTPS and www/non-www preference.
2 format issues detected.
Recommended next step
Keep the file focused, add one H1 and a short blockquote summary, and use only absolute links to your strongest canonical pages.
0 of 5 AI agents have an explicit policy.
Recommended next step
Add explicit rules for retrieval and training crawlers. Allow the retrieval bots you want to surface your content and make a deliberate choice for training bots.
AI-readable files, crawler access, bot policy, and sitemap readiness.
2 format issues detected.
A concise Markdown guide that helps AI systems understand your organization and find its most important pages.
A concise Markdown guide that helps AI systems understand your organization and find its most important pages.
A clear, valid file can reduce ambiguity when AI assistants discover and interpret your content. It is an emerging GEO signal, not a guarantee of inclusion.
Keep the file focused, add one H1 and a short blockquote summary, and use only absolute links to your strongest canonical pages.
Technical values collected from the live website.
Bytes
Has H1
Issues
Link Count
Http Status
Has Blockquote
Absolute Urls Only
Optional extended AI-readable file is available.
An optional extended reference containing deeper documentation or supporting context for AI systems.
An optional extended reference containing deeper documentation or supporting context for AI systems.
This can help content-heavy sites provide richer context, but its absence does not prevent crawling or indexing.
No urgent change is needed. Continue monitoring this check after major site updates.
Technical values collected from the live website.
0 of 5 AI agents have an explicit policy.
Checks whether major AI crawlers have clear, intentional access rules instead of relying only on a generic wildcard rule.
Checks whether major AI crawlers have clear, intentional access rules instead of relying only on a generic wildcard rule.
Blocking retrieval crawlers can prevent your pages from appearing in AI answers. Unclear rules also make your AI visibility policy harder to control.
Add explicit rules for retrieval and training crawlers. Allow the retrieval bots you want to surface your content and make a deliberate choice for training bots.
Explicit means the crawler has its own rule. Inherited means it follows the general * rule.
OAI-SearchBot
retrieval · allowed
Inherited from *
PerplexityBot
retrieval · allowed
Inherited from *
GPTBot
training · allowed
Inherited from *
ClaudeBot
training · allowed
Inherited from *
Google-Extended
ai control · allowed
Inherited from *
Technical values collected from the live website.
Retrieval and training crawler policies are shown separately.
Separates crawlers used to retrieve content for answers from crawlers that may collect content for model improvement.
Separates crawlers used to retrieve content for answers from crawlers that may collect content for model improvement.
Retrieval access is directly connected to whether supported AI search products can discover current pages. Training access is a separate business choice.
Review each bot individually. Avoid blocking retrieval by accident, and document the training policy that matches your organization’s preference.
Explicit means the crawler has its own rule. Inherited means it follows the general * rule.
OAI-SearchBot
retrieval · allowed
Inherited from *
PerplexityBot
retrieval · allowed
Inherited from *
GPTBot
training · allowed
Inherited from *
ClaudeBot
training · allowed
Inherited from *
Google-Extended
ai control · allowed
Inherited from *
2 sitemap issues detected.
Checks whether search and AI crawlers can find a valid list of canonical site URLs and understand when pages changed.
Checks whether search and AI crawlers can find a valid list of canonical site URLs and understand when pages changed.
Missing, invalid, or broken sitemap URLs can slow discovery and leave important pages outside search and AI retrieval indexes.
Publish a valid XML sitemap containing live canonical URLs, include accurate lastmod dates, and remove redirected or broken pages.
Technical values collected from the live website.
Bytes
Issues
Url Count
Http Status
Content Type
Lastmod Count
Sampled Url Count
File exists but does not contain a sitemap index.
Checks for an index that organizes multiple sitemap files when the site is too large for one sitemap.
Checks for an index that organizes multiple sitemap files when the site is too large for one sitemap.
Large sites without a sitemap index can become harder to crawl completely. Smaller sites do not need one.
Use a sitemap index when you have multiple sitemaps or approach protocol limits; otherwise no action is required.
Technical values collected from the live website.
Required
Http Status
Sitemap Count
Index controls, canonicals, redirects, and hidden crawl restrictions.
1 valid user-agent group parsed.
Validates crawler rules and looks for directives that may accidentally hide CSS, JavaScript, or important site assets.
Validates crawler rules and looks for directives that may accidentally hide CSS, JavaScript, or important site assets.
Incorrect rules can block entire site areas or prevent crawlers from rendering pages accurately, reducing indexation and answer visibility.
No urgent change is needed. Continue monitoring this check after major site updates.
Technical values collected from the live website.
Group Count
No restrictive directives found across 1 sampled key pages.
Looks for page-level instructions such as noindex, nofollow, and noarchive across a sample of important URLs.
Looks for page-level instructions such as noindex, nofollow, and noarchive across a sample of important URLs.
A noindex directive removes a page from eligible search results and can also eliminate it as a source for AI retrieval.
No urgent change is needed. Continue monitoring this check after major site updates.
Technical values collected from the live website.
Sampled Pages
1 of 1 sampled pages have missing or conflicting canonicals.
Checks whether each sampled page points to the preferred version of itself and uses one consistent hostname.
Checks whether each sampled page points to the preferred version of itself and uses one consistent hostname.
Missing or conflicting canonicals split authority across duplicate URLs and can cause crawlers to index the wrong version.
Add one self-referencing canonical to indexable pages and ensure it matches your HTTPS and www/non-www preference.
Technical values collected from the live website.
Issues
Sampled Pages
HTTP requests permanently redirect to HTTPS.
Verifies that insecure HTTP visits are permanently sent to the secure HTTPS version.
Verifies that insecure HTTP visits are permanently sent to the secure HTTPS version.
A missing or temporary redirect can create duplicate versions, dilute page signals, and reduce trust for users and crawlers.
No urgent change is needed. Continue monitoring this check after major site updates.
Technical values collected from the live website.
The alternate hostname did not clearly redirect to the canonical version.
Checks that www and non-www addresses resolve to one preferred hostname instead of behaving like separate sites.
Checks that www and non-www addresses resolve to one preferred hostname instead of behaving like separate sites.
Two accessible hostnames can duplicate content and divide links, crawl activity, and ranking signals.
Choose one hostname, permanently redirect the alternate version, and use the preferred hostname in canonicals and sitemaps.
Technical values collected from the live website.
No Sitemap directive was found in robots.txt.
Checks whether robots.txt tells crawlers where the sitemap is located.
Checks whether robots.txt tells crawlers where the sitemap is located.
The directive provides another reliable discovery path. Its absence is usually a missed optimization rather than a blocking issue.
Add an absolute Sitemap directive to robots.txt and confirm the referenced file returns a successful response.
Sitemap URL health uses a bounded sample to keep audits fast and safe. Each completed run is saved to this brand's audit history.
