robots.txt for AI Crawlers
5/10
10/10
+5
Table & List Extractability
3/10
5/10
+2
Definition Patterns
5/10
7/10
+2
Fact & Data Density
8/10
10/10
+2
Content Cannibalization
10/10
9/10
-1
Helpful Purpose Alignment
6/10
7/10
+1
Methodology Transparency
2/10
3/10
+1
Citation-Ready Writing Quality
3/10
6/10
+3
Owned Data Density
0/10
6/10
+6