robots.txt for AI Crawlers
5/10
6/10
+1
Table & List Extractability
3/10
7/10
+4
Definition Patterns
5/10
7/10
+2
Content Publishing Velocity
10/10
7/10
-3
Query-Answer Alignment
7/10
10/10
+3
Content Cannibalization
7/10
10/10
+3
Visible Date Signal
0/10
6/10
+6
Topic Coherence
8/10
9/10
+1
Helpful Purpose Alignment
6/10
5/10
-1
Creator Transparency
3/10
1/10
-2
Evidence Packaging
2/10
4/10
+2
Entity Disambiguation
4/10
6/10
+2
Extraction Friction Score
3/10
5/10
+2
Duplicate Content Blocks
10/10
9/10
-1
Answer Capsule Pattern
6/10
4/10
-2
Sentence Atomicity
10/10
8/10
-2