Toward an AI-Native Internet: Rethinking the Web Architecture for Semantic Retrieval

TL;DR

This paper proposes an AI-Native Internet architecture that enhances semantic retrieval by exposing information in chunks and enabling AI applications to discover relevant sources efficiently, addressing current web inefficiencies.

Contribution

It introduces the concept of an AI-Native Internet with semantic chunks and a Web-native resolver, outlining architectural directions and challenges for future web evolution.

Findings

Current HTML-based retrieval is inefficient for AI needs.

Semantic chunking improves information discovery for AI.

Open challenges include designing new web protocols and standards.

Abstract

The rise of Generative AI Search is fundamentally transforming how users and intelligent systems interact with the Internet. LLMs increasingly act as intermediaries between humans and web information. Yet the web remains optimized for human browsing rather than AI-driven semantic retrieval, resulting in wasted network bandwidth, lower information quality, and unnecessary complexity for developers. We introduce the concept of an AI-Native Internet, a web architecture in which servers expose semantically relevant information chunks rather than full documents, supported by a Web-native semantic resolver that allows AI applications to discover relevant information sources before retrieving fine-grained chunks. Through motivational experiments, we quantify the inefficiencies of current HTML-based retrieval, and outline architectural directions and open challenges for evolving today's document-centric web into an AI-oriented substrate that better supports semantic access to web content.