How the Web Works: From URLs to DNS, HTTP, and Rendering

This document explains the core foundations of the World Wide Web, covering the entire journey from typing a URL into a browser to a fully rendered, interactive page. It details the underlying infrastructure like the Internet and IP addressing, the Domain Name System (DNS) for resolving names, the Hypertext Transfer Protocol (HTTP) for communication, and the browser's process of rendering content and managing performance.

Internet vs. the Web

While often used interchangeably in everyday language, the Internet and the Web refer to different layers of technology:

  • Internet: This is the global infrastructure—the physical network of cables, routers, addressing systems (like IP), and protocols that enable machines to move packets of data between them. It is the substrate that supports all kinds of networked applications.
  • The Web: This is an application built on top of the Internet. It is a system for sharing and linking documents (primarily HTML) via hyperlinks. The links you click to navigate between resources are a defining feature of the Web.
  • Other Applications: Many other services, such as email, voice calls, and video streaming, also run over the Internet but are distinct from the Web, using their own specific protocols and services.
    This distinction is crucial because understanding Web behavior—which involves technologies like HTTP(S), URLs, browsers, and servers—requires recognizing it as one of many applications leveraging the foundational Internet infrastructure.

The Journey of a Web Request: A High-Level Overview

When you enter a URL like https://example.com into your browser and press Enter, a sequence of steps occurs before the page appears:

  1. Name-to-Number Translation: The human-readable host name (example.com) is resolved to a numeric IP address through the Domain Name System (DNS).
  2. Connection Establishment: A network connection is opened to the server at that IP address, typically on a specific port.
  3. Encryption and Authentication: If using HTTPS, a secure TLS session is negotiated. The server presents a certificate to prove its identity, and the connection becomes encrypted.
  4. Request/Response: The browser sends an HTTP request to the server, which processes it and returns an HTTP response.
  5. Rendering: The browser parses the response (e.g., an HTML document), builds a visual representation of the page, and fetches any additional resources referenced, like CSS, JavaScript, and images.
    Each step is essential: DNS enables routing, ports identify services, TLS secures traffic and authenticates servers, and HTTP carries the application-level instructions of the Web.

URLs: Structure and Semantics

A URL (Uniform Resource Locator) is the "address" you type into the browser's address bar. It comprises distinct parts, each with a specific meaning. Consider this example:
https://developer.mozilla.org/en-us/search?q=status#content
Key components include:

  • Scheme (Protocol): https specifies the communication protocol. It tells the client how to access the resource. Common schemes include:
    • https: Hypertext Transfer Protocol Secure (HTTP over TLS) for encrypted web traffic. Default port is 443.
    • http: HTTP without encryption. Default port is 80.
    • ftp: File Transfer Protocol, used for file transfers.
  • Host: developer.mozilla.org is the server’s domain name—the machine your client needs to contact. DNS resolves this name to an IP address.
  • Port: A port is like a "service window" on a server, allowing multiple applications to run on the same machine. The port is usually implicit, determined by the scheme (80 for http, 443 for https). It can be explicitly specified (e.g., developer.mozilla.org:8000) to reach a service on a non-default port. If no service is listening on the target port, the connection is refused.
  • Path: /en-us/search indicates the specific resource you want within the server. Conceptually, it's like a folder/subfolder path, expressing "where" on the host you want to go.
  • Query String: ?q=status passes variables or parameters with the request, similar to function arguments. It starts with a ? and consists of key-value pairs. For example, searching "hola" on Google produces a URL like https://www.google.com/search?q=hola. Websites define their own parameter names to control features like filtering, sorting, or search terms.
  • Fragment: #content navigates the browser to a specific section of the resource, identified by an element with a matching id in the HTML. This action is handled entirely client-side; it does not trigger a new request to the server. It's commonly used in long pages with internal navigation.

IP Addresses and the Domain Name System (DNS)

While humans use domain names, networks route data using IP addresses. The Domain Name System (DNS) is the critical bridge between these two.

IP Addresses: Logical Network Locations

An IP address is a logical network address used to route packets to a device. It is not a fixed physical location; the same logical address can be used by different devices over time.

  • IPv4: The older version, consisting of four numbers (octets) from 0–255, separated by dots (e.g., 192.0.2.42). Its ~4 billion addresses have been largely exhausted.
  • IPv6: The modern version, using hexadecimal numbers in colon-separated groups (e.g., 2001:db8::1). It provides a vastly expanded address space.
  • Public vs. Private IPs: Inside a local network (like a home Wi-Fi), devices have private IP addresses. When they access the internet, their traffic often exits through a router using a single, shared public IP address. This is managed by a process called Network Address Translation (NAT), which helps conserve public IP addresses.

DNS: Translating Names to Numbers

The Domain Name System (DNS) is a distributed, hierarchical system that maps human-readable domain names (e.g., wandes.cl) to machine-readable IP addresses.

How a DNS Lookup Works

  1. Client Request: Your browser asks a DNS resolver (often your ISP's, but configurable) for the IP of a domain.
  2. Hierarchical Lookup: The resolver queries the DNS hierarchy. It may start by asking root servers, which direct it to the appropriate Top-Level Domain (TLD) servers (e.g., for .com or .cl).
  3. Authoritative Nameserver: The TLD server provides the address of the domain's authoritative nameserver—the ultimate source of truth for that specific domain's records.
  4. Record Retrieval: The resolver queries the authoritative nameserver, which returns the requested DNS records (e.g., an A record containing the IPv4 address).
  5. Response to Client: The resolver sends the IP address back to the browser.

DNS Caching and TTL

To improve performance and reduce load, DNS responses are cached. Each record includes a Time To Live (TTL) value, which tells resolvers how long they can store and reuse the answer. Once the TTL expires, the resolver must perform a full lookup again. This mechanism ensures that changes to DNS records (like moving a website to a new server) eventually propagate across the internet as caches expire. While propagation is often fast, it can take up to 72 hours in some cases due to varying cache configurations.

Common DNS Record Types

  • A: Maps a name to an IPv4 address.
  • AAAA: Maps a name to an IPv6 address.
  • CNAME (Canonical Name): An alias that points one name to another name (not directly to an IP).
  • MX (Mail Exchanger): Specifies the mail servers responsible for handling email for the domain.
  • TXT: Contains arbitrary text, often used for domain verification, email security policies (SPF/DKIM), and other metadata.

Practical DNS Setup and Delegation

When managing a domain, you typically interact with two key entities: a registrar and a DNS provider.

  1. Registrar: The entity where you purchase and manage domain ownership (e.g., NIC Chile for .cl domains). Some registrars limit your configuration to only setting the domain's authoritative nameservers (NS records). This means you cannot configure A or CNAME records directly there.
  2. DNS Provider: A service (like Cloudflare) that acts as your authoritative nameserver. You delegate authority from your registrar to this provider.
    A common workflow is:
  3. Purchase a domain from a registrar (e.g., NIC Chile).
  4. In the registrar's control panel, set the domain's nameservers to those provided by your chosen DNS provider (e.g., ns1.cloudflare.com).
  5. In the DNS provider's dashboard (e.g., Cloudflare), create and manage all your records (A, CNAME, MX, etc.) to point to your web server's IP, email provider, and other services.
    For example, GitHub Pages uses DNS to host sites. You can map a custom domain to your GitHub Pages site by adding specific A and CNAME records in your DNS provider's settings, pointing your domain to GitHub's hosting infrastructure.

The HTTP Protocol: Client-Server Communication

The Hypertext Transfer Protocol (HTTP) is the foundational protocol of the Web. It operates on a client-server, request-response model: the client always initiates a request, and the server always sends a response. This simple asymmetry underlies all web interactions.

The Anatomy of an HTTP Request and Response

HTTP messages are structured plain text.

HTTP Request Structure

A request from a client (e.g., a browser) includes:

GET / HTTP/1.1
Host: uandes.cl
Accept: text/html
User-Agent: Mozilla/5.0
  • Method (Verb): GET specifies the desired action (retrieve a resource).
  • Path: / is the path to the resource on the server (the root or homepage).
  • Protocol Version: HTTP/1.1 indicates the protocol version.
  • Headers: Key-value pairs providing additional context.
    • Host: uandes.cl: Specifies the server's domain.
    • Accept: text/html: Tells the server the client can handle an HTML response.
    • User-Agent: Mozilla/5.0: Identifies the client software.

HTTP Response Structure

A response from the server includes:

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Content-Length: 12345
Set-Cookie: application_gateway=...
Server: Apache

...
  • Status Code and Message: 200 OK indicates the request was successful.
  • Headers: Key-value pairs describing the response.
    • Content-Type: text/html; charset=UTF-8: Specifies the media type and character encoding of the body.
    • Content-Length: 12345: The size of the response body in bytes.
    • Set-Cookie: ...: An instruction for the browser to store a cookie.
    • Server: Apache: Identifies the server software.
  • Body: The actual content (e.g., the HTML document), which follows a blank line.
    You can inspect these exchanges in real-time using your browser's Developer Tools under the "Network" tab.

HTTP Methods (Verbs)

HTTP methods define the intended action on a resource. Using the correct verb is a crucial convention for building robust and secure applications.

  • GET: Retrieves data. Should be idempotent (repeatable without side effects) and should not alter server state.
  • POST: Submits data to create a new resource (e.g., submitting a form). Data is sent in the request body, which is more secure for sensitive information than passing it in the URL.
  • PUT / PATCH: Updates an existing resource. PUT typically replaces the entire resource, while PATCH applies a partial modification.
  • DELETE: Deletes a specified resource.

HTTP Status Codes

Status codes are grouped by their first digit to signal the outcome of a request.

  • 2xx (Success): The request was successful. (200 OK, 201 Created).
  • 3xx (Redirection): The client needs to take further action, usually by following a new URL. (301 Moved Permanently, 302 Found). The new URL is provided in the Location header.
  • 4xx (Client Error): The error is on the client's side. (404 Not Found, 403 Forbidden).
  • 5xx (Server Error): The server failed to fulfill a valid request. (500 Internal Server Error).

The Stateless Nature of HTTP and the Role of Cookies

A fundamental characteristic of HTTP is that it is stateless. Each request is an independent transaction, and the server retains no memory of previous requests from the same client. This poses a problem for applications needing to maintain a user session (e.g., knowing you are logged in).
The solution is cookies. A cookie is a small piece of data that a server asks a browser to store.

  1. After a user logs in, the server sends a response with a Set-Cookie header containing a unique session identifier.
  2. The browser stores this cookie and automatically includes it in all subsequent requests to that same domain.
  3. The server uses the cookie to look up the session, identify the user, and process the request in the correct context.
    Cookies are essential for persisting identity and state. However, third-party cookies (set by a domain different from the one you are visiting) have been widely used for cross-site tracking, leading to privacy concerns. In response, legal frameworks like GDPR have emerged, and browsers are increasingly restricting their use.

Securing the Web with HTTPS

HTTPS (Hypertext Transfer Protocol Secure) is HTTP layered over an encrypted connection (TLS/SSL). It provides three key security guarantees:

  1. Confidentiality: Encrypts data in transit, preventing eavesdroppers from reading it.
  2. Integrity: Ensures data cannot be altered in transit without detection.
  3. Authentication: Verifies that you are communicating with the legitimate server, not an imposter. This is achieved via SSL/TLS certificates issued by trusted Certificate Authorities (CAs). When your browser connects via HTTPS, it validates the server's certificate. If it's valid and trusted, a secure connection is established.

Evolution to HTTP/2

  • HTTP/1.1: Created a bottleneck by requiring a new connection for each resource (image, script, etc.), forcing sequential downloads.
  • HTTP/2: Introduced multiplexing, allowing a browser to download multiple files in parallel over a single connection. This dramatically improves page load times for modern, asset-heavy websites.

From Response to Render: What the Browser Does

Once the browser receives the initial HTML response from the server, its work is far from over.

  1. Parsing and DOM Construction: The browser parses the HTML and constructs the Document Object Model (DOM), an in-memory tree-like representation of the page's structure.
  2. Resource Discovery: As it parses, it discovers references to external resources like CSS stylesheets, JavaScript files, images, and fonts.
  3. Fetching Additional Resources: The browser issues new HTTP requests to fetch each of these resources.
  4. Render-Blocking Resources: Some resources, particularly CSS and synchronous JavaScript, are render-blocking. The browser may pause rendering the page until these critical resources are downloaded and processed. To mitigate this, developers often place scripts at the bottom of the <body> or use async and defer attributes.
  5. Rendering: Finally, the browser combines the DOM and CSS (via the CSSOM) to create a render tree, calculates layout, and "paints" the pixels to the screen.

Web Performance and User Experience

Performance is not just a technical metric; it directly impacts user behavior and business outcomes. Users are impatient and will quickly abandon slow-loading or unresponsive sites.

Key Performance Metrics (Core Web Vitals)

  • LCP (Largest Contentful Paint): Measures loading performance. It marks the point when the page's main content has likely loaded. A good target is under 2.5 seconds.
  • INP (Interaction to Next Paint): Measures responsiveness. It assesses the latency of all user interactions (clicks, taps, key presses) and reports the worst one. An INP under 200 milliseconds feels instantaneous to the user.
  • CLS (Cumulative Layout Shift): Measures visual stability. It quantifies how much unexpected layout shifts occur as content loads. Low CLS ensures a smooth, non-frustrating user experience.

Context and Business Relevance

User tolerance for delays varies. Users may be more patient with an "obligatory" institutional portal than with an optional entertainment or e-commerce site. For transactional websites, studies consistently show that even small delays can significantly reduce conversion rates and revenue. Optimizing performance, therefore, has a direct and measurable return on investment.