HTTP Under the Hood: Here's What Actually Happens

27 March 2026 · 10 min read

Most backend engineers know HTTP as a stateless request-response protocol. That is correct, but it is not the whole story. I have been reading about HTTP in more depth recently, and in this post I want to explain what happens under the hood.

In This Post

What is HTTP?
What is a protocol?
What happens when you browse to a website?
How is this HTTP structured
How the server actually reads HTTP request
HTTP/1.0 vs HTTP/1.1 vs HTTP/2 and HTTP/3
Compression
HTTP Connection Management
HTTP Caching

HTTP is an application-layer protocol used to transfer representations of resources between clients and servers. It follows a request-response model in which a client sends a request and receives a response. The response might contain HTML, an image, JSON, a video, or another representation. Let’s start with the basic idea of a protocol.

What Is Protocol?

A protocol is a set of rules and procedures that need to be followed. In computer science the concept is also the same. A protocol defines rules that computers follow when communicating. Some protocols define both the structure and meaning of messages, while the underlying transport provides delivery characteristics. For example, HTTP/1.1 and HTTP/2 are commonly transported over TCP, while HTTP/3 uses QUIC over UDP. HTTP defines the semantics and message structure; the transport provides the underlying delivery mechanism.

So HTTP defines the semantics and message format, while the transport used by that HTTP version carries those bytes between endpoints.

For every protocol, a detailed document is published that contains all the rules that you need to follow while working with that protocol. The HTTP document can be found here.

What Happens When You Browse Any Website?

You opened your browser and typed sushantdhiman.dev in the URL bar and pressed enter, and magically, somehow, my website opened. But how did this thing happen? There are several steps involved in it.

Let’s remove the “magic” part.

Step 1: Your Browser Resolves the Domain Name

Humans like names, but computers need an address. Your browser cannot send a request to sushantdhiman.dev. It needs an IP address. So the first thing it does is try to get IP address associated with this domain. This is where DNS comes in. Think of DNS as a distributed phone book of the internet. Your browser asks:

“Give me the IP address for this domain.”

IP addresses can be cached by the browser, operating system, local resolver, or other DNS infrastructure. If a cached answer is unavailable, a DNS resolver performs the necessary queries to obtain an address.

Now the browser finally knows where to send the request.

Step 2: A Connection Is Established

For HTTP/1.1 and HTTP/2 over TCP, the client first establishes a TCP connection. For HTTPS, TLS is then negotiated before application data is exchanged. HTTP/3 is different: it uses QUIC over UDP, with TLS 1.3 integrated into QUIC.

Before any data is sent, your machine and the server establish a connection using the 3-way handshake. It is called a 3-way handshake because it requires 3 steps. A lot happens while creating connection, but we will stick to abstraction for simplicity. Here are those 3 steps:

(SYN) - Client - “Hey, can we talk?”
(SYN-ACK) - Server - “Yes, I’m ready.”
(ACK) - Client - “Cool, let’s start.”

After this, you now have a reliable communication channel. But HTTP hasn’t even started yet.

Step 3: The Secure Transport Is Established

For HTTPS, there is an additional TLS step before normal HTTP messages are exchanged. HTTP/1.1 and HTTP/2 are commonly carried over TLS over TCP, while HTTP/3 maps HTTP semantics onto QUIC over UDP.

Now your browser sends an HTTP request. A simple HTTP request looks something like this:

1GET / HTTP/1.1
2Host: sushantdhiman.dev
3User-Agent: Mozilla/5.0 ...
4Accept: text/html

Don’t worry; we will talk about this. It’s literally a string following a specific format defined by the HTTP protocol. This is the “rules” part we talked about earlier.

Step 4: Server Sends Response

The server sends something like:

1HTTP/1.1 200 OK
2Content-Type: text/html
3
4<html>...</html>

Again — just structured data. Your browser will then render the received HTML, and then you’ll see a web page.

How Is HTTP Structured

Most engineers say “HTTP is just text”. That’s true, but again, incomplete. HTTP is not just text. It’s structured text with very strict rules. If you break those rules even slightly, the server won’t “try to understand you”. It will just reject the request. Let’s look at what actually gets sent.

When your browser sends a request, it’s not sending some JSON object or fancy abstraction. It literally writes bytes to a TCP socket in a very specific format.

A typical request looks like this:

1GET /blog/http-deep-dive HTTP/1.1
2Host: sushantdhiman.dev
3User-Agent: Mozilla/5.0
4Accept: text/html
5Connection: keep-alive

For HTTP/1.x, the request line comes first, followed by headers, an empty line, and an optional body.

Request Line

GET /blog/http-deep-dive HTTP/1.1

This line tells the server three things:

What action you want (GET)
What resource you want (/blog/http-deep-dive)
Which version of HTTP you’re speaking (HTTP/1.1)

If you mess this line up, nothing else matters.

Headers

Headers are just key-value pairs. If you have built an HTTP API, you already know what headers are.

1Host: sushantdhiman.dev
2User-Agent: Mozilla/5.0
3Accept: text/html

This is where most real-world behavior is controlled. For example:

Host tells the server which website you’re trying to reach (important for shared servers)
User-Agent tells what kind of client is making the request
Accept tells what kind of response you can handle

The Host header is required in HTTP/1.1 requests, while headers such as User-Agent and Accept are optional and depend on the client and use case.

Empty Line

Then comes an empty line. It acts as the delimiter between the header section and the optional message body.

If you forget this, the server will keep waiting, thinking more headers are coming.

Body

The body is optional. POST, PUT, and PATCH commonly carry request bodies, but HTTP does not impose a universal rule that only those methods can have one. A body can contain JSON, text, binary data, or another representation.

1{
2  "name" : "Sushant"
3}

How the server actually reads HTTP request

This is where things get interesting. The server is not receiving a ready-made object; it is parsing bytes from the transport according to the HTTP version being used.

Something like this (conceptually):

Read until \r\n → that’s your request line
Keep reading lines until you hit an empty line → those are headers
Check Content-Length → read that many bytes → that’s your body

That’s it. There’s no magic parsing engine. Just careful reading of a byte stream. This is why malformed requests break everything. The server is not guessing — it’s following strict rules.

Server Response

The server also follows the same structure for the response as we used for the request. Now your browser is responsible for parsing the response and showing you the result.

HTTP Versions

There are several HTTP versions, but the major versions in common use are HTTP/1.0, HTTP/1.1, HTTP/2, and HTTP/3. The core HTTP semantics remain similar, while message framing, multiplexing, and transport behavior changed significantly across versions.

HTTP/1.0

HTTP/1.0 commonly used a non-persistent connection model: a request would be sent over a TCP connection and the connection would usually close after the response. A page with many resources could therefore require many connections.

Each new TCP connection requires a three-way handshake, and TCP congestion control also has to ramp up. Repeating this for many resources adds latency and overhead.

HTTP/1.1

HTTP/1.1 didn’t reinvent HTTP. It fixed the obvious inefficiencies. The biggest change was persistent connections. Now instead of closing the connection after one request, we can reuse it.

HTTP/1.1 introduced persistent connections. HTTP pipelining could send multiple requests without waiting for each response, but responses had to be returned in order. A slow response could therefore block later responses at the HTTP layer.

Example:

Request A (slow)
Request B (fast)

Even if B is ready first, it has to wait for A so now we have a faster connection, but still limited by ordering.

HTTP/2

HTTP/2 didn’t change what HTTP means. But it completely changed how data is sent. Some notable improvements were.

Multiplexing — HTTP/2 can carry multiple request and response streams concurrently over a single TCP connection. Independent HTTP/2 streams do not have to wait for one another at the HTTP framing layer, although TCP-level packet loss can still stall all streams on the connection.
Binary Protocol - HTTP/1.x was human-readable. HTTP/2 is binary framed. You won’t see GET / HTTP/1.1 Instead, data is split into frames and encoded efficiently. This improves parsing speed, network efficiency and compression but you lose readability. Although you can use other tools for that.
Header Compression - Headers in HTTP are often repetitive like User-Agent, Cookie, and Accept. Sending them again and again is wasteful. HTTP/2 uses HPACK compression to reduce this overhead. In real systems, this saves a lot of bandwidth.

HTTP/3

HTTP/3 keeps the same core HTTP semantics but uses QUIC instead of TCP. QUIC runs over UDP and provides reliable, encrypted, multiplexed streams. Because streams are handled independently by QUIC, packet loss on one stream does not have to block progress on unrelated streams in the same way that TCP-level head-of-line blocking can affect HTTP/2.

Compression

When you send an HTTP response, you are sending bytes over a network. Network latency and bandwidth can be significant bottlenecks, so reducing the number of bytes can improve performance. That is the idea behind HTTP content compression.

The client can tell the server which content codings it accepts using request headers such as:

Accept-Encoding: gzip, deflate, br

This means: “I can understand compressed responses using these algorithms.” Now the server decides if it supports compression, which algorithm to use and then responds like this:

Content-Encoding: gzip

The received response body is compressed, and the browser can use the gzip algorithm to decompress it.

Important detail most people miss that HTTP is not compressing anything by itself. It doesn’t “know” gzip or brotli. It just defines headers like: Accept-Encoding Content-Encoding . The actual compression is done by the server (or proxy like Nginx/CDN). Again — HTTP defines rules, not behavior.

HTTP Connection Management

You can build a perfectly working service and still take it down with bad connection management. Not because your logic is wrong, but because your connections are. For HTTP/1.1 and HTTP/2, connection management includes handling the underlying TCP connections. HTTP/3 uses QUIC instead.

Persistent Connections and Keep-Alive

Persistent connections let a client reuse a TCP connection for multiple HTTP requests and responses. Keeping connections open consumes resources such as file descriptors, memory, and kernel state, so connection reuse is a tradeoff rather than free performance.

Idle Timeouts

If you keep connections alive forever, your server will run out of resources. So every system introduces idle timeouts. “If no request comes for X seconds, close the connection.” Simple idea. Subtle problems. But!!!!

If your timeout is too low:

Connections close too often → more TCP handshakes → latency

If it’s too high:

You waste resources on idle clients

Connection Pooling

Modern clients are smart enough. They don’t open a new connection per request. They maintain a connection pool. Instead of: open → use → close. They do: open → reuse → reuse → reuse. For example: Browsers reuse connections per domain Backend services (Go, Node, Java) maintain pools internally. This improves performance a lot.

Load Balancer

In real systems, your client is not directly talking to your server. There’s usually a load balancer in between. Example flow: Client → Load Balancer → Backend . Now here’s something subtle: Client ↔ LB connection LB ↔ Backend connection These are two different connections. Load balancers often reuse backend connections aggressively.

HTTP Caching

We can cache HTTP responses and reduce load on the server. The core idea is that when a client receives a response, it can store it and reuse it later instead of hitting the server again. But this is not random. HTTP caching is controlled explicitly using headers.