HTTP Requests 101: A comprehensive guide to understanding requests

APIs play a fundamental role in developers’ daily lives by providing an efficient and versatile way to access and manipulate data on web servers.

In this article, we will delve into the concept of HTTP requests, exploring their components and execution in various programming environments. Additionally, we will examine in detail how these requests are transmitted across a network, supporting our analysis with clear and concise examples.

What is an HTTP Request?

An HTTP request, also known as an HTTP call, is the request that a client, such as a web browser, sends to a server. This request is composed of various elements and must follow a standard format for the server to interpret it correctly.

The format of an HTTP request, called the HTTP message, is extensively described in RFC documents (Request for Comments in English), in this case, RFC 7230. These documents detail the technical specifications, standards, and protocols related to HTTP communications.

HTTP messages consist of three main elements: the request line, headers, and body. These components work together to form a complete and understandable request for the server, allowing clients to request, create, modify, or delete information efficiently.

HTTP Request Components

Request Line

The first line of the message, which includes:

• Method. The action we want to perform. For example, GET is commonly used to request data and POST to send data.
• URI. The path or location of the resource. For example, /users.
• Version. The version of the HTTP protocol we are using. For example, HTTP/1.1 or HTTP/2.

These three parts, when combined, form the complete line:

POST /post HTTP/1.1

Headers

Headers are lines of additional information or metadata sent along with the request. Some very common headers (although there are many others) include:

• Host. The only required and most important header. It indicates the name of the server.
• Accept. Indicates the types of formats that the client is capable of understanding in the response, such as HTML, JSON, or XML.
• Content-Type. Indicates the format of the data to be sent in the body (we will explain what the body is in the next section). This header only makes sense in requests that send data, such as POST, PUT, or PATCH requests.
• User-Agent. Typically indicates the name and version of the client being used, such as: curl/8.5.0.

To illustrate these concepts, let’s consider a practical example of an HTTP request with its respective headers:

Host: httpbin.org
Accept: application/json
Content-Type: application/json
User-Agent: curl/8.5.0

This set of headers informs the httpbin.org server that cURL version 8.5.0 is being used, and that a response in JSON format is accepted.

Body

Finally, the body of the request is included only in those HTTP messages that want to send additional data to the server. As mentioned earlier, this component is added in requests that use the POST, PUT, or PATCH methods, allowing the transfer of additional information along with the request.

Keep in mind that to define the format of the data you are sending, it is necessary to use the Content-Type header. This directive tells the server how to interpret the information in the body. For example, if you are sending data in JSON format, the Content-Type header should be application/json; while if you are sending an HTML form, you should use application/x-www-form-urlencoded.

An example of a body could be this:

{
  "hello": "world"
}

Summary

Bringing together the examples mentioned earlier, we will execute a request and create a real HTTP message.

To do this, we will use cURL, a well-known client that allows making HTTP calls from the terminal.

curl -X POST 'https://httpbin.org/post' \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data '{ "hello": "world" }'

This cURL command is sending a POST request to the path /post on the server httpbin.org, indicating that it accepts responses in JSON format and is sending data in JSON format.

The HTTP message generated for the above command would be something like this:

POST /post HTTP/1.1
Host: httpbin.org
Accept: application/json
Content-Type: application/json

{ "hello": "world" }

How is an HTTP request transmitted?

Once we have formatted our HTTP message, how do we transmit it to the server? Through a TCP socket. This process is essential to understand how information is sent and received between the client and the server in a network.

For example, in Ruby, we can execute this using TCP sockets:

require 'socket'

# Create a TCP socket
client_socket = TCPSocket.new('httpbin.org', 443) # 443 port for HTTPS

# Define the HTTP request
http_request = <<~HTTP_REQUEST
  POST /post HTTP/1.1
  Host: httpbin.org
  Accept: application/json
  Content-Type: application/json

  { "hello": "world" }
HTTP_REQUEST

# Send request to the server
client_socket.puts http_request

# Handle the server response
response = client_socket.read
puts "Respuesta del servidor:"
puts response

# Close the connection
client_socket.close

Simplification with Libraries

Generating HTTP messages and writing repetitive code every time we want to make a request to an API can be a tedious process. Fortunately, there are libraries that simplify this process, both built into languages and external ones that can be installed.

These libraries automate low-level details and provide a more intuitive interface for interacting with web services, thus speeding up the development of applications that rely on HTTP communications.

How is an HTTP request made?

Let’s rewrite our previous example in Ruby that used sockets to others in different languages.

Javascript

const url = 'https://httpbin.org/post';
const headers = {
  'Accept': 'application/json',
  'Content-Type': 'application/json',
};
const data = { 'hello': 'world' };

fetch(url, {
  method: 'POST',
  headers: headers,
  body: JSON.stringify(data)
})
  .then(response => response.json())
  .then(data => {
    console.log(data);
  });

Python

import requests

url = 'https://httpbin.org/post'
headers = {
  'Accept': 'application/json',
  'Content-Type': 'application/json',
}
data = {'hello': 'world'}

response = requests.post(url, json=data, headers=headers)

print(response.json())

PHP

$url = 'https://httpbin.org/post';
$headers = [
  'Accept: application/json',
  'Content-Type: application/json',
];
$data = [
  'hello' => 'world',
];

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$response = curl_exec($ch);

echo $response;

curl_close($ch);

Ruby

require 'net/http'
require 'json'

url = URI.parse('https://httpbin.org/post')
headers = {
  'Accept' => 'application/json',
  'Content-Type' => 'application/json',
}
data = { 'hello' => 'world' }

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = (url.scheme == 'https')

request = Net::HTTP::Post.new(url.path, headers)
request.body = data.to_json

response = http.request(request)

puts JSON.parse(response.body)

Summary

In this article, we have analyzed the process of creating HTTP requests, exploring each of its parts and explaining the fundamental structure of this protocol. We also delved into the concept of the HTTP message, breaking down its elements.

Additionally, we have explained how requests are sent internally through TCP sockets. This part of the process is crucial to understanding how information is transmitted and received between the client and the server in a network environment.

To provide a more comprehensive understanding, we have included several practical examples in different languages, allowing visualization of how they are implemented in practice.

In summary, HTTP requests are the backbone of communication on the modern web, and understanding their operation is essential for any developer. With this knowledge and the support of the right tools, we can build robust and efficient web applications that meet the needs of our users.