The Real Hidden Cost of a Request

Hussein Nasser delves into the intricate journey of a request from a front-end to a back-end server, emphasizing the critical steps involved and their performance implications. He outlines six key stages: connection acceptance, data reading, decryption, request parsing, decoding, and processing. Nasser highlights the hidden complexities and costs at each stage, such as the overhead of decrypting data and parsing protocols like HTTP. He stresses the importance of understanding these processes to appreciate the backend's workload and efficiency, setting the stage for deeper exploration in future discussions.

• A logical object called a request undergoes multiple steps before landing in a process or thread that handles it.
• Understanding these steps helps in appreciating the performance implications of backend operations.
• The term "handle" in backend context is complex and can have multiple meanings depending on the steps involved.

"I want to take you on a journey of what we call a request sent from a front end to a back end server."

• Initiates the discussion on the journey of a request from the front end to the backend server.

"There are so many steps that this logical object that we described that is called request goes through before we land in a process or a thread that actually handles the request, whatever that means."

• Highlights the complexity and multiple steps involved in processing a request.

"A common question that we get on the back end is like, how many requests can you handle per second? You know, that's a very difficult and frankly very overloaded question to answer, because what do you mean by handle?"

• Emphasizes the ambiguity and complexity in defining what it means to "handle" a request.

• The journey of a request involves six abstracted steps that cover its lifecycle from inception to processing.
• Each step has performance implications that affect how the backend handles requests.

"If we understand what happens to the request from the moment it is incepted, pretty sure that's not a word in created on the front end until it is arriving in the backend. There is these steps that is goes."

• Describes the lifecycle of a request from its creation on the front end to its arrival at the backend.

"Appreciating these steps makes you better aware of the performance implications of what you're doing."

• Understanding these steps is crucial for optimizing backend performance.

• The term "handle" often refers to the actual processing of the request but can encompass various steps in the request's journey.
• Different interpretations of "handle" can affect how performance and capacity are measured.

"We often use. We mean the actual processing of the request when we say that. But even then, we really don't know how much that factor in."

• Points out that "handling" usually implies processing but is an ambiguous term.

"Once I go through these steps, you understand, like, wait a minute. Yeah, what does it, what does handle really mean?"

• Suggests that understanding the steps will clarify the meaning of "handle."

• The host references a Medium article and a graphic to explain the steps visually and in detail.
• Graphics and articles serve as supplementary material to understand the journey of a request better.

"This is an article I wrote on medium. It's called the journey of a request, the backend. I'm not going to read it, but I like to use it as a reference."

• Indicates the use of external resources to provide a detailed explanation.

"I'm going to have it in the description below. I want to use it as a reference, especially for the graphic that I developed here."

• Mentions the inclusion of a graphic for visual representation of the steps.

• Each step in the request's journey has specific performance implications.
• Understanding these implications can help in optimizing backend systems and handling capacity.

"And appreciating these steps makes you better aware of the performance implications of what you're doing."

• Emphasizes the importance of understanding each step for performance optimization.

"It goes back to what we this word that I used, handle. You know, a common question that we get on the back end is like, how many requests can you handle per second?"

• Connects the understanding of steps to the broader question of handling capacity and performance.

• The initial step involves establishing a connection, which is the vehicle for sending requests.
• The connection is facilitated by a transport protocol, often referred to in the OSI model's layer four.
• Although theoretically, systems can be built directly to the metal without a transport protocol, in practice, a transport protocol is essential.

"The first step which is the acceptance not really of a request, but of the vehicle of the request which is the connection."

• The connection is not about the request itself but the medium through which the request travels.

"You cannot send really request Willy nilly to the backend."

• Requests need a structured medium (connection) to reach the backend.

"This pipe is called the connection. And the vehicle is what we call the transport protocol, you know."

• The connection is the medium, and the transport protocol is the vehicle for data transmission.

• Accepting a connection is a complex task handled by proxies and web servers.
• The process of accepting connections can vary in scale, from a single process to thousands.

"In order to create a connection your connection must be accepted. And boy that is not a trivial task to do."

• Accepting a connection is a fundamental but challenging task in networking.

"A lot of proxies and web servers you know, their bread and butter is the connection. Acceptance, how can I accept you?"

• Proxies and web servers specialize in accepting connections efficiently.

"You can have one process accepting or a 10,000 process accepting."

• The scale of connection acceptance can vary significantly.

• Once a connection is established, data transmission begins in the form of raw bytes.
• At the networking level, data is transmitted as bytes without any inherent structure like HTTP or JWT.

"Read now that I have a connection that has a bunch of raw bytes coming at me. Raw bytes because yeah connections, you know in networking we only send bytes."

• Data is transmitted as raw bytes over the connection.

"There is no HTTP in networking like at low level. You look at this pipe coming, the data coming from the neck, there is no regular HTTP. All this your JWT, they don't exist. There are just a bunch of bytes."

• At the low level, data lacks structured formats like HTTP or JWT.

• The transmitted data is often encrypted and needs to be decrypted, typically using TLS (Transport Layer Security).
• Decryption is an additional computational cost in the data processing pipeline.

"Then we need to most probably decrypt this data because it is most probably encrypted because you probably use TL's because if you're using HTTPs you have to use TL's and everything is encrypted."

• Encrypted data needs to be decrypted, often using TLS when HTTPS is employed.

"So now you read something, you decrypted it as another cost."

• Decryption adds to the computational cost of data processing.

• After decryption, the data can be formed into a coherent request as agreed upon by the backend and frontend systems.
• This step involves interpreting the raw bytes into a structured format that represents a request.

"Only then you can actually start forming coherent representation of what you call, you know, the request or the backend and, and the front end agree to be what a request really is."

• The final step is to interpret the decrypted data into a structured request format.

• Hidden expenses refer to the abstracted costs involved in network protocols that users are often unaware of.
• Understanding the intricacies of these protocols reveals the hidden complexities and costs.
• The speaker emphasizes the importance of awareness regarding these hidden processes and their implications.

"It's an abstracted away, you don't really need know about it. And that's the sad part. That is the scary part that we don't know about the most of the stuff that is going on here."

• The hidden nature of network protocol expenses means users are often unaware of the complexities and costs involved.
• Lack of awareness can lead to underappreciation of the processes and resources required.

"And, and the sad thing, again, it's all hidden. When I say hidden, it's really tucked away, you know, and we don't appreciate it anymore."

• The hidden costs are deeply embedded in the processes, leading to a lack of appreciation for the resources and efforts involved.

• Parsing involves reading and interpreting the raw data of a request.
• Different versions of HTTP (e.g., HTTP/1, HTTP/2) require different parsing techniques.
• The process includes decrypting the data and forming a coherent representation of the request.

"And then after we parse that is now I have plain text bytes that says, oh, there is a get and then slash about, then space, and then there is the protocol, and then get, and then, oh, okay, okay, there's, I started understanding, you read, read, read, and you start parsing."

• Parsing converts raw data into a readable format, allowing for better understanding and processing of the request.

"You really read and start understanding this protocol until you get a unit of what you call a logical request."

• The goal of parsing is to break down the data into logical units that can be processed and understood.

• After parsing, the next step is to create a request object in the programming language being used.
• This involves storing various details such as the address, family, client information, and IP address.
• There is a cost associated with creating and managing these objects, which is often hidden from the user.

"Let's create a request object that's a node, for example, or c, or c sharp. You're going to create a dot, a request object stored in the heap and store the address, family and the information, the client and the IP address."

• Creating request objects involves allocating memory and storing necessary details, which incurs a hidden cost.

"There's always a cost to all this stuff. And, and the sad thing, again, it's all hidden."

• The process of creating and managing request objects is resource-intensive, and these costs are not always apparent.

• HTTP/2 introduces additional complexity compared to HTTP/1, including binary data and streams.
• Parsing HTTP/2 requests is more complicated due to these additional protocol elements.
• Understanding and decoding these requests require more sophisticated techniques.

"This gets more complicated with HTTP two because there are the protocol itself, unlike htbo eleven, which is a text only, this is a binary stuff. There is a stream. There's so much stuff there at the protocol level."

• HTTP/2's binary nature and stream-based protocol add layers of complexity to the parsing process.

• Decoding involves interpreting the parsed data to understand the request fully.
• Requests may include headers and bodies, which need to be decoded to extract meaningful information.
• Decoding is a crucial step to transform the raw data into a usable format for further processing.

"Now that you received a request, the request might have headers, might have, well, that's not really always the case, but it can have a body if it's a post request."

• Decoding is essential for understanding the full content of a request, including headers and bodies.

"That's why we need a decoding step where, okay, let me actually understand what this thing is, you know, and let me build."

• The decoding step is necessary to fully interpret and utilize the data within the request.

• JSON objects are initially unintelligible and require parsing to be useful.
• Tools like Node.js, body-parser, and Express are used to handle JSON parsing.
• Parsing involves converting JSON from bytes to a usable object, which can be resource-intensive.

"That's why you install the Node.js, you install the body parser, or the Express, you do express that does JSON, you execute a function to actually execute that because there is converting the bytes, that is the body that is JSON to actual JSON object is not cheap."

• JSON parsing can be more resource-intensive in some languages compared to JavaScript.

"Well, it might be a little bit cheaper in JavaScript, but other languages go, boy, they suffer. No, I've seen some libraries, some C library really suffer with JSON, you know, serialization and deserializing."

• Serialization and deserialization are key processes in handling JSON data.

"That's what it is. Serialization, deserialization. Let me make sure, actually, I got my ducks in a row. Yep, that's it."

• Character encoding, such as UTF-8 and ASCII, plays a crucial role in JSON parsing.
• The byte representation of text can change based on the encoding used.

"This is also UTF-8, right? Remember, even if you're sending in text, even if you're sending text, I need to know if this text is ASCII or UTF-8 because the byte representation really changes."

• Understanding and specifying the correct encoding is part of the parsing process.

"That's part of the parsing. I mean, a lot of people might just say, okay, these two are really the same."

• After parsing, the next step is processing the request, which involves user code.
• Typical operations include establishing a database connection and sending SQL queries asynchronously.

"The final step is actually to process this thing. Actually, now that I have request. Now it's mostly user code. They say, all right, let's go ahead and take this sucker. Oh, you want to do, I don't know, get the list of user by id. Okay, I know what to do. Let's establish a connection to the, back into the database, to the Postgres database, send a SQL asynchronously."

• Asynchronous processing allows handling multiple tasks simultaneously, even with a single thread.

"Wait. And then me, while I'm waiting, while I'm awaiting. Right. I can do another of these cycles. I can, again accept, I can read, I can decrypt I can parse. That's why the backend, even with a single thread, if it's doing mostly asynchronous work, oh boy, can do so much, so much, so much work."

• For CPU-intensive tasks, it may be necessary to split the work into separate threads or processes.
• This approach helps to alleviate the main thread, allowing it to handle other tasks efficiently.

"Do you have, if it's a CPU heavy, do you split this into its own thread or its own process to execute that while your main thread becomes, you know, alleviated to do other stuff?"

• The video concludes with a promise to delve deeper into each step in future videos.
• Each step in the parsing and processing workflow deserves detailed exploration.

"So I'm going to stop here making this video a little bit short and maybe in future videos I'll go deep into each one of them. I think because you need to, we need to pay respect to each of these steps. You know, each of these steps really deserve really more details and boy I can go into a lot of details sometimes, you know, and you guys actually let me know about this."

"All right, see you in the next one. Goodbye."