Application Layer Protocols

The application layer is where networking meets the programs people use. HTTP is the most famous application protocol, but dozens of others run the Internet quietly: SMTP moves email between servers, DHCP hands your laptop an IP address the moment it joins Wi-Fi, NTP keeps clocks in sync so that certificates and logs make sense, and SSH lets engineers manage servers securely.

Interviews commonly probe a handful of these: "how does email get from sender to receiver", "SMTP versus IMAP versus POP3", "explain DHCP's four steps", "why is FTP considered insecure", "what are SPF, DKIM and DMARC", "which port does X use", and, in backend and systems roles, "walk me through the socket API calls for a TCP server". This lesson explains each protocol step by step, gives a port reference table, and ends with a tested Python program that sends an HTTP request over a raw TCP socket so you can see that application protocols are just agreed-upon bytes.

What an application protocol defines

Every application protocol answers the same few questions:

1. Transport: TCP (reliable byte stream) or UDP (independent datagrams)? See the transport layer lesson.
2. Port: which well-known port the server listens on, so clients know where to connect.
3. Message format: text lines (SMTP, HTTP/1.1, FTP commands) or binary structures (DNS, DHCP, NTP).
4. Message sequence: who speaks first, which commands are valid in which state, how errors are reported.
5. Security: plain text, upgraded to TLS with a command (STARTTLS), or encrypted from the first byte.

Many older Internet protocols are text-based command/response protocols: the client sends a command line ending in CRLF, and the server replies with a numeric status code and a message. Once you know that pattern from SMTP, FTP and HTTP, you recognise it everywhere.

Email

Email is the best example of several protocols cooperating. Different protocols handle sending and reading.

The players

• MUA (mail user agent): the email client, such as Gmail's web UI, Outlook or Thunderbird.
• MSA (mail submission agent): the server that accepts mail from authenticated users, usually on port 587.
• MTA (mail transfer agent): a server that relays mail to other servers using SMTP on port 25 (Postfix, Exim, Microsoft Exchange, Gmail's servers).
• MDA (mail delivery agent): stores incoming mail in the recipient's mailbox.

The path of one email

Asha (asha@alpha.example) emails Ravi (ravi@beta.example):

 Asha's client                                       Ravi's client
     |  (1) SMTP submission, port 587, STARTTLS + login     ^
     v                                                      | (5) IMAP 993
 alpha.example                                          beta.example
 submission server                                      mailbox store
     |                                                      ^
     |  (2) DNS: MX record for beta.example?                | (4) delivered
     |      -> 10 mx.beta.example                           |     to mailbox
     |  (3) SMTP relay, port 25 (opportunistic TLS)         |
     +-------------------> mx.beta.example (MTA) -----------+

1. Asha's client submits the message to her provider's server with SMTP on port 587, after upgrading to TLS and logging in.
2. Her provider's MTA looks up the MX record for beta.example in DNS (see the DNS lesson) and finds mx.beta.example.
3. It opens an SMTP connection to mx.beta.example on port 25 and transfers the message. If that server is down, the sender queues the message and retries for days, which is why email tolerates outages.
4. Ravi's server checks the message (spam filters, SPF, DKIM, DMARC) and stores it in his mailbox.
5. Ravi's client retrieves it with IMAP (or POP3).

So: SMTP pushes mail toward the recipient; IMAP and POP3 pull mail from the mailbox to the reader.

SMTP

SMTP (Simple Mail Transfer Protocol) is a text protocol over TCP. Here is a real-shape session (C = client, S = server):

S: 220 mx.beta.example ESMTP ready
C: EHLO mail.alpha.example
S: 250-mx.beta.example
S: 250-SIZE 52428800
S: 250-STARTTLS
S: 250 8BITMIME
C: STARTTLS
S: 220 Ready to start TLS
   ... TLS handshake; client sends EHLO again ...
C: MAIL FROM:<asha@alpha.example>
S: 250 OK
C: RCPT TO:<ravi@beta.example>
S: 250 OK
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: Asha <asha@alpha.example>
C: To: Ravi <ravi@beta.example>
C: Subject: Lunch?
C:
C: Are you free at 1?
C: .
S: 250 OK queued as 7F3A2
C: QUIT
S: 221 Bye

Things to notice:

• Status codes like HTTP's: 2xx success, 3xx "send more", 4xx temporary failure (retry later), 5xx permanent failure (bounce).
• The envelope versus the headers. MAIL FROM and RCPT TO form the envelope, used for delivery, like the address on a postal envelope. The From: and To: lines inside DATA are headers shown to the reader, like the letter inside. They can differ (that is how BCC works), and that gap is what spoofers abuse.
• A line containing only . ends the message.
• STARTTLS upgrades the plain connection to TLS. Between servers on port 25 this is usually opportunistic: if either side does not support it, mail is sent unencrypted. Standards such as MTA-STS let a domain demand TLS.

Ports: 25 for server-to-server relay; 587 for client submission with STARTTLS and authentication; 465 for submission with TLS from the first byte (implicit TLS). Many ISPs and cloud providers block outbound port 25 from ordinary machines to limit spam.

MIME

SMTP was designed for 7-bit ASCII text. MIME (Multipurpose Internet Mail Extensions) lets an email carry other character sets, HTML and attachments by adding headers and structure:

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="XYZ"

--XYZ
Content-Type: text/plain; charset=utf-8

Report attached.
--XYZ
Content-Type: application/pdf; name="report.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="report.pdf"

JVBERi0xLjcKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIv...
--XYZ--

• multipart/mixed means "several parts follow", separated by the boundary string.
• Binary attachments are Base64-encoded into safe ASCII text, which grows them by about a third (every 3 bytes become 4 characters).
• The Content-Type labels (text/html, image/png, application/json) were invented for email and later adopted by HTTP. That is why HTTP's header is called "MIME type".

IMAP versus POP3

POP3 (Post Office Protocol 3) is simple: log in, list messages, download them, optionally delete them. It made sense when people used one computer and servers had little storage.

IMAP (Internet Message Access Protocol) treats the server as the master copy. Reading a message on your phone marks it read on your laptop. Modern providers favour IMAP or their own APIs (such as Gmail's API or Microsoft's Exchange protocols).

SPF, DKIM and DMARC

Plain SMTP lets anyone write any From: address, which makes phishing easy. Three DNS-based standards help receivers detect forged mail.

SPF (Sender Policy Framework) publishes which servers may send mail for a domain, as a TXT record:

alpha.example.  TXT  "v=spf1 ip4:198.51.100.0/24 include:_spf.mailprovider.example -all"

The receiver takes the envelope sender's domain (MAIL FROM) and checks whether the connecting server's IP is listed. -all means "reject anything else" (hard fail); ~all means "soft fail, treat with suspicion". Weakness: SPF checks the envelope, not the visible From: header, and it breaks when mail is forwarded through another server.

DKIM (DomainKeys Identified Mail) adds a digital signature to each message. The sending server signs selected headers and the body with a private key and adds a DKIM-Signature header. It names a selector (s=mail2026) and domain (d=alpha.example), and the receiver fetches the public key from DNS at mail2026._domainkey.alpha.example. A valid signature proves the message was sent by a server holding the domain's key and was not modified in transit. DKIM survives forwarding, because the signature travels with the message. (Signatures are explained in the TLS and network security lesson.)

DMARC (Domain-based Message Authentication, Reporting and Conformance) ties the two to the visible From: domain and tells receivers what to do on failure:

_dmarc.alpha.example.  TXT  "v=DMARC1; p=reject; rua=mailto:dmarc@alpha.example"

• A message passes DMARC if SPF or DKIM passes and the domain that passed aligns with (matches) the From: header's domain.
• p= is the policy for failures: none (just monitor), quarantine (send to spam), reject (refuse).
• rua= asks receivers to send aggregate reports, so the domain owner can see who sends mail claiming to be them.

Major mailbox providers (Gmail and Yahoo since 2024) require bulk senders to have all three configured, so this is practical knowledge for any backend engineer sending transactional email.

File transfer: FTP, SFTP and SCP

FTP

FTP (File Transfer Protocol) is one of the oldest Internet protocols (1971, current form RFC 959 in 1985). Its distinctive feature is two separate TCP connections:

• A control connection to server port 21, carrying text commands (USER, PASS, LIST, RETR file, STOR file) and numeric replies.
• A data connection for each file transfer or directory listing.

How the data connection is set up depends on the mode:

Active mode:                          Passive mode:
client --cmd--> server:21             client --cmd--> server:21
client: "PORT 192.168.1.23,195,80"    client: "PASV"
server:20 ---data---> client:50000    server: "227 (...,195,81)" = port 50001
  (server connects BACK to client)    client --data--> server:50001
                                        (client connects out)

In the PORT command, the port is given as two numbers: 195,80 means 195 × 256 + 80 = 50000.

• In active mode, the server connects back to the client, which fails behind NAT and client firewalls.
• In passive mode, the client opens both connections outward, which works through NAT; it is the default today.

Why FTP is insecure: usernames, passwords and file contents travel in plain text. Anyone on the path can capture the password. FTPS adds TLS to FTP, but keeps its awkward two-connection design, which complicates firewalls.

SFTP and SCP

SFTP (SSH File Transfer Protocol) is not FTP over SSH; it is a completely different protocol that runs as a subsystem inside an SSH connection on port 22. Everything (authentication, commands, data) is encrypted, uses one connection, and supports listing, resuming, renaming, permissions and deleting.

SCP (Secure Copy) also runs over SSH and simply copies files, like cp across machines:

scp report.pdf user@server:/srv/reports/          # copy to server
sftp user@server                                  # interactive session
rsync -avz ./site/ user@server:/var/www/site/     # sync only changed parts over SSH

The legacy SCP protocol had design weaknesses (the server could influence which files the client wrote). Since OpenSSH 9.0 (2022), the scp command uses the SFTP protocol underneath by default.

SSH and Telnet

Telnet (TCP port 23) gives a remote text terminal. It sends everything, including your password, in plain text, so it is unsafe on any untrusted network. It survives on old network gear, and engineers sometimes use a Telnet client as a quick "can I open a TCP connection to this port?" test, though nc (netcat) is better for that.

SSH (Secure Shell) (TCP port 22) replaced Telnet, rlogin and rsh. Its three layers:

1. Transport layer: key exchange (usually ECDH on curve25519), server authentication with the host key, and symmetric encryption with integrity protection.
2. User authentication: password, public key, or certificates; public key is preferred.
3. Connection layer: multiplexes many channels over one encrypted connection: interactive shells, single commands, SFTP, and port forwarding.

Port forwarding (tunnelling) is a frequently asked follow-up:

# Local forwarding: reach a database that only the bastion can see.
# localhost:5433 on your laptop -> (SSH) -> bastion -> db.internal:5432
ssh -L 5433:db.internal:5432 user@bastion.example.com

# Remote forwarding: expose your laptop's port 3000 on the server's port 8080
ssh -R 8080:localhost:3000 user@server.example.com

# Dynamic forwarding: a SOCKS proxy on localhost:1080 through the server
ssh -D 1080 user@server.example.com

A bastion host (or jump host) is a hardened server that is the only SSH entry point into a private network; ssh -J user@bastion user@internal-host jumps through it. Key-based authentication is explained in the TLS and network security lesson.

DHCP in detail

When your laptop joins a network it has no IP address, no idea of the subnet, the gateway or the DNS servers. DHCP (Dynamic Host Configuration Protocol) provides all of these automatically. It runs over UDP: servers listen on port 67, clients on port 68.

The DORA exchange

DHCP's four-message exchange is remembered as DORA: Discover, Offer, Request, Acknowledge.

Client (no IP yet)                            DHCP server 192.168.1.1
  |                                                |
  |-- DISCOVER  src 0.0.0.0:68                     |
  |             dst 255.255.255.255:67 (broadcast) |
  |             "I need an address; my MAC is      |
  |              a4:83:e7:11:22:33"                |
  |                                                |
  |<-- OFFER   "You can have 192.168.1.23,         |
  |             mask /24, gateway .1, DNS .1,      |
  |             lease 86400 s"                     |
  |                                                |
  |-- REQUEST  (still broadcast)                   |
  |            "I accept 192.168.1.23 from         |
  |             server 192.168.1.1"                |
  |                                                |
  |<-- ACK     "Confirmed. Lease 86400 s."         |
  |                                                |
  client may send ARP for 192.168.1.23 to check no one else uses it

Step by step:

1. Discover. The client has no address, so it sends from 0.0.0.0 to the broadcast address 255.255.255.255. Every device on the LAN receives it, including any DHCP server. It includes the client's MAC address and a transaction id.
2. Offer. Each DHCP server that can help reserves an address from its pool and offers it, along with options: subnet mask, default gateway (router), DNS servers, domain name, and the lease time.
3. Request. The client picks one offer (usually the first) and broadcasts a Request naming the chosen server. Broadcasting it tells any other servers that their offers were declined, so they can release their reserved addresses.
4. Acknowledge. The chosen server confirms with an ACK. The client configures its interface. Many clients then send an ARP probe for the new address to detect a conflict; if one is found, the client sends DHCPDECLINE and starts over.

If the server cannot honour the request (for example, the client moved to another network), it replies NAK, and the client restarts from Discover.

Leases and renewal

Addresses are leased, not given forever, so addresses of departed devices return to the pool. With a lease of 86400 seconds (24 hours):

• At 50 percent of the lease (T1, 12 hours), the client unicasts a Request directly to its server to renew. Usually the server ACKs and the lease restarts.
• If there is no answer, at 87.5 percent (T2, 21 hours), the client rebinds: it broadcasts a Request to any server.
• If the lease expires without renewal, the client must stop using the address and start DORA again.
• A client leaving cleanly may send DHCPRELEASE.

DHCP relay

Broadcasts do not cross routers. A company with dozens of subnets does not want a DHCP server in each one. A DHCP relay agent (a feature of the router, configured as an "IP helper") catches the broadcast on the local subnet and forwards it as unicast to a central DHCP server. It adds the subnet's address (the giaddr field) so the server picks an address from the correct pool.

Reservations, failures and attacks

• Reservations (static leases) always give the same IP to a given MAC, useful for printers and servers.
• If no server answers, many operating systems assign themselves a link-local address from 169.254.0.0/16 (APIPA on Windows). Seeing a 169.254.x.x address is a classic sign that DHCP failed.
• Rogue DHCP server: an attacker's device answers first and hands out itself as gateway or DNS server, enabling a man-in-the-middle attack. DHCP snooping on switches allows server messages only from trusted ports.
• DHCP starvation: an attacker requests addresses with thousands of fake MACs until the pool is empty.

IPv6 can use DHCPv6 or SLAAC (stateless address autoconfiguration), where hosts build their own address from the router's advertised prefix.

NTP: keeping clocks in sync

Computer clocks drift, often by seconds per day. Wrong clocks break things that matter: TLS certificate validity checks, Kerberos authentication (typically fails if clocks differ by more than five minutes), log correlation across servers, scheduled jobs and distributed databases that order events by time.

NTP (Network Time Protocol) synchronises clocks over UDP port 123, typically to within milliseconds over the Internet and much better on a LAN.

Stratum hierarchy

Stratum 0:  atomic clocks, GPS receivers (reference clocks, not on the network)
               |
Stratum 1:  servers directly attached to stratum-0 sources
               |
Stratum 2:  servers syncing from stratum 1
               |
Stratum 3:  servers syncing from stratum 2 ... (up to 15)

Lower stratum means closer to the reference, not necessarily more accurate. Clients usually query several servers (for example from pool.ntp.org or a cloud provider's time service) and discard outliers.

How NTP calculates offset: worked example

NTP records four timestamps:

• t1: client sends request (client clock)
• t2: server receives it (server clock)
• t3: server sends reply (server clock)
• t4: client receives reply (client clock)

Then:

round-trip delay  = (t4 - t1) - (t3 - t2)
clock offset      = ((t2 - t1) + (t3 - t4)) / 2

Example in milliseconds: t1 = 100, t2 = 150, t3 = 151, t4 = 121.

• Delay = (121 − 100) − (151 − 150) = 21 − 1 = 20 ms (time on the network, excluding the server's 1 ms of processing).
• Offset = ((150 − 100) + (151 − 121)) / 2 = (50 + 30) / 2 = 40 ms. The server's clock is 40 ms ahead of the client's.

Check: if the client's clock is 40 ms behind and each direction takes 10 ms, the request sent at client time 100 arrives at client time 110, which is server time 150. Correct. The reply sent at server time 151 (client time 111) arrives at client time 121. Correct.

The formula assumes the delay is symmetric (the same in both directions). Asymmetric paths introduce error of up to half the asymmetry.

The client does not jump its clock for small errors. It slews (slightly speeds up or slows down) the clock, because time jumping backwards can break software. Large errors at boot may be stepped once. Linux uses chronyd or systemd-timesyncd; Windows uses the Windows Time service. PTP (Precision Time Protocol) achieves sub-microsecond accuracy with hardware timestamping, used in finance and telecoms.

SNMP, briefly

SNMP (Simple Network Management Protocol) monitors and manages network devices: routers, switches, printers, UPS units.

• Manager: the monitoring system. Agent: software on each device.
• The manager polls agents with GET requests on UDP 161; agents send unsolicited alerts called traps to the manager on UDP 162.
• Values are organised in a MIB (Management Information Base) and identified by OIDs (object identifiers), dotted numbers such as 1.3.6.1.2.1.1.3.0 for system uptime.
• SNMPv1 and v2c authenticate with a "community string" sent in plain text (the default public is famously left unchanged on many devices). SNMPv3 adds proper authentication and encryption.

Modern infrastructure increasingly uses streaming telemetry and HTTP-based metrics (such as Prometheus) instead, but SNMP remains everywhere in network operations.

Peer-to-peer and BitTorrent

Every protocol so far is client-server: a server holds the resource, clients ask for it. In peer-to-peer (P2P) systems, every participant (a peer) both downloads and uploads. Capacity grows with the number of users, instead of the server becoming the bottleneck.

BitTorrent is the best-known P2P file-sharing protocol:

1. The file is split into fixed-size pieces (for example 256 KB to a few MB), each with a SHA-1 (or SHA-256 in BitTorrent v2) hash listed in a .torrent file or referenced by a magnet link.
2. A peer finds others sharing the same file through a tracker (a server that lists peers) or a DHT (distributed hash table), a decentralised lookup spread across peers.
3. The peer downloads different pieces from different peers in parallel and verifies each piece's hash, so a malicious peer cannot inject bad data.
4. It requests the rarest pieces first, keeping all pieces available in the swarm.
5. Tit-for-tat choking: a peer uploads preferentially to peers that upload to it, rewarding sharing. It also periodically "optimistically unchokes" a random peer to discover better partners and help newcomers.
6. A peer with the complete file is a seeder; one still downloading is a leecher.

The same ideas (content split into hashed chunks, swarm distribution) appear in software update distribution, some video streaming systems and blockchain networks.

Well-known ports

A port is a 16-bit number (0 to 65535) identifying a service on a host. Ranges:

• 0 to 1023: well-known ports, assigned by IANA to standard services. On Unix systems, binding to them traditionally requires root privileges.
• 1024 to 49151: registered ports, assigned to specific applications (databases, for example).
• 49152 to 65535: dynamic or ephemeral ports, picked by the OS for the client side of connections. (Linux by default uses 32768 to 60999.)

The table to memorise:

Ports are conventions, not laws. Nothing stops you running SSH on 2222 or HTTP on 8000; clients just need to know.

The socket API

Application protocols are implemented on top of the socket API, the interface operating systems expose for network communication (Berkeley sockets, originating in BSD Unix and copied by almost every language). A socket is an endpoint for communication: the program reads and writes it like a file, and the OS handles TCP or UDP underneath.

A TCP connection is uniquely identified by its 4-tuple: source IP, source port, destination IP, destination port (plus the protocol). That is how a web server on port 443 can hold thousands of connections at once: each has a different client IP and port.

Lifecycle of a TCP server and client

        SERVER                               CLIENT
   socket()   create endpoint           socket()
      |                                    |
   bind()     attach to IP:port            |  (OS picks an ephemeral port)
      |                                    |
   listen()   mark as passive,             |
      |       set backlog queue            |
      |                                    |
   accept()   block until a client <---- connect()   TCP 3-way handshake
      |       connects; returns a NEW      |
      |       socket for that client       |
      |                                    |
   recv() / send()  <================>  send() / recv()
      |                                    |
   close()    (FIN exchange)            close()

What each call does:

• socket(family, type): create an endpoint. AF_INET = IPv4 (AF_INET6 for IPv6); SOCK_STREAM = TCP, SOCK_DGRAM = UDP. Returns a file descriptor.
• bind(addr, port): attach the socket to a local address and port. Servers bind to a well-known port; clients usually skip this, and the OS picks an ephemeral port during connect.
• listen(backlog): turn the socket into a listening (passive) socket. The kernel now completes TCP handshakes on its own and queues established connections; backlog caps that queue.
• accept(): take the next completed connection from the queue, blocking if empty. It returns a new socket dedicated to that client. The listening socket keeps listening for others.
• connect(addr, port): client side; triggers the three-way handshake and returns when the connection is established (or fails).
• send() / recv(): write and read bytes. TCP is a byte stream: one send of 100 bytes may arrive as two recv calls of 60 and 40 bytes, and two sends may arrive in one recv. Protocols must define their own message boundaries (HTTP uses headers and Content-Length; others use length prefixes or delimiters).
• close(): release the socket; for TCP, starts the FIN exchange. The side that closes first enters TIME_WAIT for a while.

For UDP there is no listen, accept or connect requirement: a server does socket, bind, then recvfrom/sendto, each call carrying one datagram and the peer's address.

A tested echo server and client in Python

Server (run first, in one terminal):

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server:
    server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    server.bind(("127.0.0.1", 9000))
    server.listen()
    print("listening on 127.0.0.1:9000")

    conn, addr = server.accept()      # blocks until a client connects
    with conn:
        print("client connected from", addr)
        data = conn.recv(1024)
        conn.sendall(b"echo: " + data)

Client (in a second terminal):

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as client:
    client.connect(("127.0.0.1", 9000))
    client.sendall(b"hello")
    print(client.recv(1024).decode())

Output:

# server terminal
listening on 127.0.0.1:9000
client connected from ('127.0.0.1', 65205)

# client terminal
echo: hello

The client's port (65205 here) is ephemeral and will differ on each run. SO_REUSEADDR lets you restart the server immediately instead of waiting for the old socket's TIME_WAIT to clear. sendall keeps calling send until every byte is written, since a single send may write only part of the buffer. This server handles one client and exits; real servers loop on accept and handle each connection in a thread, process or event loop (select/epoll, Python's asyncio).

HTTP over a raw socket

HTTP is just text on a TCP connection, so you can speak it yourself. This program connects to example.com on port 80, writes an HTTP/1.1 request by hand, and reads the response until the server closes the connection:

import socket

HOST = "example.com"
PORT = 80

request = (
    "GET / HTTP/1.1\r\n"
    f"Host: {HOST}\r\n"
    "User-Agent: raw-socket-demo/1.0\r\n"
    "Connection: close\r\n"
    "\r\n"
)

with socket.create_connection((HOST, PORT), timeout=10) as sock:
    print("connected:", sock.getsockname(), "->", sock.getpeername())
    sock.sendall(request.encode("ascii"))

    chunks = []
    while True:
        data = sock.recv(4096)
        if not data:          # empty bytes = server closed the connection
            break
        chunks.append(data)

response = b"".join(chunks)
head, _, body = response.partition(b"\r\n\r\n")
print(head.decode("iso-8859-1"))
print("body bytes:", len(body))

Sample output (trimmed; addresses and headers depend on your network and the site's hosting provider):

connected: ('192.168.1.3', 65186) -> ('172.66.147.243', 80)
HTTP/1.1 200 OK
Date: Fri, 09 Oct 2026 19:11:22 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Server: cloudflare
Last-Modified: Sun, 04 Oct 2026 20:44:03 GMT
Age: 10740
body bytes: 589

What the program shows, step by step:

1. create_connection is a convenience wrapper: it resolves example.com through DNS (getaddrinfo), creates a socket, and calls connect, which performs the TCP handshake.
2. getsockname shows the local address and the ephemeral port the OS chose; getpeername shows the server's IP and port 80.
3. The request is exactly the format from the HTTP lesson: request line, Host header, a blank line, each line ending in \r\n. Forget the final blank line and the server waits forever for the rest of the headers.
4. Connection: close asks the server to close after responding, so "read until recv returns empty bytes" is a valid way to find the end. On a keep-alive connection you would instead have to parse Content-Length or chunked encoding.
5. The response is split at the first blank line into headers and body. Transfer-Encoding: chunked means the body still contains chunk-size lines; a real client would decode them.
6. Port 80 is plain HTTP, so anyone on the path could read this exchange. For HTTPS you would wrap the socket with ssl.create_default_context().wrap_socket(sock, server_hostname=HOST) and connect to port 443; the HTTP text stays identical, it just travels inside TLS.

Interview questions

Q1. How does an email travel from sender to receiver?

The sender's client submits the message to its provider over SMTP on port 587 with TLS and authentication. The provider looks up the recipient domain's MX record and relays the message with SMTP to that mail server on port 25, retrying later if it is unreachable. The receiving server checks SPF, DKIM and DMARC and stores the message, and the recipient reads it using IMAP or POP3.

Q2. What is the difference between SMTP, POP3 and IMAP?

SMTP sends and relays mail between clients and servers. POP3 and IMAP retrieve mail from a mailbox. POP3 downloads messages and traditionally deletes them from the server, while IMAP keeps mail on the server and synchronises folders and read status across devices.

Q3. What are SPF, DKIM and DMARC?

SPF is a DNS record listing the servers allowed to send for a domain, checked against the envelope sender. DKIM signs messages with a private key whose public key is published in DNS, proving integrity and the signing domain. DMARC requires SPF or DKIM to pass with a domain aligned to the visible From address and sets a policy (none, quarantine, reject) plus reporting.

Q4. Why is FTP insecure, and what would you use instead?

FTP sends usernames, passwords and data in plain text, and its separate data connections are awkward for firewalls and NAT. Use SFTP or SCP, which run over SSH on port 22, or FTPS if FTP compatibility is required.

Q5. What is the difference between active and passive FTP?

In active mode, the client tells the server a port and the server connects back to the client for data, which fails behind NAT or firewalls. In passive mode, the server tells the client a port and the client opens the data connection, which works through NAT and is the common default.

Q6. Explain the DHCP process.

DHCP uses DORA over UDP 67/68. The client broadcasts Discover, servers reply with Offers of an address and options, the client broadcasts a Request for one offer, and the chosen server confirms with an Acknowledge. The address is leased; the client renews at half the lease time and rebinds at 87.5 percent.

Q7. Why does DHCP use UDP and broadcasts?

The client has no IP address yet and does not know the server's address, so it cannot open a TCP connection; it broadcasts UDP datagrams from 0.0.0.0. Broadcasting the Request also informs other servers that their offers were declined. Relay agents forward these broadcasts across subnets.

Q8. What does a 169.254.x.x address indicate?

A link-local, self-assigned address, which the OS uses when it cannot reach a DHCP server. It usually means DHCP failed: a cable, Wi-Fi association, VLAN or DHCP server problem.

Q9. Why is time synchronisation important, and how does NTP compute the offset?

Certificates, Kerberos, logs, schedulers and distributed systems all depend on correct time. NTP records the send and receive times at both ends and computes offset as ((t2 − t1) + (t3 − t4)) / 2 and delay as (t4 − t1) − (t3 − t2), assuming symmetric network delay, and slews the clock gradually.

Q10. What is the difference between SSH and Telnet?

Both provide remote terminals, but Telnet sends everything including passwords in plain text, while SSH encrypts the session, authenticates the server with a host key, supports public-key user authentication, and can multiplex shells, file transfer and port forwarding over one connection.

Q11. What is SSH local port forwarding used for?

It forwards a port on your machine through the SSH connection to a host reachable from the SSH server, for example ssh -L 5433:db.internal:5432 bastion lets you reach a private database on localhost:5433. It is commonly used to access internal services securely via a bastion host.

Q12. Explain the socket calls for a TCP server.

The server calls socket to create an endpoint, bind to attach it to an IP and port, listen to accept connections into a backlog queue, and accept to take one connection, which returns a new socket for that client. It then uses recv and send on that socket and close when done, while the listening socket continues accepting.

Q13. How can one server port handle thousands of clients?

Each TCP connection is identified by the 4-tuple of source IP, source port, destination IP and destination port. All connections share the destination port but differ in client IP or port, so the kernel can route each segment to the right connected socket.

Q14. Why must protocols on TCP define message boundaries?

TCP is a byte stream: it does not preserve the boundaries of individual send calls, so data may be split or merged across recv calls. Protocols mark boundaries with lengths (Content-Length, length prefixes), delimiters (CRLF, the SMTP dot line) or framing (HTTP/2 frames).

Q15. How does BitTorrent work?

Files are split into hashed pieces. Peers find each other through a tracker or a DHT, download different pieces from many peers in parallel, verify each piece by hash, fetch the rarest pieces first, and upload to peers who upload to them (tit-for-tat), so capacity grows with the number of users.

Key takeaways

• Application protocols define transport, port, message format, message sequence and security.
• Email: SMTP pushes (587 submission, 25 relay); IMAP or POP3 pull. MIME adds attachments and character sets; SPF, DKIM and DMARC fight spoofing.
• FTP is plain text with separate control and data connections; prefer SFTP or SCP over SSH.
• SSH encrypts remote access, authenticates servers by host key and users preferably by key, and supports tunnels.
• DHCP is DORA over UDP 67/68 with leases renewed at 50 percent; relays cross subnets; 169.254.x.x means DHCP failed.
• NTP uses four timestamps to compute offset and delay over UDP 123 and slews clocks.
• Know the well-known ports table, including DNS on TCP too and HTTP/3 on UDP 443.
• Socket lifecycle: socket, bind, listen, accept on the server; socket, connect on the client; TCP is a byte stream.

Next lesson

Continue with Modern networking and tools.