ConceptDBISAI-written · not yet proofread

HTTP Server

Also Web Server · Search Server

An HTTP server is a program that accepts connections on a server socket and, for each one, reads the request bytes until the header ends, parses the request line, computes the requested resource (a file, a filled-in template or a JSON result) and sends back a status line, headers and the content as bytes.

Know these firstNetwork Socket HTTP Protocol

Formula#

Skeleton of the request handler (structure of the course’s live-coded search_server.py, simplified):

def handle(connection):
    request = bytearray()
    while b"\r\n\r\n" not in request:               # read in rounds
        chunk = connection.recv(4096)
        if not chunk: return                          # client aborted
        request.extend(chunk)
    request = request.decode("utf-8")
    path = request[request.find("GET /") + 5 : request.find(" HTTP/1.1")]
    if "?" in path:                                    # e.g. search.html?query=bla
        path, params = path.split("?", 1)
        query = url_decode(params.split("=", 1)[1])    # "+" -> " ", "%C3%A4" -> "ä"
    if path not in WHITELIST: status = "403 Forbidden"
    elif not os.path.exists(path): status = "404 Not Found"
    else: status = "200 OK"; content = ...             # file / filled template / JSON
    content = content.encode("utf-8")
    header = (f"HTTP/1.1 {status}\r\n"
              f"Content-Length: {len(content)}\r\n"    # length of the BYTES
              f"Content-Type: {media_type}; charset=utf-8\r\n"
              f"Connection: close\r\n\r\n")
    connection.sendall(header.encode("utf-8") + content)
    connection.close()

Properties#

Response headers. At least the status line, Content-Length (computed on the encoded bytes) and Content-Type with the media type matching the file ending: text/html, text/css, text/plain, application/javascript, application/json, image/x-icon for favicon.ico.
Security: serve only what is meant to be served. A server that serves any existing path leaks private files, e.g. http://tura:8888//etc/passwd. Restrict access to a directory subtree, or safer to a whitelist of files, and answer 404 Not Found or 403 Forbidden for everything else (Sheet 9: 403 for files outside the server directory).
Static vs. dynamic. For the static app (Sheet 9) the server fills %QUERY%/%RESULT% into search.html and returns a whole page. For the dynamic app (Sheet 10) it additionally answers API paths (/api/search?query=…, /api/relations?id=…) with JSON, which the page fetches with fetch.
CORS. If pages from other origins may use the API, the API response carries Access-Control-Allow-Origin: * (Same-Origin Policy > CORS).
Robustness. The client may connect and abort, connect and send nothing (use a timeout), or send the request in several, possibly empty, packets.

Concurrency#

Browsers keep connections open (Connection: keep-alive). A server that serves one request at a time gets stuck if the browser sends a new request on the same connection while the server is still busy. Fix: send Connection: close; the browser then opens a new connection for the next request.
“Search as you type” generates concurrent requests (a new keystroke before the previous answer). A typical server handles each request in its own thread, so an expensive request does not block the others.
Python’s global interpreter lock (GIL) lets only one thread interpret Python code at a time; the multiprocessing module forks the whole process including its data (here the Q-Gram Index). There is now a GIL-free python3.14-nogil.

Example#

ExampleOne request, end to end

The browser sends GET /search.html?query=b%C3%A4r HTTP/1.1 plus headers and an empty line. The server reads until \r\n\r\n, extracts the path search.html?query=b%C3%A4r, URL-decodes the query to bär, computes the fuzzy prefix matches, substitutes %QUERY% by bär and %RESULT% by the matches in search.html, encodes the page as UTF-8 (say 1532 bytes) and sends

HTTP/1.1 200 OK
Content-Length: 1532
Content-Type: text/html; charset=utf-8
Connection: close

<html>...

A request for /../server.py gets 403 Forbidden, one for /nothere.html gets 404 Not Found.

Sources#

Slides 09, p. 8 (server code, read in rounds), pp. 11–13 (response format, 403/404, media types), p. 19 (template variables), p. 21 (reusing connections)
Slides 10, p. 16 (multi-threaded server, GIL), p. 18 (access to private data)
DBIS_VL09 > Server implementation in Python, DBIS_VL09 > Reusing connections, DBIS_VL10 > Multi-threaded server, DBIS_VL10 > Access to private data
Sheet 09, Exercise 1, Sheet 10, Exercise 1.1–1.2
DBIS Exam WS1819 (not published), Task 4.2 (search server)

Connections

Mentioned in inline links10

AJAX and Fetch API
Typing fast produces concurrent requests, so answers may arrive out of order and the server should handle requests in parallel (HTTP Server). §
Character Encoding
Length in bytes ≠ length in characters. “knödel” has 6 characters but 7 UTF-8 bytes. Anything that counts bytes, e.g. the Content-Length header of an HTTP response, must be computed on the encoded bytes (HTTP Server). §
Code Injection Protection
It is one of three kinds of web-app vulnerabilities, next to access to private data (HTTP Server) and leaking trusted information to untrusted sites (Same-Origin Policy); overview: the OWASP Top Ten. §
HTML Form
A SERP (search engine result page) should be the original page, with the input field showing the query and augmented by the results. The HTML file contains template variables that the server substitutes before sending the page (HTTP Server… §
HTTP Protocol
Connection reuse: browsers send Connection: keep-alive to reuse the connection; a server answering Connection: close forces a new connection per request (HTTP Server). §
Network Socket
Read in rounds. recv(num_bytes) returns at most num_bytes bytes, possibly fewer, possibly an empty chunk. There is no universal “end of message” token, so the receiver loops until the application protocol says the message is complete. For… §
The server socket only listens. All reading and writing happens on the per-client socket returned by accept(). A server that handles one connection at a time blocks every other client while it computes; see HTTP Server. §
Same-Origin Policy
The SOP addresses the third kind of web-app vulnerability, “communication of trusted information to an untrusted site”; the other two are access to private data (HTTP Server) and executing injected code (Code Injection Protection). §
VL09 – Web Applications, Part 1
HTTP Server — accepts connections on a server socket, parses the request line, computes the resource and sends status line, headers and body §
VL10 – Web Applications, Part 2
HTTP Server — the Python server answering the fetch requests with JSON §
Web Applications
HTTP Server — the server loop: read until \r\n\r\n, parse the path, serve file/template/JSON safely §
| Access to private data | http://tura:8888//etc/passwd | serve only a whitelist; 403/404 otherwise | HTTP Server | §

Search in these mentions →