I am currently working on a project that uses Haskell’s http-client
library. I wanted to ensure that my application does not use excessive memory. http-client
provides ways to control consumption of response bodies, but the library consumes HTTP headers behind the scenes, so I needed to check that the processes it uses control their memory use appropriately.
It turns out that http-client
is indeed robust. It provides protection from memory exhaustion as a result of overlong headers by limiting the number of headers in a response and their lengths:
http-client
limits responses to 100 headers and raises theOverlongHeaders
exception if there are more than that.100 _ = throwHttp OverlongHeaders parseHeaders = do parseHeaders count front <- connectionReadLine conn line if S.null line then return $ front [] else do <- parseHeader line mheader case mheader of Just header -> + 1) $ front . (header:) parseHeaders (count Nothing -> -- Unparseable header line; rather than throwing -- an exception, ignore it for robustness. parseHeaders count front
When reading lines from the socket, the library only reads a new chunk if the current line has not exceeded 4096 bytes. Again, it raises the
OverlongHeaders
exception if a line carries over into a chunk beyond this.= connectionReadLineWith conn bs0 id 0 go bs0 where = go bs front total case S.break (== charLF) bs of "") -> do (_, let total' = total + S.length bs > 4096) $ throwHttp OverlongHeaders when (total' <- connectionRead conn bs' $ throwHttp IncompleteHeaders when (S.null bs') . (bs:)) total' go bs' (front 1 -> y) -> do (x, S.drop $! connectionUnread conn y unless (S.null y) return $! killCR $! S.concat $! front [x]