• chevron_right

      Dumping Common Lisp streams

      Michał "phoe" Herda · Thursday, 20 December, 2018 - 15:16 · 1 minute

    I have been writing a small utility for debugging HTTP requests towards a particular application. The application uses multipart/form-data encoding in a very peculiar way. It made using tools like Postman very hard to use - Postman autodetected individual parts' content type based on file extensions, where I had to set it manually in order for the application to accept the request at all.

    One of the goals of that application was dumping the full HTTP request and response into separate files for debugging purposes. After realizing that Drakma's main function is a freaking huge monolith that is impossible to extend without modifying the source code, I have asked #lisp on Freenode for advice. jackdaniel and jdz suggested that I manually create the stream, modify it to tap into its elements, and then pass it to Drakma, my HTTP client.

    After a bit of tweaking (and help from nirved on #lisp), I found the proper way to manually create the original stream and pass it to Drakma. (Spoiler: Drakma always expects octet-streams passed by the :STREAM keyword to DRAKMA:HTTP-RESPONSE.)

    Now, for tapping into that stream, I thought that I was forced to use Gray streams for achieving that. It was logical to me that I had to grab every element read from or written to the wrapper stream, and duplicate it across the original stream and any other streams I wanted to dump the I/O to.

    Then pjb from #lisp suggested a much better alternative. It did not require Gray streams at all - in fact, it used only functions from standard Common Lisp, without any extensions at all.

    (defun make-dumped-stream (i/o input-output output-output)
      (make-two-way-stream
       (make-echo-stream i/o input-output)
       (make-broadcast-stream i/o output-output)))
    

    I have polished this function a little bit and added it to my personal toolbox library. Its final source code is also posted below.

    (defun make-dumped-stream
        (input-output-stream &key dump-input-stream dump-output-stream)
      "Returns a wrapper stream around the original stream. All data read from the
    wrapper stream is additionally sent to DUMP-INPUT-STREAM. All data written to
    the wrapper stream is additionally sent to the DUMP-OUTPUT-STREAM."
      (if (or dump-input-stream dump-output-stream)
          (make-two-way-stream
           (if dump-input-stream
               (make-echo-stream input-output-stream dump-input-stream)
               input-output-stream)
           (if dump-output-stream
               (make-broadcast-stream input-output-stream dump-output-stream)
               input-output-stream))
          input-output-stream))