Ring's idiosyncrasies

The promise of Ring is to let us assemble web applications from modular parts. In other words, Ring endows us with composability. Images of Lego blocks neatly piled up into a larger structure come to mind, but as usual in the software world, things are a bit more involved. In this post, we will look in greater detail at Ring’s idiosyncrasies.

For starters, let’s recap.

A Ring handler is a function that takes a request and returns a response.
A request is a map with standard keys, notably the :uri and :request-method.
A response is a map containing :status, :headers and :body.
Ring middleware is a regular function that receives a handler (and optionally, additional arguments) and returns a new handler that will call the original handler.
A Ring adapter takes a handler and converts it into a running web server.

Note: A handler is a first-order function, Ring middleware is higher-order: it takes a function as input and returns one as output.

Requests are being processed by the handler, which in turn generates an appropriate response.

(defn handler [req] 
 {:status  200
  :headers {"content-type" "text/plain"}
  :body    "Some text"})

You can now typically write something like:

(run handler {:port 4040})

Where run or serve is the entry point for a particular adapter implementation interfacing with Jetty or any other underlying Java web server.

(serve handler {:port 4040})

These are the foundations of web application development in Clojure. There is nothing else to it. Isn’t that neat?

(handler (mock/request :get "/"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Some text"}

Note: Try the code snippets in a Clojure REPL.

In The Little Schemer by Dan P. Friedman et al, a socratic dialogue leads the reader through a concept by discussing every aspect of it. In this primer, which I dub The Little Ringleader, we are going on a little adventure where the idea is to explore the implications of a few premises.

Let’s define middleware that does nothing.

(defn middleware [handler]
  (fn [request]
    (handler request)))

((middleware handler)  (mock/request :get "/doc/10"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Some text"}

Let's define some more middleware. You can chain them with impunity.

(defn wrap-noop1 [handler]
  (fn [request]
    (handler request)))

(defn wrap-noop2 [handler]
  (fn [request]
    (handler request)))

(((comp wrap-noop1 wrap-noop2) handler)
 (mock/request :get "/doc/10"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Some text"}

This handler returns the same response regardless of the request. Not very useful. What we want is to match endpoints (uris) with handlers. In other words, we want a routing mechanism. Here is a naive implementation.

(defn handler-with-routing [req] 
  (if (= (:uri req) "/doc/10")
    {:status  200
     :headers {"content-type" "text/plain"}
     :body    "Some text"}
    {:status  200
     :headers {"content-type" "text/plain"}
     :body    "404"}))

(((comp wrap-noop1 wrap-noop2) handler-with-routing)
 (mock/request :get "/doc/10"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Some text"}

(((comp wrap-noop1 wrap-noop2) handler-with-routing)
 (mock/request :get "/doc/11"))

{:status 200, :headers {"content-type" "text/plain"}, :body "404"}

We've got ourselves some routing. Hang on, can you see a problem with this approach? If we do the routing inside the handler, we are sacrificing the most desirable property of Ring: modularity. We really would like to be able to define handlers in arbitrary namespaces, and ultimately assemble them together as one.

This problem space is covered by many routing libaries/frameworks, but the remainder of this post will focus on compojure.

Compojure is a small routing library for Ring that allows web applications to be composed of small, independent parts.

In Compojure, one combines handlers with the routes function. Building on our previous example, it looks like this:

(in-ns 'france)
(clojure.core/refer 'clojure.core)

(defn handler [req] 
 {:status  200
  :headers {"content-type" "text/plain"}
  :body    "Bonsoir"})

(in-ns 'wales)
(clojure.core/refer 'clojure.core)

(defn handler [req] 
 {:status  200
  :headers {"content-type" "text/plain"}
  :body    "Noswaith dda"})

((routes france/handler wales/handler)
 (mock/request :get "/doc/10"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Bonsoir"}

Did you notice something? Our response was in French! Let’s try again, but with the order reversed.

((routes wales/handler france/handler)
 (mock/request :get "/doc/10"))

{:status 200,
 :headers {"content-type" "text/plain"},
 :body "Noswaith dda"}

Back to Welsh! This leads us to our first takeaway: Compojure routes are matched in order.

When you combine two handlers like that, we are not really leveraging the routing capabilites of Compojure. Compojure works best when we compile routes with the macros GET, POST, etc. Those will create handlers for us!

Note: Under the hood, Compojure relies on a library called clout to compile routes. The syntax is borrowed from Ruby on Rails’s routing machinery.

(require '[compojure.core :refer [GET]])

(def app (routes (GET "/french" []
                   "Bonsoir")
                 (GET "/welsh" []
                   "Noswaith dda")))

(app (mock/request :get "/french"))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

(app (mock/request :get "/welsh"))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Noswaith dda"}

If you wonder why the response suddenly has a different content-type, that’s because Compojure defaults to text/html; charset=utf-8 when you pass a string. The protocol by which Compojure determines how to handle the return value of compiled routes is extensible.

Typically, you would pass a response map.

((routes
  (GET "/french" [] {:status  200
                     :headers {"content-type" "text/plain"}
                     :body    "Bonsoir"})
  (GET "/welsh" [] "Noswaith dda"))
 (mock/request :get "/french"))

{:status 200, :headers {"content-type" "text/plain"}, :body "Bonsoir"}

This time, Compojure rendered the response in text/plain.

Compojure routes are semantically the same as Ring handlers, with the exception that routes may return nil to indicate they do not match.

(app (mock/request :get "/doc/10"))

At the risk of being pedantic, let’s prove that we can combine compiled routes across namespaces just the same as regular handlers.

(in-ns 'france)
(require '[compojure.core :refer [routes GET]])

(def french-routes (routes
                    (GET "/french" []
                      "Bonsoir")))

(in-ns 'wales)
(require '[compojure.core :refer [routes GET]])

(def welsh-routes (routes
                   (GET "/welsh" []
                     "Noswaith dda")))

As before, we have a composable system:

((routes wales/welsh-routes france/french-routes)
 (mock/request :get "/french"))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

Look how we can freely compose middleware and handlers in this system.

((routes (wrap-noop1 wales/welsh-routes) 
         (wrap-noop2 france/french-routes))
 (mock/request :get "/french"))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

Or even:

((middleware (routes
              (wrap-noop1 wales/welsh-routes)
              (wrap-noop2 france/french-routes)))
 (mock/request :get "/french"))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

Until now, everything is perfect because our middleware is pure and everything is composable. We are talking Lego-like composability. That’s because we have avoided the elephant in the room: mutable state. Indeed, middleware is not always pure, it can and does mutate the request.

Remember the standard keys? So far we have mentioned two: :uri and :request-method. But requests that originate from html forms or ajax calls, for example, may have a :body key present that correspond to user input or call arguments encoded in an InputStream.

Can you spot the InputStream in the generated request?

(mock/request :post "/french" {:msg "Bonsoir"})

{:remote-addr "localhost",
 :headers
 {"host" "localhost",
  "content-type" "application/x-www-form-urlencoded",
  "content-length" "11"},
 :server-port 80,
 :content-length 11,
 :content-type "application/x-www-form-urlencoded",
 :uri "/french",
 :server-name "localhost",
 :query-string ,
 :body
 #object[java.io.ByteArrayInputStream 0x5f3369ed "java.io.ByteArrayInputStream@5f3369ed"],
 :scheme :http,
 :request-method :post}

Let’s compose a route with middleware that extracts that InputStream and adds a params key to the request.

Note: Middleware often add keys to the request map in addition to the standard keys.

(require '[compojure.core :refer [POST]] 
         '[ring.middleware.params :refer [wrap-params]])

((wrap-params (POST "/french" [msg] msg))
 (mock/request :post "/french" {:msg "Bonsoir"}))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

Note: Actually, wrap-params adds three keys to the request:

:query-params - a map of parameters from the query string
:form-params - a map of parameters from the body
:params - a merged map of all types of parameter

(in-ns 'france)

(require '[compojure.core :refer [routes POST]])

(def french-routes (routes (POST "/french" [msg] msg)))

(in-ns 'wales)
(require '[compojure.core :refer [routes POST]])

(def welsh-routes (routes (POST "/welsh" [msg] msg)))

You might be tempted to write something like this:

((routes
  (wrap-params wales/welsh-routes)
  (wrap-params france/french-routes))
 (mock/request :post "/welsh" {:msg "Noswaith dda"}))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Noswaith dda"}

((routes
  (wrap-params wales/welsh-routes)
  (wrap-params france/french-routes))
 (mock/request :post "/french" {:msg "Bonsoir"}))

Oops, we got nil. Side-effectful middleware such as wrap-params consumes the request body upon reading it. The InputStream in the body is available until it is read, and if it is accessed before a route is matched, it is gone forever.

We can fix that if we shuffle things around a bit.

((wrap-params
  (routes wales/welsh-routes france/french-routes))
 (mock/request :post "/french" {:msg "Bonsoir"}))

{:status 200,
 :headers {"Content-Type" "text/html; charset=utf-8"},
 :body "Bonsoir"}

As you can see, routes defined in separate namespaces have to be combined before applying effectful middleware. In the real world, this will come up when integrating routes from third-party libraries in your web application. Think authentication and authorization, for example. Safe for now, but that issue comes up time and again.

Sente endpoints, for example, cannot share the same routes as the application. Application routes will typically rely on libraries such as ring-middleware-format or its successor, muuntaja, for encoding and decoding of data on the wire, while Sente does data serialization on its own.

To mitigate pesky middleware ordering issues, Ring provides sane defaults for common use cases. Still, you need to beware of conflicting middleware and remember that, despite Clojure's embrace of immutability, we are not entirely out of the tar pit.