Building A Clojure REST Service: Part One

Iteration 0

Ever since I started playing around with Sublime Text 2 and Clojure, I've wanted to put all I've learned into a simple but complete project to make sure I understand how I would work in a real project environment from inception to deployment, so that if (when?) I come to use Clojure in a commercial project there are few surprises.

In a fit of madness, I've also decided it would make a good series of posts for the blog, so I'm going to air all that dirty laundry right here while I work on the project - mistakes & all.

In true agile fashion, I'm going to make up a lot of the features of the app as I go along. The initial requirement from the user (me) is: I want to keep a to-do list online, accessible from a desktop browser or mobile device. I'll explore other features & options as I go, as I want to see what effect adding or changing major features has on how I refactor the code.

Here are the basic principles of the series:

• The project itself will be simple: the focus is on the implementation, not the project domain. I've chosen to do a To-Do list I call: Do It, Do It Now!
• I'm going to start by writing a RESTful web service API, which will eventually be fronted  by a web client, mobile web client, Android client, etc.
• The API will be written in Clojure, with Compojure& Ring providing the basic HTTP/REST functionality. There are libraries I could use, such as Liberator, but Compojure & Ring are ubiquitous in the Clojure world and I want to experiment with the "nuts & bolts" of the service instead of hiding behind higher-level abstractions.
• Unit tests will be kept up-to-date and run clean at the end of each iteration.
• I will try to adhere as closely as possible to the HTTP specification and the principles of REST, and make full use of HTTP methods, headers, return codes, etc.
• The REST data format of choice will be JSON: data sent to the API will be sent as a request body and data received from the API will be in the response body.
• The API will be secured using HTTP Basic Authentication, and the app will be multi-user.
• The data store will be a MongoDB database.
• The app should be capable of running in a basic, (free) developer Heroku environment, so that I can get some experience with Heroku, and use it over the Internet.

Iteration 1

First, let's create a basic project skeleton using Leiningen: lein new compojure doitnow. This uses a very convenient template from James Reeves (weavejester) to create a Compojure application.

Open Sublime Text 2, make sure there's no project open and add the new app folder to the empty project. Save the project files out to the app folder. You should have the initial project structure shown here. If you've got your environment set up from my previous posting, you can use Sublime Text to build the project (ctrl-b), run tests (ctrl-shift-p, "Test") or generate documentation (ctrl-shift-p, "Documentation").

Take some time to create a Git repository around the project, stage & commit. I like to add the doc folder (generated by the Leiningen documentation build) to .gitignore - the documentation can be easily generated and I don't want it checked in. It's also a good idea to ignore the Sublime Text project & workspace files, as they contain machine/user/path-specific details that will likely cause anyone else's workstation to throw a fit, so add those to .gitignore as well.

Now let's set up the project.clj file with some details and our initial dependencies:

(defproject doitnow "0.1.0-SNAPSHOT"
  :description "Sample project for blog article: Building A Clojure REST Service"
  :url "https://github.com/3rddog/doitnow"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]
                 [compojure "1.1.3"]
                 [ring-middleware-format "0.2.2"]]
  :plugins [[lein-ring "0.7.5"]]
  :ring {:handler doitnow.handler/app}
  :profiles
    {:dev
      {:dependencies [[ring-mock "0.1.3"]]}})

(ns doitnow.handler
  (:use compojure.core
        ring.util.response
        [ring.middleware.format-response :only [wrap-restful-response]])
  (:require [compojure.handler :as handler]
            [compojure.route :as route]))

(defroutes api-routes
  (context "/api" []
    (OPTIONS "/" []
      (->
        (response {:version "0.1.0-SNAPSHOT"})
        (header "Allow" "OPTIONS"))))
  (route/not-found "Nothing to see here, move along now"))

(def app
  (->
    (handler/api api-routes)
    (wrap-restful-response)))

What we have here are two API routes defined.

The first is under the URL context /api - I wanted to distinguish REST API calls from the rest of the application as I may want to serve an HTML5/JavaScript client from the root context in future. To the API context, I've added a single route: an OPTIONS request to /api will return a JSON map with some API settings and information - in this case, just the version number of the API for starters.

The second route will catch any URL that is not defined as a route and automatically return an HTTP 404 (Not Found) response. So, if we hit a URL that doesn't exist, we will always get a valid (if unexpected) 404 response.

Then we set up the handler chain to respond to incoming requests. Our api-routes are added using the Compojure handler/api function. If you go back and take a look at the project.clj file you will see the line :ring {:handler doitnow.handler/app}, which tells Ring that the application entry point is the app function we just defined.

I've also added a piece of Ring middleware: wrap-restful-response. This middleware function looks at the Accept header in the incoming request and tries to return the response in the specified format. So, if you ask for application/json in the request, you should get JSON in the response body. So, I can just return a map (for example) from my handler functions and that will be converted to a JSON map in the response body by the middleware.

Now we can start the server with lein ring server and look for the status message:

[main] INFO org.eclipse.jetty.server.Server - jetty-7.6.1.v20120215
Started server on port 3000
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started SelectChannelConnector@0.0.0.0:3000

Leiningen & Ring are also good enough to open a browser for us and take us straight to the root URL, which in this case will show the "not found" message: Nothing to see here, move along now. We can check that the OPTIONS URL is working with:

[paul@ubuntu ~]$curl -i -X OPTIONS -H "Accept: application/json" http://localhost:3000/api

HTTP/1.1 200 OK
Content-Length: 28
Allow: OPTIONS
Content-Type: application/json; charset=utf-8
Server: Jetty(7.6.1.v20120215)

{"version":"0.1.0-SNAPSHOT"}

As per the W3 specification, the response to OPTIONS includes a Content-Type header (courtesy of the wrap-restful-response function), since we are returning a response body, and an Allow header specifying the HTTP methods that the URL allows.

Note that right now, if we hit the URL with a GET request we will get a 404 (Not Found) response which really should be a 405 (Method Not Allowed). Let's fix that now. Change the route definitions to look like this:

(defroutes api-routes
  (context "/api" []
    (OPTIONS "/" []
      (->
        (response {:version "0.1.0-SNAPSHOT"})
        (header "Allow" "OPTIONS")))
    (ANY "/" [] 
      (->
        (response nil)
        (status 405)
        (header "Allow" "OPTIONS"))))
  (route/not-found "Nothing to see here, move along now"))

Because Ring handlers are applied in the order they are declared, we can simply add an ANY handler within the /api context after the OPTIONS handler. So, if an incoming request uses the OPTIONS method, it will be caught by the first handler and a 200 response returned. If any other method is used the request falls through to the ANY handler and a 405 is returned.

Lastly, let's get the unit tests working. Edit test/doitnow/test/handler.clj to this:

(ns doitnow.test.handler
  (:use clojure.test
        ring.mock.request  
        doitnow.handler))

(deftest test-api-routes
  (testing "API Options"
    (let [response (api-routes (request :options "/api"))]
      (is (= (response :status) 200))
      (is (contains? (response :body) :version))))
  (testing "API Get"
    (let [response (api-routes (request :get "/api"))]
      (is (= (response :status) 405))
      (is (nil? (response :body)))))
  (testing "Not Found"
    (let [response (api-routes (request :get "/invalid"))]
      (is (= (response :status) 404)))))

We're checking that when we hit the URL /api we get back a response with a status code of 200 and a body that consists of a map containing the key version, and that when we hit a non-existent URL, we get a 404 response. There's also a test in there for the 405 response when we hit the URL with anything other than an OPTIONS request ... well, a GET request anyways - I could add all the other HTTP methods for completeness, but not right now.

That's it for now. More iterations will follow when I get the time, and if you don't want to manually set up the project as described above, you can always clone or fork it from my Github repository and work from there.