Category Archives: Leiningen

Incanter has migrated to Leiningen

I have completed the process of migrating Incanter from the Maven build tool to Leiningen.

Incanter now includes several project.clj files, one for the overall project and one for each submodule. The top-level project.clj file builds what was formerly called incanter-app/incanter-full using the modules incanter-core, incanter-charts, incanter-processing, incanter-io, incanter-pdf, and incanter-mongodb.

The modules live in the modules/ directory, and each is an independent Leiningen project that can be built as a standalone library.

The script/ directory has repl and swank scripts (both sh and bat) stolen from labrepl, and simple scripts for managing the builds of all the modules (install, test, clean).

One bonus of this change is that I have been able to push Incanter and its modules back on to Clojars.org.

To include the full library as a dependency in your Leiningen project, add the following entry to your project.clj file:

[incanter "1.2.1-SNAPSHOT"]

If you only want to include a subset of Incanter’s functionality in your project, use one of the following modules instead:

 [incanter/incanter-core "1.2.1-SNAPSHOT"]
 [incanter/incanter-charts "1.2.1-SNAPSHOT"]
 [incanter/incanter-io "1.2.1-SNAPSHOT"]
 [incanter/incanter-processing "1.2.1-SNAPSHOT"]
 [incanter/incanter-pdf "1.2.1-SNAPSHOT"]
 [incanter/incanter-mongodb "1.2.1-SNAPSHOT"]

Updated build instructions can be found in the new README available in Incanter’s Github repository.

Getting started with Incanter using labrepl

Stuart Halloway created a great Clojure tutorial and development environment for the PragmaticStudio Clojure workshop called labrepl, which is available on Github. A nice feature of labrepl is that is comes with several Clojure libraries, including Clojure-contrib, Compojure, and Incanter. It also includes scripts for starting a Clojure REPL and/or Swank server, and instructions for setting up several different IDEs for Clojure development.

In addition to starting a REPL, the included repl script starts a webserver, on port 8080, that includes several Clojure lab exercises and solutions.

It’s a great way to get started with Clojure and Incanter, so check it out. And if you’re an experienced Clojure programmer, Stuart is looking for contributors to create additional labs, so pitch in if you can. I plan to begin by creating some labs covering the material in my Incanter charts and datasets presentation.

Incanter code repository

It’s been a long time coming, but Incanter is available in a public code repository (http://repo.incanter.org) once again.

The version available in the Clojars repository has grown increasingly out of date, but couldn’t be updated due to Incanter’s new modular build structure; a structure that lets developers include only the subset of Incanter functionality that they need.

For instance, if you only need the functionality found in incanter.core and incanter.stats, then include the incanter-core dependency in your project.clj file. If you need charts but, not Processing visualizations, include incanter-charts. If you want access to the incanter.chrono library, but nothing else, use the incanter-chrono dependency.

Instructions for building Leiningen-based projects using the repository are available on the Incanter repo’s homepage, http://repo.incanter.org.

Building a Clojure Web application with Incanter, Compojure, and Leiningen

This post will demonstrate how to build a simple Clojure Web application, with Compojure, that will use Incanter to generate a sample from a normal distribution with parameters (size, mean, and standard deviation) provided by the user, and then return a PNG image of the histogram of the data.

This example is simple, but it demonstrates how to dynamically generate an Incanter chart and pass it directly to the Web client as a PNG image.

For more information on the Compojure Web framework, visit its Github repository, its Google group, its WikiBooks site, and this quick tutorial.

I will use the Leiningen build tool to download dependencies, compile the application, and package it up as jar file. For more information on installing and using Leiningen, see my previous post.

Let’s start off by creating a project directory, I’ll call it incanter-webapp, and give it a src subdirectory. Then create the following Leiningen project.clj file in the project directory.

(defproject incanter-webapp "0.1.0"
  :description "A simple Incanter web-app."
  :dependencies [[incanter "1.2.1-SNAPSHOT"]
                 [compojure "0.3.2"]]
  :main simple_web_app)

This project file includes two dependencies, Incanter and Compojure, and indicates that the -main function for the project is in the file simple_web_app.clj in the src subdirectory of the project.

Next create the file called simple_web_app.clj in the src subdirectory.

(ns simple_web_app
  (:gen-class)
  (:use [compojure]
        [compojure.http response]
        [incanter core stats charts])
  (:import (java.io ByteArrayOutputStream
                    ByteArrayInputStream)))

Note that the namespace for this file is simple_web_app with underscores like in the file name and in the :main line of the project.clj file.

In Compojure, you need to define routes, which in this case associate the function sample-form with “/”, and the function gen-samp-hist-png with “/sample-normal”.

(defroutes webservice
  (GET "/"
    sample-form)
  (GET "/sample-normal"
    (gen-samp-hist-png request
                       (params :size)
                       (params :mean)
                       (params :sd))))

The function gen-samp-hist-png takes four arguments, a request object, the sample size, the population mean, and the population standard deviation, and then updates the Compojure response object to include an Incanter histogram of a sample from a normal distribution with the given parameters.

Now define the gen-samp-hist-png function; start by converting the string-arguments, passed in from the params object, into numbers.

(defn gen-samp-hist-png
  [request size-str mean-str sd-str]
    (let [size (if (nil? size-str)
                 1000
                 (Integer/parseInt size-str))
          m (if (nil? mean-str)
              0
              (Double/parseDouble mean-str))
          s (if (nil? sd-str)
              1
              (Double/parseDouble sd-str))

Each argument has a default value, 1000 for size, 0 for mean, and 1 for the standard deviation (sd).

Next generate the sample from the normal distribution using Incanter’s sample-normal function with the given size, mean, and standard deviation, and then create a histogram of the resulting data.

          samp (sample-normal size
                              :mean m
                              :sd s)
          chart (histogram
                  samp
                  :title "Normal Sample"
                  :x-label (str "sample-size = " size
                                ", mean = " m
                                ", sd = " s))

I’ve used the x-label option of the histogram function to customize the x-axis label so that it includes the given sample size, mean, and standard deviation.

Next, we need to convert the histogram into a byte array, and feed that into a ByteArrayInputStream that can be passed to Compojure’s update-response function, along with a header that sets the Content-Type to “image/png”.

        out-stream (ByteArrayOutputStream.)
        in-stream (do
                    (save chart out-stream)
                    (ByteArrayInputStream.
                      (.toByteArray out-stream)))
        header {:status 200
                :headers {"Content-Type" "image/png"}}]
    (update-response request
                     header
                     in-stream)))

The above code is the key to sending a dynamically generated chart directly to the client. In addition to a filename, an OutputStream can be passed to Incanter’s save function; in this case, we use a ByteArrayOutputStream, which is then converted to a byte array that is used to initialize the ByteArrayInputStream that is passed to update-response.

We can create a Web form for submitting requests to /sample-normal. The following function is a helper function that creates a simple html page skeleton.

(defn html-doc
  [title & body]
  (html
    (doctype :html4)
    [:html
      [:head
        [:title title]]
      [:body
       [:div
	[:h2
	 [:a {:href "/"}
          "Generate a normal sample"]]]
        body]]))

Now define the sample-form function, which will generate a Web form using html-doc. This function was associated with “/” earlier using the defroutes macro.

(def sample-form
  (html-doc "sample-normal histogram"
    (form-to [:get "/sample-normal"]
      "sample size: " (text-field {:size 4} :size)
      "mean: " (text-field {:size 4} :mean)
      "sd: " (text-field {:size 4} :sd)
      (submit-button "view"))))

Now, we need to create a -main function that will be called when the incanter-webapp.jar is executed. This function will call Compojure’s run-server function, starting the built-in Jetty server on port 8080.

(defn -main [& args]
  (run-server {:port 8080}
    "/*" (servlet webservice)))

Now we can use Leiningen to download and install all the necessary dependencies, including Incanter and Compojure, and then build and package the app.

$ lein deps
$ lein compile
$ lein uberjar

Finally, to start the server, run

$ java -jar incanter-webapp-standalone.jar 

and visit http://localhost:8080 to submit a request, or go directly to the sample-normal app by constructing a URL like the following:

http://localhost:8080/sample-normal?size=500&mean=100&sd=10

Which returns a PNG image of a histogram like this:

The complete code for this example can be found here, and the Leiningen project file can be found here.

Building Incanter applications with Leiningen

[NOTE: Make sure to use Leiningen version 1.1.0 or greater. The version can be determined with ‘lein version’]

This is a quick guide to using Leiningen to build applications that use Incanter, based on the very useful post by Zef Hemel, “Building Clojure Projects with Leiningen“.

(Note: the following tutorial uses Unix shell commands (i.e. Mac OS X or Linux), but Roland Sadowski has created a Leiningen PowerShell script for Windows that works with this example: lein.ps1)

First get a copy of Leiningen’s self-installing script:

$ cd ~/bin
$ wget http://github.com/technomancy/leiningen/raw/stable/bin/lein
$ chmod +x lein

and run the self-installer.

$ lein self-install

Now create a project directory called ‘incanter-helloword’ and give it a ‘src’ subdirectory.

$ mkdir incanter-helloworld
$ mkdir incanter-helloworld/src
$ cd incanter-helloworld

Next create the following project.clj file, which includes Incanter in the dependencies section:

(defproject incanter-helloworld "0.1.0"
  :description "The Incanter version of hello world."
  :dependencies [[incanter "1.2.3-SNAPSHOT"]]
  :main hello)

The project will have a single source file called hello.clj, which will contain a -main function that will be called when the jar file is executed. Create it in the incanter-helloworld/src directory:

(ns hello
  (:gen-class)
  (:use (incanter core stats charts)))

(defn -main [& args]
  (view (histogram (sample-normal 1000))))

The program will display a histogram of sample data from a standard normal distribution.

Now download and install all the project’s dependencies, including Incanter, using Leiningen’s ‘deps’ command:

$ lein deps

Compile the project:

$ lein compile

And build an ‘uberjar’ that includes all the Incanter jar files:

$ lein uberjar

And finally, run incanter-helloworld,

$ java -jar incanter-helloworld-standalone.jar

which displays the following histogram.

You can also start a Clojure REPL with a CLASSPATH that includes all your project’s dependencies, including Incanter and its dependencies, using the following command:

$ java -cp 'lib/*' clojure.main

The lein repl command won’t work correctly, since it uses Clojure 1.1 and Incanter requires Clojure 1.2. You can also use the repl and swank scripts available in the scripts directory of the Incanter distribution on Github.