Category Archives: Incanter

Incanter 1.5.4 has been released

After several weeks since 1.5.2 release the new release of Incanter (with version number 1.5.4) has been deployed to Clojars.  This release contains several fixes & adds several new functions to incanter.core & incanter.charts.  See Changelog for more details.

P.S. 1.5.3 release was skipped because of an error that I made during deployment to Clojars. Don’t use it!

Incanter 1.5.2 (bugfix release)

I’ve just pushed the new release of Incanter to Clojars. This is mostly bugfix release that fixes several bugs, of different severity. There is also one enhancement – you can use logarithmic axes in charts. More information about fixed bugs and new functions is at changelog.

I want to thank all people who helped with new release by creating issues & sending pull requests!

Incanter 1.5.1: Bugfix release

The bugfix release of Incater was just pushed to Clojars.

It fixes 2 bugs – one nasty bug when functions called from transform-with were working on underlying data, modifying them, and 2nd – when 2-argument version solve thrown an error.

Incanter 1.5.0 has been released!

After long development, the new version of Incanter has been released to clojars. This version contains a lot of changes, including following:

  • For matrix operations the clatrix/jblas libraries are used – this should greatly improve performance (but you may need to install libgfortran3 package on Linux 64-bit. See jblas wiki for more information)
  • Interpolation functions were added to incanter-core module
  • You can save charts in SVG format by using the new incanter-svg module
  • The incanter-chart module was extended with support for scatter plots
  • Many bugfixes

Full list of changes could be found in changelog. There are 2 open issues that could affect your use of Incanter, please read about them in changelog. We’ll try to address them in the next release.

I want to say “Thank You!” to all people who provided new functionality, fixed the bugs, filed the issues, and participated in discussions in mailing list. Without you, this release simply didn’t happen!

The next steps in development will happen in direction of release 2.0, where the main change will be use of core.matrix library for all matrix operations.

Incanter executables

For a long time I’ve wanted a simple, double-clickable executable for Incanter, and now thanks to Alan Dipert’s swingrepl library I have it.

I have made an executable jar file, called incanter.jar, that launches a swingrepl, that includes Incanter and all of its dependencies, when double-clicked or started with the following command

java -jar incanter.jar

Note: Make sure to save incanter.jar with the .jar extension to ensure it executes correctly.

I have also used Apple’s Jar Bundler and SkoobySoft’s DMG Packager to create a Mac OS X executable that’s downloadable as a dmg file, Incanter.dmg.

Finally, I used Launch4j to create the Windows executable file,
Incanter.exe.

To build incanter.jar from source, use script/package in the Incanter distribution on Github, or just use lein uberjar (lein will call the file incanter-standalone.jar, and will create a non-executable jar called incanter.jar).

Reading and writing Excel (xls) files with Incanter

I have just added David James Humphreys’ incanter-excel module to the Incanter distribution, providing basic capabilities for reading Microsoft Excel spreadsheets in as Incanter datasets and saving datasets back out as xls files.

I have posted a simple spreadsheet of Australian airline passenger data from the 1950s to the Incanter website for the following example. The read-xls function can read xls files given either a filename or URL, so you won’t need to download the file.

Start by loading the necessary libraries, including incanter.excel.

(use '(incanter core charts excel))

Next, we can use the with-data macro to bind a dataset converted from the above xls file, using the read-xls function, and then view it.

(with-data (read-xls "http://incanter.org/data/aus-airline-passengers.xls")
  (view $data)

The read-xls function takes an optional argument called :sheet that takes either the name or index of the worksheet from the xls file to read (in this case either “dataset” or 0) , it defaults to 0.

[NOTE: A current weakness of read-xls is that cells containing formulae, as opposed to actual data, are not imported (i.e. the cells remain empty).]

Finally, we’ll create a time-series plot of the data. However, the time-series-plot needs time in milliseconds, so we’ll first create a function that converts the date column from Java Date objects to milliseconds, and then view the plot.

  (let [to-millis (fn [dates] (map #(.getTime %) dates))] 
    (view (time-series-plot (to-millis ($ :date)) ($ :passengers)))))

Datasets can also be saved as Excel files using the save-xls function. The following example just reads in one of the sample datasets using incanter.datasets/get-dataset and then saves it as an xls file.

(save-xls (get-dataset :cars) "/tmp/cars.xls")

The incanter-excel module is now included in the Incanter distribution on Github, and is available as a separate dependency from the Clojars repository. The complete code from this example can be found here.

Infix mathematical notation in Incanter

I obviously love Clojure’s prefix notation, but there are times when it is more concise, and familiar, to represent mathematical formulae using infix notation, so I have integrated the infix portion of Jeffrey Bester’s Clojure math library into Incanter. There is now a formula macro called $= for evaluating infix mathematical formulas in incanter.core.

Here’s an example,

user> (use 'incanter.core)
nil
user> ($= 7 + 8 - 2 * 6 / 2)
9

Note that there must be spaces between values and operators.

Vectors can be used instead of scalars. For instance, to add 5 to each element of the vector [1 2 3],

user> ($= [1 2 3] + 5)
(6 7 8)

or perform element-by-element arithmetic on vectors and matrices.

user> ($= [1 2 3] + [4 5 6])
(5 7 9)

user> ($= [1 2 3] * [1 2 3])
(1 4 9)

user> ($= [1 2 3] / [1 2 3])
(1 1 1)

user> ($= (matrix [[1 2] [4 5]]) + 6)
[ 7.0000  8.0000
10.0000 11.0000]

user> ($= (trans [[1 2] [4 5]]) + 6)
[7.0000 10.0000
8.0000 11.0000]

Examples of exponents using the ** function.

user> ($= 8 ** 3)
512.0

user> ($= 8 ** 1/2)
2.8284271247461903

user> ($= 2 ** -2)
0.25

user> ($= [1 2 3] ** 2)
(1.0 4.0 9.0)

Parens can be used for grouping,

user> ($= 10 + 20 * (4 - 5) / 6)
20/3

user> ($= (10 + 20) * 4 - 5 / 6)
715/6

user> ($= 10 + 20 * (4 - 5 / 6))
220/3

including arbitrarily nested groupings.

user> ($= ((((5 + 4) * 5))))
45

Of course, variables can be used within the formula macro,

user> (let [x 10
            y -5]
        ($= x + y / -10))
21/2

and mathematical functions like sin, cos, tan, sq, sqrt, etc. can be used with standard Clojure prefix notation.

user> ($= (sqrt 5) * 5 + 3 * 3)
20.18033988749895

user> ($= sq [1 2 3] + [1 2 3])
(2 6 12)

user> ($= sin 2 * Math/PI * 2)
5.713284232087328

user> ($= (cos 0) * 10)
10.0

user> ($= (tan 2) * Math/PI * 10)
-68.64505182223235

user> ($= (asin 1/2) * 10)
5.23598775598299

user> ($= (acos 1/2) * 10)
10.47197551196598

user> ($= (atan 1/2) * 10)
4.636476090008061

Functions can also be applied to vectors,

user> ($= [1 2 3] / (sq [1 2 3]) + [5 6 7])
(6 13/2 22/3)

user> ($= [1 2 3] + (sin [4 5 6]))
(0.2431975046920718 1.0410757253368614 2.720584501801074)

Boolean tests are also supported.

user> ($= 3 > (5 * 2/7))
true

user> ($= 3 <= (5 * 2/7))
false

user> ($= 3 != (5 * 2/7))
true

user> ($= 3 == (5 * 2/7))
false

user> ($= 3 != 8 && 6 < 12)
true

user> ($= 3 != 8 || 6 > 12)
true

Matrix multiplication uses the <*> function (equivalent to R’s %*% operator).

user> ($= [1 2 3] <*> (trans [1 2 3]))
[1.0000 2.0000 3.0000
2.0000 4.0000 6.0000
3.0000 6.0000 9.0000]

user> ($= (trans [[1 2] [4 5]]) <*> (matrix [[1 2] [4 5]]))
[17.0000 22.0000
22.0000 29.0000]

user> ($= (trans [1 2 3 4]) <*> [1 2 3 4])
30.0

user> ($= [1 2 3 4] <*> (trans [1 2 3 4]))
[1.0000 2.0000  3.0000  4.0000
2.0000 4.0000  6.0000  8.0000
3.0000 6.0000  9.0000 12.0000
4.0000 8.0000 12.0000 16.0000]

The Kronecker product uses the <x> function (equivalent to R’s %x% operator).

user> ($= [1 2 3] <x> [1 2 3])
[1.0000
2.0000
3.0000
2.0000
4.0000
6.0000
3.0000
6.0000
9.0000]

user> ($= (matrix [[1 2] [3 4] [5 6]]) <x> 4)
[ 4.0000  8.0000
12.0000 16.0000
20.0000 24.0000]

user> ($= (matrix [[1 2] [3 4] [5 6]]) <x> (matrix [[1 2] [3 4]]))
[ 1.0000  2.0000  2.0000  4.0000
 3.0000  4.0000  6.0000  8.0000
 3.0000  6.0000  4.0000  8.0000
 9.0000 12.0000 12.0000 16.0000
 5.0000 10.0000  6.0000 12.0000
15.0000 20.0000 18.0000 24.0000]

user> ($= [1 2 3 4] <x> 4)
[ 4.0000
 8.0000
12.0000
16.0000]

user> ($= [1 2 3 4] <x> (trans [1 2 3 4]))
[1.0000 2.0000  3.0000  4.0000
2.0000 4.0000  6.0000  8.0000
3.0000 6.0000  9.0000 12.0000
4.0000 8.0000 12.0000 16.0000]

Here’s an example using the formula macro in a function-plot of x^3 - 5x^2 + 3x +5 , (I will use the incanter.latex/add-latex function to add a LaTeX annotation to the chart).

(use '(incanter core charts latex))
(doto (function-plot (fn [x] ($= x ** 3 - 5 * x ** 2 + 3 * x + 5)) -10 10)
  (add-latex 0 250 "x^3 - 5x^2 + 3x +5")
  view)

The next example will use the car-speed-to-breaking-distance sample data (the :cars dataset). I’ll partition the data, first selecting rows where the ratio of dist/speed is less than 2, then selecting rows where the ratio is greater than 2, and finally adding a line showing this threshold.

First, use the with-data macro, passing it the :cars sample data. Then apply the $where function in order to filter the data; pass it a $fn, which is a bit of syntax sugar around Clojure’s fn function that does map destructuring for functions that operate on dataset rows. For instance

($fn [x y] (…))

becomes

(fn [{:keys x y}] (…))

The parameter names specified in $fn must correspond to the column names of the dataset.

(use '(incanter core datasets charts))
(with-data (get-dataset :cars)
  (doto (scatter-plot :speed :dist 
          :data ($where ($fn [speed dist] 
                          ($= dist / speed < 2))))
    (add-points :speed :dist 
      :data ($where ($fn [speed dist] 
                      ($= dist / speed >= 2))))
    (add-lines ($ :speed) ($= 2 * ($ :speed)))
    view))

The complete code for this post is available here.