hm: generate a heatmap from 3D numeric data / ariel faigon

Note: there's a newer/better version of this software called 'xyz' available from my github.com/scripts repository. It is written in python, and has more options. Most importantly: it deals better with interpolation so it can accept wider data-set variations, even when they are missing grid points, and do a reasonable job figuring out the heat-map from the data.

Table of Contents:

Intro

Screenshots

Features

Requirements

Download

Usage

Documentation

GPL License

Thanks

Like it?

Introduction

hm (abbreviation for 'heat-map') is a little utility to quickly and easily visualize 3D numeric data. It is implemented as a handy perl script on top of R and ImageMagick

You give hm a flat data file (CSV, TSV) with numbers and you get back an image showing your data in a more understandable way. It is that simple.

In essence, hm is just a user-friendly wrapper. I didn't try to reinvent wheels where they already existed. The heavy lifting is done by R and ImageMagick. hm provides an intuitive user interface to both the data (simple TSV or CSV files) and the image rendering parameters. In addition if there are problems with the data, hm should detect them and hopefully give a more helpful/actionable message than R does.

Screenshots

A picture is worth a 1000-words. Here are some hm generated images. All the images were generated from the same data-set: The volcano data-set by Ross Ihaka, which was converted by me to (X,Y,Z) tsv format. The actual images are bigger, I shrunk these just to save bandwidth.

You may also change the labels (xlabel='...', ylabel='...' zlabel='...') add a grid (grid=1), and more.

Features

In brief: you give hm a data file (CSV/TSV) with numeric columns and it displays an image showing your data in a more understandable way. By default, the 1st data column is mapped to the X-axis, the 2nd data column, to the Y-axis, and the 3rd data column to a color (Z-axis).

hm supports:

• setting graphical parameters from the command-line
• support any non-ambiguous abbreviation of a parameter name
• mapping any data-column to any image axis (X, Y, Z). This is handy if you have, say, more than 3 columns, or you want to remap columns to different axes.
• double mapping a data column to two or more axes (handy in case you have less than 3 columns)
• auto-detection of the column separator (CSV, TSV, semicolon) More requests?
• auto-detection of non-numeric header (1st line) in TSV/CSV and auto axis-labeling.
• increasing or decreasing X, Y, Z resolution
• range clipping/trimming into min/max values on any axis
• Z range clamping (max forced down, min forced up, instead of being trimmed)
• setting image size: width, height
• color control: 1) number of distinct colors, 2) gamma value
• Adding contour lines and/or a rectangular grid
• several color-schemes for the Z dimension at the time of this writing the supported schemes are: 'rainbow', 'brainbow', 'danger', 'topo', 'terrain', 'gray', 'cm'. The default is my personal favorite: rainbow.
• linear transformations and log transformations on the Z-axis: stretching/shrinking, displacing via (zcmin, zcmax).
• Inspecting your data and telling you exactly where (file, line, column, value) it found a problem. This is something that R cryptic errors are pretty unhelpful about.

Requirements

hm should work out of the box on any Linux system which has R and ImageMagick installed. I haven't tried it in other environments but I see no reason why it shouldn't work if you have R and ImageMagick. If you got it to work in any env other than Linux, please let me know, thanks.

[Update: many thanks to Kurt Kincaid who reports he got hm to work on Windows-7 64bit, running Strawberry Perl (v5.16.0) by changing the path delimiter from ":" to ";", and adding ".exe" to the end of the executable prerequisites (R.exe and display.exe) ]

Download

hm is just one self-contained perl-script, click here to download it

Usage

When you call hm without paramaters, it gives a brief usage message like this:

  $ hm
  You must supply a datafile as input
  Usage: hm [options] [varname=value]... [idx..] DataFile [ImgFile]
        Options:
            -v          verbose
            -D          don't display the image
            -S          Sum z-values (instead of averaging them)
            -s sep     Use sep as input column separator
            -i iopt    Pass iopt to display program (default display)

        Supported Varnames (prefix-abbreviations work too):
            black cocolor colabels colinetype colinewidth colorscheme
	    contour frame gamma grcolor grid grlinetype grlinewidth
	    header height imgfile maxres ncolors nxtics nytics redder
	    sep sum title unset_val width xindex xlabel xlog xmax xmin
	    xres yindex ylabel ylog ymax ymin yres zadd zcmax zcmin
	    zindex zlabel zlevels zlog zmax zmin zres zscale ztrim

        idx             Numeric (1st=1) indices of (x y z) columns in DataFile
        x,y,z           Numeric Indexes (as above) as one argument

        DataFile must be a CSV/TSV file with numeric columns
        typically there will be 3 columns (x, y, z) but you can have
        more (or less) and pick any 3 columns using [idx...] numeric
        arguments

        Supported ColorSchemes (prefix-abbreviations work too):
        grey.colors     Grayscale: dark grey .. light grey
        terrain.colors  Terrain/Topographical: Green .. Brown, White
        cm.colors       Light Blue .. Light Pink
        topo.colors     Violet, Blue, Green, Yellow, Brown, White
        brainbow.colors BlueRainbow: Deep Blue .. Bright Red (default)
        heat.colors     Red, Orange, Yellow, White
        rainbow.colors  Rainbow: Deep Violet .. Bright Red (default)
        danger.colors   Green .. Red (sub-slice of rainbow)

Documentation

The full documentation of hm is embedded in the source (as perl POD), to view the full documentation, simply use 'perldoc':

$ perldoc hm

Or just look at the source (scroll down till after the code)

GPL License

hm is copyrighted (c) 2007-2008 by its author, Ariel Faigon, and released under the GNU General Public Licence (GPL). See: www.gnu.org/licenses/gpl.html

Thanks to

Gilbert Leung and Kostas Tsioutsiouliklis for inspiration, initial feedback and ideas for improvement. Brian Gerard for early feedback that made hm much better. Larry Wall and the perl mongers, for perl. The thousands of contributors to GNU-R. Ross Ihaka for the original volcano data-set. ImageMagick Studio LLC for ImageMagick. Linus Torvalds and the thousands of Linux contributors for my productive development environment.

You can have your name added here, just teach me how to do things more efficiently or elegantly with R, send me a patch, etc.

Like it?

This is free software. If you find it useful, or have a patch to make it better, or have any other comment, good or bad, I'd love to hear your input

-- ariel