Macadamian Blog

Visual Discovery the #4 Obstacle to BI and Analytics

This is the fourth in a four part series about the Obstacles to BI and Analytics. If you missed them, please read the first two obstacles: #1 Ease of Use   #2Verticalization and   #3 Mobile BI.

Recognized industry analyst Cindi Howson predicts the major analytics trend of 2012 and beyond will be visual discovery. Most solutions offer visual analytics in the form of dashboards, allowing the user to monitor specific metrics and KPIs and take action on them. But visual discovery is about exploring data without a pre-determined goal, digging in to different view—heat maps, spark lines, trellis plots—and visually uncovering insights.

Visual discovery is the holy grail of self-service. Users of all skill levels and across different industries explore views of data and spot patterns, from a government worker digging for ways of optimizing a grant program to a sales manager exploring geographic sales history using a virtual map.

GE Healthcare, a leader in healthcare products, recently revealed a new initiative called SimIndia, a visual discovery tool showing health data across a virtual map of underserved populations in India. The tool would be used by government decision-makers to explore the data and make decisions on how to serve different areas, for example by building a new hospital. Howson lists the leaders in visual discovery as QlikTech, Tableau and TIBCO.

Certainly these companies are ahead of the pack, but the race is only beginning. Visual discovery is a nascent field where any software vendor with the right mix of domain expertise, usability and design skills can innovate.

Interactive Dynamics

Howson provides a detailed list of criteria for evaluating a visual discovery solution. Are advanced visualizations such as trellis plots available? Can multiple data sources be displayed at once and joined? Can the user perform tabular manipulations and aggregations? But the most important criterion is simply “ease of use for the author”, which is listed as “essential, a “show stopper” or feature that significantly affects deployability and/or cost of ownership”. Traditionally, the field of data visualization has focused mainly on the creation of charts and graphs that were easy to interpret. Stephen Few is a thought leader in this area, educating the industry in the best practices of table and graph design and dashboard design.

But in visual discovery, these are table stakes. The new challenge is guiding users through the interactive dynamics of a visual discovery session, in other words, the entire flow and experience from starting the initial views of data, to exploring the data, to annotating and recording the results. Jeffrey Heer from Stanford University and Ben Schneiderman from the University of Maryland have published a paper, “Interactive Dynamics for Visual Analysis,” one of the few publications to date that begins to establish best practices around the overall workflow of a visual discovery session.

The paper essentially divides a visual discovery session into 3 parts: data & view specification, view manipulation, and process & provenance. Each is then further broken down into different usability considerations, from filtering and sorting to navigation and annotation.

Visual Discovery—Open for Design Innovation

Heer and Schneiderman’s paper is remarkable in that it encompasses a framework for the ease of use in visual analysis, but it also reveals many areas that are very new and open to design innovation.

For example, within the Data & View Specification area, the paper recognizes that most design solutions today are still complex to the point that novice users would need help from power users to get started. “Novel interfaces for visualization specification are still needed. A formal grammar that uses graphical marks (rectangles, lines, plotting symbols, etc.) as its basic primitives provides a conceptual model compatible with interactive design tools. New tools requiring little to no programming might place custom visualization design in the hands of a broader audience.”

In the View Manipulation section, the authors point out the potential to significantly improve ways that users could visually select data for deeper investigation: “Designing more expressive selection methods remains an active area of research. For example, researchers have proposed methods to map mouse gestures over a time-series visualization to select perceptually salient data regions such as peaks, valleys, and slopes or to query complex patterns of temporal variation. [...]

Of course, selection need not be limited to the mouse and keyboard: input modalities such as touch, gesture, and speech might enable new, effective forms of selection.” Unlike Mobile BI where the fundamentals of mobile design are established and the challenge is simply finding usability professionals with the right experience, designing for visual discovery is wide open for design research and innovation.

Academic circles and industry thought leaders have begun to establish frameworks and point out areas for further research. The industry is beginning to copy a handful of leaders like QlikTech, Tableau and TIBCO. But when it comes to the interactive dynamics of the entire visual discovery experience, particularly for non-expert users, the area is still wide open for new leaders to make a name for themselves through design innovation.