Interview: Discovering Business Opportunity in Linked Data

Richard Brath & David Jonker, authors of “Graph Analysis and Visualization: Discovering Business Opportunity in Linked Data”, recently sat down with us for an interview. These two experts shared their views on the role of graph visualization, how companies can use it and where it’s going.

David: Richard and I are long-time visualization industry creatives and business partners at Uncharted Software (formerly Oculus Info Inc). Uncharted is in the business of designing and building visual analytics technology for unsolved decision-oriented information problems, in both applied research and commercial contexts. We have the privilege of working behind the scenes for many of the world’s leading companies and government agencies, which is incredibly interesting, and rewarding when you can make a difference in some pretty significant ways. Our DARPA Memex work on technology for fighting human trafficking highlighted in a recent 60 Minutes episode and Wall Street Journal article is a good example of that. As it happens graphs featured heavily there, as they have in a few of the other tools we are actively developing.

Richard: I work with commercial customers, who are facing challenging problems to see correlations in their timeseries data, portfolios and clickstream. Our approach for these complex problems is to design effective high-density visualizations which then enables the viewers to see patterns that are otherwise hard to extract algorithmically. Some of these are graph problems too.

David: The book is really a case for more mainstream use of graphs in helping to guide day-to-day business decisions. People use maps all the time and they are very familiar with reading charts, but ask them about graphs and many won’t even know what they are. Graphs are one of the primary structural forms of visualization but the least understood. We wanted to help change that.

The other gap we wanted to address, and I see this as related to the longstanding disconnect with mainstream business analysts, is a limited awareness of principles and techniques when it comes to effective graph visualization. The bar set by most tools and applications in this regard is very low. I see that as having helped reinforce a relatively common perception that graphs are too abstract to be intuitive. Another of the primary goals of the book is to promote more principled and imaginative thinking about graph visualization problems.

Richard: We’re seeing graph type problems becoming more common in our customers’ businesses. Sometimes it’s a simple graph, such as who communicates with whom in the organization or sometimes much richer graphs with a lot of additional data. Quite often, people in business don’t realize that their specific problem is a graph problem: they’ve seen graphs on the Internet but can’t recognize when the problem or the data is a graph.

One thing we wanted to do with the book is build incrementally so you could start with an overview about what graphs are (chapters 1-2) and learn basic graph and visualization principles using point and click tools for smaller scale graphs (chapters 3-7). In the second half we introduce a bit of light weight programming and move into more specific use cases, some of which may involve a bit of programming (chapters 8-13). We get into graph databases and big data at the end in chapter 14.

Richard: I’ve been interested in graphs since I was a kid. I loved process flow diagrams showing how, say, a lumber mill works. I worked in 3D special effects for movies in the early 90’s and we were doing some really cool visualizations of graph data (subway maps) and I wanted to make it real, not just a special effect.

David: I started working in visualization design back in my university days, and graphs have been a part of that design vocabulary ever since I can remember. More recently I have been part of a DARPA driven mission to dramatically improve visual analytic tools for exploring and understanding big data, and graphs have emerged as one of the key areas of focus there. We have been working on new techniques and technology for exploring very large transactional graphs in both bottom up fashion, starting with nodes and branching out, and in top down fashion, starting with large communities.

Richard: There are so many different types of graph problems and so many insights to be gained that it’s hard to pick one. One that will likely resonate now is the analysis of social networks. Figuring who’s connected to what. Here’s a fairly simple example.

This set of images shows three different product subgraphs from within the same e-mail graph network all using the same fixed underlying layout. The largest three nodes (upper management) are highlighted in each graph to facilitate visual comparison between graphs – you can clearly see different teams around the executives for the different products. The top product (blue links) clearly involves many more people and connections than the other two products.

I like the aphorism success has many fathers but failure is an orphan. In business I see this all the time – lots of people claim responsibility for successful projects and no one claims responsibility for a failure. With a graph you could just plot all the email traffic related to the project and see who really was involved, who showed up late, who sat on the side. So if you’re trying to figure out who really deserves the promotion, look at the graph.

David: One of the most fundamental facets of business that graphs are really good at revealing is influence. Dashboards are great at showing you bottom line metrics of success: am I achieving the things I want to achieve and avoiding the things I want to avoid. Dashboards can give you the who, the what and the when. Graphs can take it a step further and give you the how and the why. If you want to improve an outcome, you really need to understand the dynamics of the factors involved to figure out what actions you can take to influence that outcome. Graphs can represent those relationships for you in order to gain that understanding.

Richard: Right now, you need to be able to recognize when a problem is a graph problem. Then some enthusiasm to tackle it, because right now the tools are still not mature. Maybe point and click will be enough? Maybe a bit of programming? Or perhaps it’s a bit more complex and you need assistance. You need to be able to take that first step and a willingness to explore. The answers may not be immediately obvious and there may be some really useful serendipitous patterns that pop-out too.

David: I have to agree with Richard here. Given how new the techniques and tools are in this area, purpose, vision and some willingness to experiment are often essential to finding the right solution to a problem. Visualization is bound to be an essential component of any solution, since the complex and nuanced relationships expressed by graphs are typically difficult if not impossible to reduce to any other form of representation.

David: With the rise in interest in big data, awareness of the value of graphs in being able to link and derive insights from all of it seems to be rapidly rising. The notion of what constitutes a graph is also more often expanding beyond the typical small scale social network and diagrammatic applications that traditional desktop tools like Analyst Notebook and Palantir were designed for, often involving a great deal of manual maintenance, to very large computationally derived graphs that take advantage of distributed analytics platforms like GraphX. On the research side I am also seeing increasing interest in the time element of graph data, in pursuit of better approaches to understanding dynamics and behaviors in graph data which have often been ignored.

Richard: Graphs are the next level beyond BI and traditional analytics. The last financial crisis that almost took down the entire economy was really a graph problem. BI only showed you that you had a lot of exposure to risk and it was changing fast. The risk, though, was spread out through different customers in different countries and was connected across counterparties through various obligations forming networks of credit. Since BI couldn’t answer the puzzle of those interrelationships everyone just stopped lending money to each other, which froze the financial markets and made things worse. Being able to model, analyze, visualize and assess complex dynamic networks will enable us to work through much more challenging problems in the future.

One of the exciting things about graph analysis and visualization is that the field is rapidly advancing. Today the typical business user may need to put in some effort to identify, collect and organize graph data into files and import data. But with the evolution of graph databases and visual analytic tools that connect directly to these databases, such as Linkurious to Neo4J, it will become much easier to explore and analyze graphs.

Richard Brath and David Jonker’s book “Graph Analysis and Visualization: Discovering Business Opportunity in Linked Data” is now available. Graphs are not mainstream, regular business users lack the concepts and tools to apply them to their challenges. Thus, efforts to educate the public to the potential of graph analysis are very important.

With examples like stock trading, marketing or logistics, Richard Brath and David Jonker show that the opportunities to use graphs are everywhere. With a little training, analysts, software engineers, decision-makers, everyone can identify these opportunities. And now there are tools that make working with graph data simple. Extracting information from complex graphs used to be reserved for PhDs and data scientists. At Linkurious we believe in democratization. That’s why we build tools that can be used by every-day business users to find answers in graph data. Read “Graph Analysis and Visualization: Discovering Business Opportunity in Linked Data”, take the red pill and start using your graph data!