Manage software dependencies with graph visualization

Discover how Pathfinder helps developers analyse software dependencies using the linkurious.js open-source JS graph visualization library. In this article Michele Sacchetti, Head of Configuration Management and DevOps team at Facilitylive, explains why he built Pathfinder and chose linkurious.js.

Almost every software program has third parties dependencies. As a software project grows with new features, software dependencies grow as well. And with dependencies, the project becomes more complex to manage. Some companies like Facilitylive manage multiple software projects. Here, the code may be split in several modules across different teams, which are sometimes geographically distributed. This makes “legacy” software dependencies and their size an even more serious problem.

For these reasons, developers introducing new software dependencies should always check if they will conflict with existing ones or if a former version of the same library is already in use. Change Request over core legacy code must be deeply analysed to know all the pieces of code impacted by such change. Some teams adopt lifecycle tools like Maven to provide partial control on dependency management, but they lack the overall view on the whole project ecosystem.

Pathfinder helps developers and release managers (or any other role in a DevOps-oriented team) address these kinds of issues in a quick and intuitive way.

Pathfinder integrates in a company Continuous Integration (CI) chain and collects all dependency information at each build, aggregating them in a global view.

Pathfinder is organised into three elements for a flexible integration over different CI architecture:

• A Client, actually in the form of a Maven plugin, collects build data and send it forward to the central server
• A Server, which gather all informations from several clients/builds and aggregates them. Due to its relations-oriented data structure data is stored into an embedded Neo4J server. It provides the performances needed to store an high number of nodes/relations. Communication flow between client-server and server-web interface is exposed via a JSON REST API. All basic actions to manipulate or show graph details can be managed using simple http calls.
• A User interface, in the form of a standalone JavaScript application, which provide end users with an intuitive relational graph with all dependencies information. All graph functionalities are rendered via the linkurious.js graph visualization library which allows a quick and flexible data manipulation.

Clients and server usually work in the background, connected to the CI chain and collecting information and expanding dependency analysis coverage. The user interface provides end-users with tools to analyse the data.

Many developers work on a few projects at once and are not aware of all other teams’ work. To have a full picture of all software dependencies, they should analyze all projects’ candidates which have direct or indirect relations with their work and note down each dependency which may cause conflicts. It is not trivial for a single project but it may become almost impossible to accomplish, and even harder to maintain, over all available dependencies.

Integrating Pathfinder analysis in the CI chain ensure all projects dependencies are gathered and sent to the server to be processed and updated, while the graph visualization interface provides an immediate feedback on many-to-many relations across all possible dependencies.

Moreover agile developers may create additional development branches over the code to isolate their works or to experiment new ideas. This approach gives birth to “branch-packages” containing such experimental code which should be tracked carefully since not production-ready.

In one glance Pathfinder provides useful informations about your project’s dependencies. Nodes color allow users to distinguish types of software dependencies and their status. They can focus their attention on areas which are under development or comes from separate development branches:

• RED artifacts are the SNAPSHOT packages, the ones which need the most care, if you are a developer or a configuration manager. Keeping them under control and being sure your release package doesn’t contain any of them is crucial.
• YELLOW artifacts are the BRANCH packages, which means any stable artifact which doesn’t respect the standard regular expression of Maven versions. They still need some care to make sure they are stable code and not some Release Candidate or even worst, feature branches.
• BLUE artifact are well-know standard release code, you can usually trust them.

The Pathfinder interface built with linkurious.js makes it easier to accomplish tasks like graph filtering, impact analysis or dependency analysis.

Large software projects may contain hundred of dependencies making analysis harder. That’s why Pathfinder provides a filtering interface which allow users to filter nodes based on their Maven attributes.

Both inner and outer nodes can be filtered separately to achieve the best granularity for your analysis.

If your software ecosystem is large enough, you will struggle to evaluate the impact a code modification may have over other teams’ code. Changing a method behavior, even in an apparently insignificant way, can wreak havoc on your company if you are not 100% sure about which other project are invoking such method and why. Just imagine adding a couple of log lines to increase execution feedback, just to discover (usually too late) that your method is used in several time-critical applications where the few milliseconds of delay introduced by printing logs are causing deadlines to be missed. Pathfinder impact analysis helps you answer such question: given the Maven attribute of the artifact you are modifying, it will output the graph of all other artifacts which have a relation with it within a customizable number of “hops”.

In medium/large project different developers will likely re-use the same libraries for similar purposes in different projects. But, since they will probably do so at different times, you can be pretty sure they will include different versions of the same artifact.

Pathfinder can analyze the whole dependency graph of your projects and provide you with a detailed report of duplicated inclusions with version mismatch. The same report can also be converted into Maven dependency management syntax for better centralized inheritance.

Pathfinder adopted the linkurious.js JavaScript graph visualization library as its UI rendering engine from the very first release. As stated above, Neo4j was a natural choice as a backend service to store the data model but even if it also provided an amazing graph browser interface, it was immediately clear that a more fine tuned and customisable graph rendering engine was mandatory to create a rich user experience.

The first version of the Pathfinder graph was directly connected to the Neo4j REST API via native linkurious.js methods. This gave me the possibility to concentrate immediately on the look and feel and usability I had in mind with no additional effort in writing dedicated method.

Then, once the project got its own API implemented, it was really easy to connect them to the already existing UI simply shifting to custom jQuery Ajax call.

Being more of a backend java developer, linkurious.js provided me with the perfect balance of abstraction and customisation flexibility to implement the pathfinder UI in a quick and effective way, keeping also an eye on rendering performances.

Pathfinder is one of the open-source and commercial projects which integrates linkurious.js. Are you trying to help your users understand graph data? Use linkurious.js to quickly build a custom interactive graph visualization interface.