Share
Embed
Software Dependency Analysis with Graph Databases (Neo4J)
Talk with Philipp Haußleiter at GraphConnect 2015.
Oliver Tigges
May 07, 2015
Transcript
34 slides
-
Agenda Software Dependency Management Repo Census: Use Cases and Demo
Graph Model for Software Dependencies Queries and Cypher -
Declaration of Dependencies <project> <modelVersion>4.0.0</modelVersion> <groupId>com.innoq.census</groupId> <artifactId>census-webapp</artifactId> <version>1.0</version> <name>repo-census-webapp</name> <dependencies>
<dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.1.0</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>org.springframework</artifactId> <version>4.1.3.RELEASE</version> </dependency> <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-core</artifactId> <version>4.3.9.Final</version> </dependency> </dependencies> </project> -
Use Cases Keep overview and be aware of changes Estimate
impacts of software updates License issues Security issues -
System Architecture Akka (Worker) Play2 (Web UI) Query Result neo4j
(GraphDB) Add Artifact Process Artifact Job Request Artifact Job Add Artifact Job Eclipse Aether Client … Process Artifact Request Artifact (Maven) Repository … activeMQ (Queue) -
Transfer to Neo4j Node Elements to handle: Dependencies License Version
GroupId ArtifactId -
Conceptual model <project> <parent> <groupId>com.explodingpixels</groupId> <artifactId>macwidgets-maven-modul</artifactId> <version>0.9.6-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <groupId>com.explodingpixels</groupId>
<artifactId>macwidgets-demo</artifactId> <name>macwidgets-demo</name> <licenses> <license> <name>GNU Lesser General Public License</name> <url>http://www.gnu.org/licenses/lgpl.html</url </license> </licenses> <dependencies> <dependency> <groupId>com.explodingpixels</groupId> <artifactId>macwidgets</artifactId> <version>0.9.6-SNAPSHOT</version> </dependency> </dependencies> </project> -
Search Find by unique version ID MATCH (v:Version {uId: 'com.innoq.census:census-webapp:jar:1.0'})
RETURN v.uId as UID, v:ToDo AS todo, v:Processing AS processing Search with regular expression MATCH (v:Version) WHERE v.uId =~ 'com.innoq.*' RETURN v.uId as UID -
Get license info MATCH (v:Version {uId: {0}})-[:HAS_LICENSE]->(l:LicenseInfo) RETURN l.name AS
lic_name, l.url AS url ORDER BY lic_name -
Show other versions MATCH (v:Version)-[:VERSION_OF]->(ae:ArtifactElement) <-[:VERSION_OF]-(o:Version) WHERE v.uId = {0}
RETURN o.uId AS id ORDER BY id -
Show direct dependencies MATCH (v:Version {uId: {0}}) OPTIONAL MATCH v-[dr:DEPENDS_ON]->other
RETURN other.uId AS dep, dr.scope AS scope ORDER BY dep -
Show transitive dependencies MATCH (v:Version) WHERE v.uId =~ 'com.innoq.*1.0' OPTIONAL
MATCH (v)-[r:DEPENDS_ON_COMPILE|DEPENDS_ON_RUNTIME*1..10]->other WITH distinct other as deps RETURN deps.uId -
Transitive dependency paths MATCH (v:Version) WHERE v.uId =~ 'com.innoq.*1.0' OPTIONAL
MATCH p = (v-[r:DEPENDS_ON_COMPILE|DEPENDS_ON_RUNTIME*1..10]->oth WITH tail ( extract (d IN nodes(p) | d.uId) ) as deps RETURN deps -
License overview of all deps MATCH (v:Version) WHERE v.uId =~
'com.innoq.*1.0' OPTIONAL MATCH v-[r:DEPENDS_ON_COMPILE|DEPENDS_ON_RUNTIME*1..5]->other -[:HAS_LICENSE]->(l:LicenseInfo) RETURN DISTINCT l.name as License, l.url as URL ORDER BY l.name -
Changes between versions MATCH (orig:Version {uId: {0}}), (other:Version {uId: {1}})
OPTIONAL MATCH (orig)-[:DEPENDS_ON]->(common:Version)<-[:DEPENDS_ON]-(other) WITH orig, other, collect(DISTINCT common.uId) AS common OPTIONAL MATCH (orig)-[:DEPENDS_ON]->(origVersion:Version)-[:VERSION_OF]->(commonArtifact:ArtifactElement), (other)-[:DEPENDS_ON]->(otherVersion:Version)-[:VERSION_OF]->(commonArtifact) WITH orig, other, common, collect({origVersion: origVersion.uId, otherVersion: otherVersion.uId}) AS commonArtifacts OPTIONAL MATCH (orig)-[:DEPENDS_ON]->(origOnly:Version)-[:VERSION_OF]->(a:ArtifactElement) WHERE NOT (other)-[:DEPENDS_ON]->(:Version)-[:VERSION_OF]->(a) WITH orig, other, common, commonArtifacts, collect(DISTINCT origOnly.uId) AS origOnly OPTIONAL MATCH (other)-[:DEPENDS_ON]->(otherOnly:Version)-[:VERSION_OF]->(a:ArtifactElement) WHERE NOT (orig)-[:DEPENDS_ON]->(:Version)-[:VERSION_OF]->(a) RETURN common, commonArtifacts, origOnly, collect(DISTINCT otherOnly.uId) AS otherOnly -
Planned features Support for Multiple Repositories Additional Attributes: Organization, Developers,
Properties, Repositories, Plugins Support for (Multi-)User Project Upload (Pom) Social Login (Github/Twitter/...) -
Integration & Cooperation Support other Repositories: Gems, RPMs, NPMs, ...
Integration in build process Integration of jQAssistant -
Availability Project Page How to get, test and contribute SaaS
Appliance (OVA)