Change Password
  • Content format error The content can not be blank
  • Content format error The content can not be blank
  • Content format error The content can not be blank
The Previous Article:

Neo4j 4.07 Enterprise Edition

vs

Ultipa v2.1

Feature & Benchmark Testing Report

By A Major Commerical & Retail Bank

 

December, 2020

 

           

Table of Contents

Benchmark Testing Environment 3

Overview of Testing Plan. 3

Benchmark Testing Contents. 4

Functional Test 4

Meta-Data Operations. 4

Node-Edge Queries. 4

Node-Edge Insertion. 5

Path Queries. 6

K-hop Neighbor Queries. 8

Graph Algorithms. 10

Degrees. 10

Similarities. 11

Communities and Label Propagations. 12

Functionality & Feature Testing and Comparison. 12

Overall User Experience & Visualization. 13

Usability & Maintainability. 13

Extensibility. 14

Summaries and Findings. 15

Benchmark Testing Environment

  • OS:  RedHat 7.0
  • Hardware/Cloud Setup:
  • CPU: 8-core (Intel Xeon Gold 6148 2.4GHz, cache 28160KB)
  • DRAM: 64GB
  • Disks: 1TB SSD
  • Networks: ACS Network (powered by Microsoft Azure Cloud Services)
  • Cluster Setup: 3-instances
    • Neo4j: Causal cluster
    • Ultipa: HTAP cluster
  • Software:
    • Neo4j 4.07 Enterprise Edition
    • Ultipa Server v2.1
    • Ultipa KG v2.1
    • Docker 19.0+
  • Datasets:
    • Alimama published dataset
    • User transactional behaviour dataset (5.15M vertices, 100M edges)
    • Vertex Entities: users and products.
    • Edge Relationships: Page Views, Add-to-cart, Add-to-favorite, buy (purchase), etc.
    • Average node degree ~= 20 (100M/5M)

Overview of Testing Plan

The testing includes side-by-side feature comparison and performance benchmarking between Neo4j v4.07 Enterprise edition and Ultipa v2.1 edition.

Based on retail banking needs, the testing will be primarily focusing on graph queries and graph computing performances:

  • Meta-data related operations: CRUD operations on nodes and edges;
  • Graph operations: Path, k-hop queries
  • Graph algorithms: Benchmark against a few typical graph algorithms

For non-performance functionality related features, the following areas are tested:

  • Support of visualization (GUI features)
  • Support of high-availability and cluster deployment and management
  • Support of extensibility
  • Support of toolkits/toolchains
  • Overall usability and user-experience.

Both systems are deployed in cluster setup:

  • Neo4j Enterprise v4.07 with 3 instances causal cluster setup
  • Ultipa v2.1 with 3 instances HTAP cluster setup

Note that, the bank’s on-premise cloud environment is supposedly ACS (Advanced Cloud Services) which is based on Microsoft Azure with some special network adaptations, think ACS for the bank’s private cloud as AWS for the public cloud. Due to Neo4j’s incompatibility with ACS, we had to resolve to use bare-metal cluster setup to conduct the following testing.

 

Benchmark Testing Contents

Functional Test

Meta-Data Operations

Node-Edge Queries

Randomized queries against nodes and edges by their IDs. The queries are repeated 1,000 times for each type, total time and average individual query time are counted. If TPS (transactions per second) is counted, it’s also recorded.

Note that all numbers below are in milliseconds (ms), unless otherwise annotated.

Want to read more?
The Next Article: