Node2Vec

Overview

Node2Vec is a semi-supervised algorithm designed for feature learning of nodes in graphs while efficiently preserving their neighborhoods. It introduces a versatile search strategy that can explore both the BFS and DFS neighborhoods of nodes. It also extends the Skip-gram model to graphs for training node embeddings. Node2Vec was developed by A. Grover and J. Leskovec at Stanford University in 2016.

• A. Grover, J. Leskovec, node2vec: Scalable Feature Learning for Networks (2016)

Concepts

Node Similarity

Node2Vec learns a mapping of nodes into a low-dimensional vector space, intending to ensure that similar nodes in the network exhibit close embeddings in the vector space.

Nodes in network often shuttle between two kinds of similarities:

1. Homophily

Homophily in networks refers to the phenomenon that nodes with similar properties, characteristics, or behaviors are more likely to be connected together or belong to the same or similar communities (nodes u and s₁ in the graph above belong to the same community).

For example, in social networks, individuals with similar backgrounds, interests, or opinions are more likely to form connections.

2. Structural Equivalence

Structural equivalence in networks refers to the concept where nodes are considered equivalent based on their structural roles within the network. Nodes that are structurally equivalent have similar connectivity patterns and relationships to other nodes (i.e., the local topology), even if their individual characteristics are different (nodes u and v in the graph above act as hubs of their corresponding communities).

For example, in social networks, individuals that are structurally equivalent might occupy similar positions in their social groups.

Unlike homophily, structural equivalence does not emphasize connectivity; nodes could be far apart in the network and still have the same structural role.

When discussing structural equivalence, it's important to keep in mind two key points: Firstly, achieving complete structural equivalence in a real network is uncommon, leading us to focus on assessing structural similarity instead. Secondly, as the scope of the neighborhood being analyzed expands, the level of structural similarity between the two nodes tends to decrease.

Search Strategies

Generally, there are two extreme search strategies for generating a neighborhood set N_S of k nodes:

• Breadth-first Search (BFS): N_S is restricted to nodes which are immediate neighbors of the start node. E.g., N_S(u) = s₁, s₂, s₃ of size k = 3 in the graph above.
• Depth-first Search (DFS): N_S consists of nodes sequentially searched at increasing distances from the start node. E.g., N_S(u) = s₄, s₅, v of size k = 3 in the graph above.

The BFS and DFS strategies play a key role in producing embeddings that reflect homophily or structural equivalence between nodes:

• The neighborhoods sampled by BFS lead to embeddings that correspond closely to structural equivalence. By restricting search to nearby nodes, BFS obtains a microscopic view of the neighborhood which is often sufficient to characterize the local topology.
• The neighborhoods sampled by DFS lead to embeddings that correspond closely to homophily. By moving further away from the start node, DFS obtains a macro-view of the neighborhood which is essential in inferring node-to-node dependencies exist in a community.

Node2Vec Framework

1. Node2Vec Walk

Node2Vec employs a biased random walk with the return parameter p and in-out parameter q to guide the walk.

Consider the random walk that just traversed edge (t,v) and now arrives at node v, the next step of the walk is determined by the transition probabilities on edges (v,x) originating from v, which are proportional to the edge weights (weights are 1 in unweighted graphs). The weights of edges (v,x) are adjusted by p and q based on the shortest distance d_tx between nodes t and x:

• If d_tx = 0, the edge weight is scaled by 1/p. In the provided graph, d_tt = 0. Parameter p influences the inclination to revisit the node just left. When p < 1, backtracking a step becomes more probable; when p > 1, otherwise.
• If d_tx = 1, the edge weight remains unaltered. In the provided graph, d_tx1 = 1.
• If d_tx = 2, the edge weight is scaled by 1/q. In the provided graph, d_tx2 = 2. Parameter q determines whether the walk moves inward (q > 1) or outward (q < 1).

Note that d_tx must be one of {0, 1, 2}.

Through the two parameters, Node2Vec provides a way of controlling the trade-off between exploration and exploitation during random walk generation, which leads to representations obeying a spectrum of equivalences from homophily to structural equivalence.

2. Node Embeddings

The node sequences obtained from the random walks serve as input to the Skip-gram model. SGD is used to optimize the model's parameters based on the prediction error, and the model is optimized by techniques such as negative sampling and subsampling.

Considerations

• The Node2Vec algorithm ignores the direction of edges but calculates them as undirected edges.

Syntax

• Command：algo(node2vec)
• Parameters:

Example

File Writeback

UQL
algo(node2vec).params({
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  buffer_size: 1000,
  window_size: 5,
  dimension: 20,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}).write({
  file:{
    filename: 'embeddings'
}})

Property Writeback

UQL
algo(node2vec).params({
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  buffer_size: 1000,
  window_size: 5,
  dimension: 20,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}).write({
  db:{
    property: 'vector'
}})

Direct Return

UQL
algo(node2vec).params({
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  buffer_size: 1000,
  window_size: 5,
  dimension: 20,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}) as embeddings
return embeddings

Stream Return

UQL
algo(node2vec).params({
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  buffer_size: 1000,
  window_size: 5,
  dimension: 20,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}).stream() as embeddings
return embeddings