Node2Vec

Overview

Node2Vec is a semi-supervised algorithm designed for feature learning of nodes in graphs, while efficiently preserving their neighborhood structure. It introduces a flexible search strategy that enables exploration of node neighborhoods using both BFS and DFS approaches. Additionally, it extends the Skip-gram model to graphs for training node embeddings. Node2Vec was proposed by A. Grover and J. Leskovec at Stanford University in 2016.

• A. Grover, J. Leskovec, node2vec: Scalable Feature Learning for Networks (2016)

Concepts

Node Similarity

Node2Vec learns a mapping of nodes into a low-dimensional vector space, aiming to ensure that similar nodes in the network exhibit close embeddings in the vector space.

Nodes in a network often alternate between two types of similarities:

1. Homophily

Homophily in networks refers to the tendency of nodes with similar properties, characteristics, or behaviors to be more likely connected or grouped into the same or similar communities (nodes u and s₁ in the graph above belong to the same community).

For example, in social networks, people with similar backgrounds, interests, or opinions are more likely to form connections.

2. Structural Equivalence

Structural equivalence in networks refers to the concept that nodes are considered equivalent if they occupy similar structural roles. This means they share similar patterns of connections to other nodes, also known as similar local topology, regardless of their individual attributes. For example, nodes u and v in the graph above act as hubs within their respective communities, which indicates structural equivalence.

In social networks, structurally equivalent individuals may hold similar roles or positions within their groups, even if they are not directly connected.

Unlike homophily, structural equivalence does not require nodes to be adjacent or close in the network. Nodes can be far apart and still perform the same structural function.

There are two key points to keep in mind when discussing structural equivalence. First, perfect structural equivalence is uncommon in real-world networks, so the focus is often on measuring structural similarity. Second, as the neighborhood range being analyzed increases, the degree of structural similarity between two nodes tends to decrease.

Search Strategies

Generally, there are two extreme search strategies for generating a neighborhood set N_S of k nodes:

• Breadth-first Search (BFS): N_S is restricted to nodes which are immediate neighbors of the start node. E.g., N_S(u) = s₁, s₂, s₃ of size k = 3 in the graph above.
• Depth-first Search (DFS): N_S consists of nodes sequentially searched at increasing distances from the start node. E.g., N_S(u) = s₄, s₅, v of size k = 3 in the graph above.

BFS and DFS strategies play a key role in generating node embeddings that capture either homophily or structural equivalence:

• BFS samples nodes that are close to the starting node, resulting in embeddings that emphasize structural equivalence. This approach provides a detailed, microscopic view of the local neighborhood, which is often sufficient to characterize the local topology.
• DFS explores nodes farther from the starting node, producing embeddings that emphasize homophily. This broader, macro-level view of the neighborhood is useful for capturing community-level patterns and relationships based on shared properties or affiliations.

Node2Vec Framework

1. Node2Vec Walk

Node2Vec employs a biased random walk with the return parameter p and in-out parameter q to guide the walk.

Consider a random walk that has just traversed edge (t,v) and arrived at node v. The next step is determined by the transition probabilities on edges (v,x) originating from v, which are proportional to the edge weights (which are 1 in unweighted graphs). The weights of edges (v,x) are adjusted using parameters p and q based on the shortest distance d_tx between nodes t and x:

• If d_tx = 0, the edge weight is scaled by 1/p. In the provided graph, d_tt = 0. Parameter p influences the inclination to revisit the node just left. When p < 1, backtracking a step becomes more probable; when p > 1, otherwise.
• If d_tx = 1, the edge weight remains unaltered. In the provided graph, d_tx1 = 1.
• If d_tx = 2, the edge weight is scaled by 1/q. In the provided graph, d_tx2 = 2. Parameter q determines whether the walk moves inward (q > 1) or outward (q < 1).

Note that d_tx must be one of {0, 1, 2}.

Through the two parameters, Node2Vec enables control over the trade-off between exploration and exploitation during random walk generation. This flexibility allows the algorithm to learn node representations that span a spectrum—from homophily to structural equivalence.

2. Node Embeddings

The node sequences generated from random walks serve as input to the Skip-gram model. SGD is used to optimize the model's parameters based on the prediction errors, and the model is optimized by techniques such as negative sampling and subsampling.

Considerations

• The Node2Vec algorithm treats all edges as undirected, ignoring their original direction.

Example Graph

Run the following statements on an empty graph to define its structure and insert data:

ALTER EDGE default ADD PROPERTY {
  score float
};
INSERT (A:default {_id: "A"}),
       (B:default {_id: "B"}),
       (C:default {_id: "C"}),
       (D:default {_id: "D"}),
       (E:default {_id: "E"}),
       (F:default {_id: "F"}),
       (G:default {_id: "G"}),
       (H:default {_id: "H"}),
       (I:default {_id: "I"}),
       (J:default {_id: "J"}),
       (K:default {_id: "K"}),
       (A)-[:default {score: 1}]->(B),
       (A)-[:default {score: 3}]->(C),
       (C)-[:default {score: 1.5}]->(D),
       (D)-[:default {score: 2.4}]->(C),
       (D)-[:default {score: 5}]->(F),
       (E)-[:default {score: 2.2}]->(C),
       (E)-[:default {score: 0.6}]->(F),
       (F)-[:default {score: 1.5}]->(G),
       (G)-[:default {score: 2}]->(J),
       (H)-[:default {score: 2.5}]->(G),
       (H)-[:default {score: 1}]->(I),
       (I)-[:default {score: 3.1}]->(I),
       (J)-[:default {score: 2.6}]->(G);

create().edge_property(@default, "score", float);
insert().into(@default).nodes([{_id:"A"},{_id:"B"},{_id:"C"},{_id:"D"},{_id:"E"},{_id:"F"},{_id:"G"},{_id:"H"},{_id:"I"},{_id:"J"},{_id:"K"}]);
insert().into(@default).edges([{_from:"A", _to:"B", score:1}, {_from:"A", _to:"C", score:3}, {_from:"C", _to:"D", score:1.5}, {_from:"D", _to:"C", score:2.4}, {_from:"D", _to:"F", score:5}, {_from:"E", _to:"C", score:2.2}, {_from:"E", _to:"F", score:0.6}, {_from:"F", _to:"G", score:1.5}, {_from:"G", _to:"J", score:2}, {_from:"H", _to:"G", score:2.5}, {_from:"H", _to:"I", score:1}, {_from:"I", _to:"I", score:3.1}, {_from:"J", _to:"G", score:2.6}]);

Creating HDC Graph

To load the entire graph to the HDC server hdc-server-1 as my_hdc_graph:

CREATE HDC GRAPH my_hdc_graph ON "hdc-server-1" OPTIONS {
  nodes: {"*": ["*"]},
  edges: {"*": ["*"]},
  direction: "undirected",
  load_id: true,
  update: "static"
}

hdc.graph.create("my_hdc_graph", {
  nodes: {"*": ["*"]},
  edges: {"*": ["*"]},
  direction: "undirected",
  load_id: true,
  update: "static"
}).to("hdc-server-1")

Parameters

Algorithm name: node2vec

File Writeback

CALL algo.node2vec.write("my_hdc_graph", {
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 5,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}, {
  file: {
    filename: "embeddings"
  }
})

algo(node2vec).params({
  projection: "my_hdc_graph",
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 5,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}).write({
  file:{
    filename: 'embeddings'
  }
})

_id,embedding_result
J,0.0800537,0.0883881,-0.0766052,-0.0655609,0.0273315,
D,0.0604218,0.0188171,0.00422668,-0.0720703,0.0443695,
F,-0.0871277,0.0249908,-0.0150269,-0.0191437,-0.0663147,
H,-0.0376434,0.0515869,0.0605072,0.0593811,0.0319489,
B,0.030896,-0.0760529,-0.0819153,0.0993927,0.0760254,
A,0.0618011,-0.0120789,0.0803131,-0.0098999,0.0146942,
E,-0.00298462,-0.0596649,0.0262451,-0.0267487,-0.0765076,
K,0.0950836,0.0875854,-0.0219025,-0.0045227,0.0101837,
C,-0.0727539,-0.0801422,0.091095,0.00126038,-0.0516479,
I,-0.0608429,-0.0615295,0.0339386,0.00402832,0.0266205,
G,-0.0842712,-0.0761566,-0.0026001,0.0228729,0.0509949,

DB Writeback

Writes the embedding_result values from the results to the specified node property. The property type is float[].

CALL algo.node2vec.write("my_hdc_graph", {
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 5,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}, {
  db: {
    property: "vector"
  }
})

algo(node2vec).params({
  projection: "my_hdc_graph",
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 5,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}).write({
  db: {
      property: "vector"
    }
})

Full Return

CALL algo.node2vec.run("my_hdc_graph", {
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 4,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}) YIELD embeddings
RETURN embeddings

exec{
  algo(node2vec).params({
    return_id_uuid: "id",
    walk_length: 10,
    walk_num: 20,
    p: 0.5,
    q: 1000,
    window_size: 5,
    dimension: 4,
    loop_number: 10,
    learning_rate: 0.01,
    min_learning_rate: 0.0001,
    neg_number: 9,
    resolution: 100,
    sub_sample_alpha: 0.001,
    min_frequency: 3
  }) as embeddings
  return embeddings
} on my_hdc_graph

Result:

Stream Return

CALL algo.node2vec.stream("my_hdc_graph", {
  return_id_uuid: "id",
  walk_length: 10,
  walk_num: 20,
  p: 0.5,
  q: 1000,
  window_size: 5,
  dimension: 5,
  loop_number: 10,
  learning_rate: 0.01,
  min_learning_rate: 0.0001,
  neg_number: 9,
  resolution: 100,
  sub_sample_alpha: 0.001,
  min_frequency: 3
}) YIELD embeddings
RETURN embeddings

exec{
  algo(node2vec).params({
    return_id_uuid: "id",
    walk_length: 10,
    walk_num: 20,
    p: 0.5,
    q: 1000,
    window_size: 5,
    dimension: 5,
    loop_number: 10,
    learning_rate: 0.01,
    min_learning_rate: 0.0001,
    neg_number: 9,
    resolution: 100,
    sub_sample_alpha: 0.001,
    min_frequency: 3
  }) as embeddings
  return embeddings
} on my_hdc_graph

Result: