A power-law graph as a distributed hash table with quick search and small tables

Hannu Reittu, Illkka Norros

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

We analyze the possibility of using an 'Internet-like' power-law graph as a basis for peer-to-peer distributed hash table applications. Our work is based on previous studies of power-law random graph models that showed emergence of a spontaneous hierarchy of nodes based on their degrees, called the 'soft hierarchy'. The soft hierarchy indicates very short paths, leading to the top of the hierarchy, where the top consists of a clique of highest degree nodes. Such paths have lengths that scale as log log N, with number of nodes N. Further, such paths can be found by a heuristic rule: 'the next hop to highest degree neighbor'. We suggest that these circumstances could be used as a basis of an efficient distributed hash table. The idea is that the hash-entries, needed to locate content, are stored at the periphery of the hierarchy, consisting of large enough set of nodes to guarantee small tables. It is required that any node, say, i in the hierarchy is aware which nodes are below it in the hierarchy, provided it is not in the periphery. The node i places its 'down-hill' neighbors in the hash-ring with equal intervals between them. When node i gets a request to store or search a given hash-entry, it uses some locally defined function that places the hash value on this ring and forwards it to the down-hill neighbor closest to this value. Our main result is a probabilistic estimate of the number of hash-values stored in a periphery node. It appears to be sub log log N and super log log log N, with N -> infinity, and with probability tending to 1. Another contribution is a sketch of a novel algorithm that would create such topologies in a self-organizing manner.
Original languageEnglish
Title of host publication4th International ICST Conference on Performance Evaluation Methodologies and Tools
Number of pages10
DOIs
Publication statusPublished - 2009
MoE publication typeA4 Article in a conference publication
Event4th International ICST Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2009 - Pisa, Italy
Duration: 20 Oct 200922 Oct 2009

Conference

Conference4th International ICST Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2009
Abbreviated titleVALUETOOLS 2009
CountryItaly
CityPisa
Period20/10/0922/10/09

Fingerprint

Topology
Internet

Keywords

  • peer to peer network
  • distributed hash tables
  • power-law graphs

Cite this

Reittu, H., & Norros, I. (2009). A power-law graph as a distributed hash table with quick search and small tables. In 4th International ICST Conference on Performance Evaluation Methodologies and Tools https://doi.org/10.4108/ICST.VALUETOOLS2009.7724
Reittu, Hannu ; Norros, Illkka. / A power-law graph as a distributed hash table with quick search and small tables. 4th International ICST Conference on Performance Evaluation Methodologies and Tools. 2009.
@inproceedings{0bbee37c4590425ca189a3f91a4b52c5,
title = "A power-law graph as a distributed hash table with quick search and small tables",
abstract = "We analyze the possibility of using an 'Internet-like' power-law graph as a basis for peer-to-peer distributed hash table applications. Our work is based on previous studies of power-law random graph models that showed emergence of a spontaneous hierarchy of nodes based on their degrees, called the 'soft hierarchy'. The soft hierarchy indicates very short paths, leading to the top of the hierarchy, where the top consists of a clique of highest degree nodes. Such paths have lengths that scale as log log N, with number of nodes N. Further, such paths can be found by a heuristic rule: 'the next hop to highest degree neighbor'. We suggest that these circumstances could be used as a basis of an efficient distributed hash table. The idea is that the hash-entries, needed to locate content, are stored at the periphery of the hierarchy, consisting of large enough set of nodes to guarantee small tables. It is required that any node, say, i in the hierarchy is aware which nodes are below it in the hierarchy, provided it is not in the periphery. The node i places its 'down-hill' neighbors in the hash-ring with equal intervals between them. When node i gets a request to store or search a given hash-entry, it uses some locally defined function that places the hash value on this ring and forwards it to the down-hill neighbor closest to this value. Our main result is a probabilistic estimate of the number of hash-values stored in a periphery node. It appears to be sub log log N and super log log log N, with N -> infinity, and with probability tending to 1. Another contribution is a sketch of a novel algorithm that would create such topologies in a self-organizing manner.",
keywords = "peer to peer network, distributed hash tables, power-law graphs",
author = "Hannu Reittu and Illkka Norros",
year = "2009",
doi = "10.4108/ICST.VALUETOOLS2009.7724",
language = "English",
booktitle = "4th International ICST Conference on Performance Evaluation Methodologies and Tools",

}

Reittu, H & Norros, I 2009, A power-law graph as a distributed hash table with quick search and small tables. in 4th International ICST Conference on Performance Evaluation Methodologies and Tools. 4th International ICST Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2009, Pisa, Italy, 20/10/09. https://doi.org/10.4108/ICST.VALUETOOLS2009.7724

A power-law graph as a distributed hash table with quick search and small tables. / Reittu, Hannu; Norros, Illkka.

4th International ICST Conference on Performance Evaluation Methodologies and Tools. 2009.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - A power-law graph as a distributed hash table with quick search and small tables

AU - Reittu, Hannu

AU - Norros, Illkka

PY - 2009

Y1 - 2009

N2 - We analyze the possibility of using an 'Internet-like' power-law graph as a basis for peer-to-peer distributed hash table applications. Our work is based on previous studies of power-law random graph models that showed emergence of a spontaneous hierarchy of nodes based on their degrees, called the 'soft hierarchy'. The soft hierarchy indicates very short paths, leading to the top of the hierarchy, where the top consists of a clique of highest degree nodes. Such paths have lengths that scale as log log N, with number of nodes N. Further, such paths can be found by a heuristic rule: 'the next hop to highest degree neighbor'. We suggest that these circumstances could be used as a basis of an efficient distributed hash table. The idea is that the hash-entries, needed to locate content, are stored at the periphery of the hierarchy, consisting of large enough set of nodes to guarantee small tables. It is required that any node, say, i in the hierarchy is aware which nodes are below it in the hierarchy, provided it is not in the periphery. The node i places its 'down-hill' neighbors in the hash-ring with equal intervals between them. When node i gets a request to store or search a given hash-entry, it uses some locally defined function that places the hash value on this ring and forwards it to the down-hill neighbor closest to this value. Our main result is a probabilistic estimate of the number of hash-values stored in a periphery node. It appears to be sub log log N and super log log log N, with N -> infinity, and with probability tending to 1. Another contribution is a sketch of a novel algorithm that would create such topologies in a self-organizing manner.

AB - We analyze the possibility of using an 'Internet-like' power-law graph as a basis for peer-to-peer distributed hash table applications. Our work is based on previous studies of power-law random graph models that showed emergence of a spontaneous hierarchy of nodes based on their degrees, called the 'soft hierarchy'. The soft hierarchy indicates very short paths, leading to the top of the hierarchy, where the top consists of a clique of highest degree nodes. Such paths have lengths that scale as log log N, with number of nodes N. Further, such paths can be found by a heuristic rule: 'the next hop to highest degree neighbor'. We suggest that these circumstances could be used as a basis of an efficient distributed hash table. The idea is that the hash-entries, needed to locate content, are stored at the periphery of the hierarchy, consisting of large enough set of nodes to guarantee small tables. It is required that any node, say, i in the hierarchy is aware which nodes are below it in the hierarchy, provided it is not in the periphery. The node i places its 'down-hill' neighbors in the hash-ring with equal intervals between them. When node i gets a request to store or search a given hash-entry, it uses some locally defined function that places the hash value on this ring and forwards it to the down-hill neighbor closest to this value. Our main result is a probabilistic estimate of the number of hash-values stored in a periphery node. It appears to be sub log log N and super log log log N, with N -> infinity, and with probability tending to 1. Another contribution is a sketch of a novel algorithm that would create such topologies in a self-organizing manner.

KW - peer to peer network

KW - distributed hash tables

KW - power-law graphs

U2 - 10.4108/ICST.VALUETOOLS2009.7724

DO - 10.4108/ICST.VALUETOOLS2009.7724

M3 - Conference article in proceedings

BT - 4th International ICST Conference on Performance Evaluation Methodologies and Tools

ER -

Reittu H, Norros I. A power-law graph as a distributed hash table with quick search and small tables. In 4th International ICST Conference on Performance Evaluation Methodologies and Tools. 2009 https://doi.org/10.4108/ICST.VALUETOOLS2009.7724