The purpose of this project is to develop a program to solve the single source shortest distance problem using Spark. solve the single source shortest distance problem using Spark: for each node in a directed graph, calculate the shortest distance from the node with id=0 to this node. The input directed graph is a dataset of edges, where an edge from the node i to the node j is represented in the input text file as:
i,d,j
where d is the distance from node i to node j. (Numbers i,j, and d are long integers.) Let distance[i] be the shortest distance from the node with id=0 to the node with id=i. The pseudo-code to calculate distance[i] is as follows:
distance[0] = 0
for each node i <> 0:
distance[i] = Long.MAX_VALUE
repeat 4 times:
for each edge (i,d,j):
if distance[j] > distance[i]+d
distance[j] = distance[i]+d
Your code that calculates the new distances from the old must be repeated 4 times only.
Hint: You may want to group edges by their destination as a triple (j,distance[j],{(i1,d1),(i2,d2),...}), which are derived from the edges (i1,d1,j),(i2,d2,j),...
, where distance[j] is initially Long.MAX_VALUE.
Dear Customer,
My name is Yuriy Tumakha.
I am interested in your shortest path spark project.
I am Senior Scala/Java Developer with 12 years of experience and strong problem-solving skills.
Relevant Skills and Experience
Scala, Spark
Proposed Milestones
$125 USD - Task 1
$125 USD - Task 2
Hi, I am Amit. I have experience in Spark and Scala. I can implement this in Spark using Scala
Relevant Skills and Experience
Spark Scala
Proposed Milestones
$110 USD - implementation using Spark and Scala
Please share a sample graph with the expected output