write a software 4

The MTBF of a Hadoop node and an enterprise storage

server is three and six years respectively [6], [20]. Based on

these values, Table I compares the probability of data loss per

day in HDFS and NAS filers using Equations 3. We compare

the HDFS cluster with 1000 nodes to a filer with 100 nodes

assuming that they can offer the same storage capacity. This

is valid assumption given recent trends in storage capacity.

Enterprise systems can easily support more than 240 TB of

storage [24], while a typical Hadoop node has 12 TB to 24 TB

of storage.

Filers can offer probability of data loss with one replica of

4.4 ∗ 10−2

, compared to HDFS with 3 replicas, which has a

6.44 ∗ 10−2 probability of data loss. Given the high number

of disks in a single enterprise storage node, fault tolerance

is handled by a RAID controller with a probability of data

loss of 4.4 ∗ 10−3

, and hence in this case we decrease the

replication factor to 1. Disks are arranged in, for example, a

(10, 2) RAID array, which protects from a simultaneous loss

of two disks with ten-fold decrease in storage overhead when

compared to two replicas. Thus, the use of filers as secondary

storage is promising in AptStore and offer a most cost-efficient

yet robust solution

Hadoop Java

Proje NO: #14602887

Proje hakkında