Saturday, July 11, 2015

HDFS Goals


In last tip, we saw Google’s Whitepapers on Big Data.  Apache Hadoop has been originated from Google’s Whitepapers:

  1. Apache HDFS is derived from GFS  (Google File System).
  2. Apache MapReduce is derived from Google MapReduce
  3. Apache HBase is derived from Google BigTable.


This Tip#2, is on HDFS goals & roles in Big Data.  What is HDFS?

HDFS is a distributed and scalable file system designed for storing very large files with streaming data access patterns, running clusters on COMMODITY hardware. HDFS is not a POSIX-compliant filesystem.

In HDFS, each machine in a cluster stores a subset of the data (blocks) that makes up the complete filesystem. Itz metadata is stored on a centralized server, acting as a directory of block data and providing a global picture of the filesystem's state.

Top 5 Goals of HDFS

  1. Store millions of large files, each greater than tens of gigabytes, and filesystem sizes reaching tens of petabytes.
  2. Use a scale-out model based on inexpensive commodity servers with internal JBOD  ("Just a bunch of disks") rather than RAID to achieve large-scale storage. 
  3. Accomplish availability and high throughput through application-level replication of data.
  4. Optimize for large, streaming reads and writes rather than low-latency access to many small files. 
  5. Support the functionality and scale requirements of MapReduce processing.

23 comments:

  1. Nice post,Keep sharing more articles with us.
    thank you,keep updating

    Big data training

    Big data hadoop certification

    ReplyDelete
  2. This is actually good to read the content of this blog. here is very general and the huge knowledgeable platform has been known by this blog. Kindly I share the most useful topic which is important for every user. google play

    ReplyDelete
  3. Thank you very much for great information. It really helpful for me and I am satisfied with the arrangement of your post. I would love to learn more things from your post. Here I have the one of the best thing to share that will be useful for everyone to aware about their health, just visit color vision deficiency and it's facts .

    ReplyDelete
  4. Thanks for sharing your thoughts. I truly appreciate your efforts and I will be waiting for your further post thank you once again. Nice article and pleasant arguments commented here, I am genuinely enjoying by these. I am also share with everyone my work experience to in my blog so visit Gay Test online

    ReplyDelete
  5. Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck for the upcoming articles. If you have time you can check DownloadRobloxPlayer.Exe

    ReplyDelete
  6. Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck for the upcoming articles. If you have time you can check DownloadRobloxPlayer.Exe

    ReplyDelete
  7. Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck for the upcoming articles. If you have time you can check DownloadRobloxPlayer.Exe

    ReplyDelete
  8. Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck for the upcoming articles. If you have time you can check DownloadRobloxPlayer.Exe

    ReplyDelete
  9. I am always searching online for articles that can help me. There is obviously a lot to know about this. I think you made some good points in Features also. Keep working, great job ! also, you can check Are You Gay Test

    ReplyDelete
  10. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don’t know what to say except that I have enjoyed reading. Nice blog, I will keep visiting this blog very often color blindness test.

    ReplyDelete
  11. This blog contains a good information about engineering colleges in mumbai. very helpfull for students. inall a very informative blog. here is a human reaction time test.

    ReplyDelete

  12. Great post! I am actually getting ready to across this information, It's very helpful for this blog. Also great with all of the valuable information you have Keep up the good work you are doing well. click on homosexuality test

    ReplyDelete
  13. Very Nice Blog. Thanks for sharing this information with us. Here i am sharing some information about training institute hopefully helps you out credit card generator.

    ReplyDelete
  14. After looking into a number of the blog articles on your web site, I honestly like your technique of blogging. I saved it to my bookmark site list and will be checking back soon. Please visit my web site as well and let me know how you feel. I am sharing related topic which is mostly important for ultimate reaction time

    ReplyDelete