METECLOUD: A PRIVATE CLOUD PLATFORM FOR METEOROLOGICAL DATA STORAGE USING HADOOP

Publications

Share / Export Citation / Email / Print / Text size:

International Journal on Smart Sensing and Intelligent Systems

Professor Subhas Chandra Mukhopadhyay

Exeley Inc. (New York)

Subject: Computational Science & Engineering , Engineering, Electrical & Electronic

GET ALERTS

eISSN: 1178-5608

DESCRIPTION

0
Reader(s)
0
Visit(s)
0
Comment(s)
0
Share(s)

VOLUME 6 , ISSUE 2 (April 2013) > List of articles

METECLOUD: A PRIVATE CLOUD PLATFORM FOR METEOROLOGICAL DATA STORAGE USING HADOOP

Xue Shengjun / Xu Xiaolong * / Wang Delong / Zhang Jie / Ji Feng

Keywords : MeteCloud, Hadoop, Hive, meteorological data, transfer storage

Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 6, Issue 2, Pages 648-663, DOI: https://doi.org/10.21307/ijssis-2017-559

License : (CC BY-NC-ND 4.0)

Received Date : 26-December-2012 / Accepted: 19-March-2013 / Published Online: 10-April-2013

ARTICLE

ABSTRACT

With the increasing popularity of open-source platform Hadoop, the meteorological industry is available to create a Meteorological Cloud (MeteCloud) platform to store and deploy applications. In this paper, we propose an idea to build the MeteCloud platform for meteorological departments using Hadoop. We also present a backup policy for meteorological data. In addition, one kind of storage process of the meteorological A-format file is presented. Furthermore, we experiment with one-year historical data on the platform by varying many related parameters such as the number of nodes, meteorological records and fields. Finally, the proposed MeteCloud platform proves to be efficient and suitable for the storage of meteorological data.

Content not available PDF Share

FIGURES & TABLES

REFERENCES

[1] M. Armbrust, A. Fox, R. Griffth, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, M. Zaharia, Above the clouds: A berkeleyview of cloud computing. Tech. rep, 2009.
[2] S. Kamara and K. Lauter, “Cryptographic cloud storage”, Proceedings of Financial Cryptography: Workshop on Real-Life Cryptographic Protocols and Standardization 2010, January 2010.
[3] X. Shengjun, Z. Jie, X. Xiaolong, “An improved algorithm based on ACO for cloud service PDTs scheduling”, Advances in Information Sciences and Service Sciences, Vol. 4, no.18, pp. 340-348, 2012.
[4] R. Zhang, C. Zhao, “Automated Deployment Mechanism for Computational Chemistry Services in Cloud”, Advances in Information Sciences and Service Sciences, Vol. 4, no. 3, pp. 83-90, 2012.
[5] R. Grossman, “The case for cloud computing”, IT Professional, Vol.11, no. 2, pp. 23–27, 2009.
[6] X. Zhao, Y. Yang, L. Sun, H. Huang, Metadata-Aware Small Files Storage Architecture on Hadoop, Web Information Systems and Mining, Lecture Notes in Computer Science Vol. 7529, 2012, pp. 136-143
[7] G. Makrai and Z. Prekopcsák, Scaling out data preprocessing with Hive. In Proceedings of the 15th International Student Conference on Electrical Engineering, 2011.
[8] H. Jeong, J. Park, “An Efficient Cloud Storage Model for Cloud Computing Environment”, In Advances in Grid and Pervasive Computing, vol. 7296 pp. 370-376, 2012.
[9] A. Joint, E. Baker, E. Eccles, “Hey, you, get off of that cloud?”, Computer Law & Security Review, Vol. 25, no. 3, pp. 270–274, 2009.
[10] “About AWS and solutions”, what is AWS? (http://aws.amazon.com/what-is-aws/), 2012.
[11] “OpenStack®: the open alternative to cloud lock-in Invented by Rackspace and NASA”, OpenStack | Open cloud Operating System supported by Rackspace (http://www. rackspace.com/ cloud/ openstack/), 2012.
[12] “IBM SmartCloud”, IBM Cloud Computing Overview (http://www.ibm.com/cloud-computing /us/en/?lnk=msoST-ccom-usen), 2012.
[13] “Platform Services”, Oracle | Cloud Computing (http://www.oracle.com/us/solutions/cloud/ over-view/ index.html),2012.
[14] “Welcome to Apache Hadoop”, Hadoop (http://hadoop.apache.org/), 2012.
[15] R. P. Padhy, “Big Data Processing with Hadoop-MapReduce in Cloud Systems”, International Journal of Cloud Computing and Services Science, Vol. 2, no. 1, pp. 16-27, 2013.
[16] J. S. Hare, S. Samangooei and P. H. Lewis, “Practical scalable image analysis and indexing using Hadoop”, Multimedia Tools and Applications, 2012.
[17] Y. Pingle, V. Kohli, S. Kamat and N. Poladia, “Big Data Processing using Apache Hadoop in Cloud System”, National Conference on Emerging Trends in Engineering & Technology, 2012.
[18] A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff and R. Murthy, “Hive: a warehousing solution over a map-reduce framework”, The Proceedings of the VLDB Endowment, Vol. 2, no. 2, pp. 1626-1629, 2009.
[19] N. Pansare and Z. Cai, “Using Hive to perform medium-scale data analysis” (http://www.cs.rice.edu /~np6/Papers/), 2010.
[20] A. Gruenheid, E. Omiecinski and L. Mark, Query optimization using column statistics in hive, IDEAS’ 11 Proceedings of the 15th Symposium on International Database Engineering & Applications, USA 2011, pp. 97-105, 2011.

EXTRA FILES

COMMENTS