How Will Astronomy Archives Survive the Data Tsunami - Case Study Example

?How Will Astronomy Archives Survive the Data Tsunami? Performance Degradation Issues: The astronomical data is usually very large as compared to other data stored in systems. An example of data taking huge memory space is textual data versus graphical data. Graphical data takes more memory resources as the information is huge as compared to simple textual information. Performance is degraded mainly in databases that have a very large dataset to search from. Performance is measured as a time delay from the query given to a system to its response from the same system or disparate systems. Optimization techniques fail to create an efficient programmable dynamic query in case of astronomical data as data is present in bulk and is not usually centrally located. The disparate sources of data induce issue of network and bandwidth as well. When large chunks of data will be downloaded from disparate sources performance will be impacted as connections need to be established and maintained. Data is divided into different sources by a set criteria. This could be arranged date wise or could be based on specific type. The criteria even if incorporated in queries may not help much as the search that runs in dataset is time consuming. If data does not exist in data set time is required to load data from archives and search becomes intensive. There are various techniques used to provide better response times in such scenarios but the ever increasing data of astronomy that is stored in the database requires some special technique for data mining, machine learning and then update in the systems (McLeod & Schell, 2007, p. 145). The requests made for specific data can be optimized by saving results of frequent query. Emerging researches will always produce new datasets which will take standard time. The infrastructure of archival data can be improved however the budget constraint of astronomical research hinders them from such an upgrade at the moment. Incompetent Archival Techniques: Modern day data is stored in a special environment that provides virtual drives hosted by third parties. This allows users to store data from different places online. This technology uses a web service that connects the client with the third party host systems. The customer has to depend on the reliability of the hosting party to provide reliability and security. In order to make systems available on cloud the whole time, third parties usually create redundant data on different virtual machines. If one machine is made unavailable due to a fault or network issue, other machines having the same data will be made available instantly. The astronomical data can be set on cloud to make it readily available. However, cloud computing has its own challenges as well. Such challenges include vulnerability of data attacks as the cloud is shared through a network and makes the data available through internet. The second issue is that of the third party supplier stability. The companies might lose down or merge with other companies resulting is partial or complete loss of data or other similar issues hence decreasing reliability of data on the cloud. There are a number of reasons why such a situation can arise. A supplier can go bankrupt, be bought by other large companies, it may expand and change the direction and interest of their business etc. Performance is again an issue in cloud as it is dependent on the available network bandwidth. Availability is impacted if a software failure occurs, network has a bottleneck or if a hardware fails. The last concern is cost of hosting a cloud platform. The budget constraints for an astronomical foundation may not allocate sufficient finances to meet the target of moving astronomical data on cloud. Emerging Technologies: The technologies currently in practice involve applying indexes on databases. These indexes are stored outside the database and increases the data access or recovery time. This does need advancement to a Peta byte level data management though. It is a challenge today to make efficient algorithms and devices that may reduce overall both financial and computational cost. Reduction in computational cost will guarantee performance improvement. Since astronomical data is not based on textual information alone and its expansion is mainly attributed to its graphical counter parts, so graphical processing units have been of great interest for performance upgrade and reduction in costs. The processors of a graphical processing unit is the most complex of all processors used in any computational system. Mathematics is at the core of a graphical system and a simple graphical image has many complex arithmetic calculations of numbers at its back end. Graphical image processors deal with floating point numbers. Research and successful experiments have been conducted to create a graphical image and process it in just no time to make it readily available for display. These manufacturers claim to have produced a product that is 100 times more efficient as compared to what is available in market today. However there is a short coming in this method. Astronomical data have double precision calculations for accuracy. The graphical processing unis are providing a single precision accuracy. Apart from the precision issue, data coming into graphical processing units and going out of them need considerable space and hit performance. It is proved from research that applications that provide machine learning, volume rendering and simulation with fixed resolution mesh, may prove to be efficient and economical for graphical processing units. The reason behind this is performance enhancement under parallelization through brute force. Code profiling also has a role in it as it may help identify initially which applications re well suited for the graphical processing units. Cloud computing is another hot area of research for astronomical data. It seems to be best suited for data other than graphics due to availability, storage space and especially when performance is not much of an issue (Leung, 2013). The graphical data may require parallel file systems and dedicated bandwidths in order to perform in an acceptable way. Since the astronomical data is very huge, it is not very convenient to use a cloud platform in financial terms. The cost of purchasing a cloud is very expensive as the data keeps on increasing. The renting option does not seem very viable for astronomical data based on the fact that data will be tremendously increasing (Rhea et al., 2001, p. 40-49). The adoption of cloud systems for astronomical data has remained controversial due to its drawbacks. While some companies tradeoff these short comings and accept cloud, others have rejected it and have proposed other secure systems. Proposed Methods: The methods proposed to tackle the data tsunami challenge are at 3 levels: 1) Compute Infrastructure: infrastructures developed for distributed environments may immensely help the astronomical community. The data driven models that focus on optimizing performance of workflow, scheduling of tasks etc need to be formally worked on to make it available for use. 2) Cultural changes are required to incorporate IT related technologies and advancement to handle this data specifically. It is open for research to include a scientific framework for the resolution of this issue. Communities need to come forward to encourage research in this area by publishing journals and leading conferences. People should be given specific recognition for researches conducted in this field so they remain in the field and strive more. Computer science professionals need to team up with the astronomical community to suggest effective methods of archival, retrieval and temporary storage with a tweak of performance. 3) Educational changes are required within astronomical experts to make them capable to suggest efficient models. Currently the models defined are inefficient because proper training was not given to the creators of that model. One such suggestion given in the paper is to include a subject of software engineering for astronomical students and to create a branch that specializes in handling of such issues. They should be taught the basic courses that equip the students well enough to understand the issues at hand, suggest solutions and provide a learning mode to keep enhancing the solutions for optimum results. References Leung, L. (2013, September 9). 4 reasons why cloud and on-premises storage are different, but equally good for people data | Oxygen Cloud's Blog. Retrieved November 19, 2013, from http://blog.oxygencloud.com/2013/09/09/4-reasons-why-cloud-and-on-premises-storage-are-different/ McLeod, R., & Schell, G. P. (2007). Management information systems (p. 145). Upper Saddle River, N.J: Pearson/Prentice Hall. Rhea, S. C., Wells, C., Eaton, P. R., Geels, D., Zhao, B. Y., Weatherspoon, H., & Kubiatowicz, J. (2001). Maintenance-Free Global Data Storage. IEEE Internet Computing, 5(5), 40-49. doi:10.1109/4236.957894 Read More