Evaluation of Information System - Essay Example

EVALUATION OF INFORMATION SYSTEMS Introduction Information System (IS) can be described as a grouping of software, hardware, the infrastructure together with trained personnel which are organised with the aim of facilitating, planning, coordination, system control and organisation decision making. Other words it can be defined as the software that assists in organising as well as analysing data which makes it easy to soft problems in an organisation. (Anderson and Aydin, 2005). Whereas evaluation of IS means that methods, diverse approaches and knowledge interest are given priority. It comprises both the invention from evaluation and the process of evaluation from which it follows any differentiation in terms of knowledge interests and evaluation process (Cronholm and Goldkuhl, 2003). Evaluating IS has become very important for organisation because it helps in management structure functions and assessment of the internal operations (AIMTech, 2014). To evaluating IS, some consideration has to be put on the components of IS. This comprises of information which is being fed into the system, processing of the data fed in, output from the system and finally the simple IPO model at the base. With the evaluation of IS special concerned has to be on the effectiveness as well as efficiency of the system, the quality of information being keyed in and the impacts of the information to the individual or an organisation. As evaluations occur in stages, therefore, when and what stage to evaluate must be knew. Whether is at the design stage, maturity or whole life (Anderson et al., 1993). This essay will focus on compare and contrast the contemporary approaches of evaluating IS, identity the challenge and benefits, describe the internal and external factors and trends and advances in IT, and also the recommendation of organisations. Contemporary approaches DeLone and McLean IS success model (D&M Model) The D&M model is an IS theory which tries to give a far reaching understanding of IS accomplishment by distinguishing, depicting, and clarifying the connections among six of the most critical dimensions of success along which IS are commonly evaluated (Petter, 2008). Initial improvement of the theory was attempted by DeLone and McLean in 1992, and was further refined by the first creators after 10 years because of the input got from different researchers working in the region (DeLone and McLean, 2003). The D&M model has been referred to in a huge number of investigative papers, and is thought to be a standout among the most powerful hypotheses in contemporary IS research (Petter, 2008). In D&M model, there are six measurements analysed at separate level, specialised level, semantic level, and viability or impact level. The first measurement of the model is Systems Quality which will be considered over the accomplishment at specialised level. It concentrates on the coveted qualities of the data framework which creates the data. The second measurement is Information Quality which concentrates on the data item and attributes at the semantic level such as the accomplishment of the data in passing on the expected importance (Dwivedi et al., 2012). Moreover, DeLone and Mclean proposed a redesigned model in 2003, which added Service Quality like IS backing as a paramount measurement. Likewise, addition of Intention to Use as an option measure in light of the fact that a disposition is advantageous to device in some setting. Finally, the consolidation of Individual and Organisational Impact to one measurement and named Net Benefits (DeLone and McLean, 2003). Figure 1: Updated DeLone and McLean IS success model, 2003. Exception-Based Approach This approach means IS can be evaluated by focusing in exceptions. It field tested in various industrial organisations over the past eight years and presented in the paper can be used to analyse systems functionality in the user’s level, to compare systems, and to provide factual information on the system and process flaws. As exceptions are undesired events indicating a mismatch between an IS and its area, analysing the number and kind of exceptions provides valuable information which not only about exceptional events, but also about the entire system (Saastamoinen, 2005). For example, chances in the number and kind of certain exceptions are a good measure when the value and benefits of a new system in place are evaluated and the new and old systems are compared. Furthermore, the number and kind of exceptions can also be used to evaluate system adaptation, as the number should decrease and severity diminish when the users learn the new process and system. (Harmon, 2005). Using the number and kind of exceptions for the above mentioned or other purposes is rather an approach than a method. However, practice has proven that certain steps are to be taken for the evaluation to be reliable and to form a solid basis for further development activities (Petter, S. et al. 2008). This approach is not inherently limited to certain kinds or types of systems only, however, for optimal results certain precautions have to be made. These issues need to be noted in advance and also to be able to explain the results. The employees participating in the evaluation have to be well informed (Petter, S. et al. 2008). Unless the IS to be evaluated can be used to collect the data by itself, or there is another system associated with it which can be used for the purpose, the collection of the data relies completely on employees (Ellis, 1983). Interpretative Approach The formal- rational model and interpretative approach can be described as deals with quantitative measures and the possible or achieved cost or benefit (Barnes, 1988). The Interpretative approach uses IS as social systems with Information Technology embedded into systems (Dařena, 2011). There are several strategies used in achieving these approaches. These are the goal based which are driven by the expressed aims of the system in the context (Bosselaers and Preneel, 1995). The Goal free strategy comprises of the inductive examination of systems in place and practice. Finally, is the criteria driven, based on general criteria which are broader that the specific context of the system (Ellis, 1983). Since evaluation occurs in stages, the first stage is the investigative stage which involves investigation that is done on the stated, suggested, unknown benefits and risks. Then analysis is done followed by identification of the benefits or risks to achieve efficiency, effectiveness and understanding (Bellahsène et al., 2002). Challenges and Benefits of Evaluating IS and Approaches Benefits of IS Evaluation. The importance of evaluating IS projects for management decision making purposes is emphasized. It includes that cost benefit analysis of IS projects should be comprehensive, which means based on all factors of potential relevance to the decision-making process and not only based on economic efficiency arguments (Wang, 2011). Since IS evaluation measures the performance. As a system administrator, the measure of system performance through evaluation provides tangible evidence that placement of resources into IS benefits users and organisations (Wiederhold, G. (1992). More importantly, it helps in directing those scarce resources to support IS that work. Moreover, evaluation is also just as useful to determine what doesn't work in a system and provides information users can use to improve the current efforts. Evaluation demonstrates IS benefits to funding sources and to the users (Tandon, R. et al. 2003.). For example, if a violence prevention system works, it necessary to share this success with funding sources, residents, and the community at large (Tandon, R. et al. 2003.). Agencies also often require IS to measure performance or provide information on a system results, service quality, and customer satisfaction (Brusilovsky et al., 2001). IS Evaluation can provide useful information on the impact of the violence prevention of a system to a variety of audiences who are in the position to support the efforts, such as State and local officials, local law enforcement agencies, neighbourhood associations, and community leaders (Noetzel, 1984). Through evaluating the user will obtain the evidences supporting the systems. Then it can be used to solicit funds from other funding sources, to support a request for additional funds to expand the system, or to justify offering the same system in another location (Wiederhold, G. (1992). Information System Evaluation can help improve system's effectiveness. Another benefit of conducting a system evaluation is that the findings will help improve the system. The ability of changes or improvements in the system is directly related to the systems evaluated intervention (Brusilovsky et al., 2001). Moreover, evaluations create an opportunity to share information about what works with similar agencies. For instance, if a system is identified to be effective through evaluation, then sharing this valuable information with other public or users of the system is recommended (Noetzel, 1984). Challenges of IS Evaluation and Approaches. While IS evaluation is crucial and beneficial to information services most organisations, it is the least practiced management tool in majority of organisations (Tandon, R. et al. 2003.)The major observed problems in relation to organisations is lack of awareness which is common in organisations because the issue of IS evaluation is not enough known by people. Furthermore, usually no funds are allocated for this process by the management (Altman and Lyman, 1998). As a management. (Tendon, R. et al. 2003.) e-technique, IS evaluation is an important step in the administration of IS established that lack of finance is another problem. Even if the importance of IS evaluation is recognised, the cost of implementing the exercise is still an impediment. In many organisations, managers of IS do not appreciate the value of IS evaluation (Rosenberg and Roseroka, 2002). Moreover, the shortage of qualified staff is also a notified challenge. In many organisations, lack of qualified staff especially at the senior management level is not unusual. Finally, lack of evaluation tools also cannot be left out because the use of some approaches may not be as viable as the desired results would be (Altman and Lyman, 1998). Internal and external factors and trends and advances in IT Nowadays, IT or IS is not only known as one of the main assets of any organisations, but also considered as tools for effective management of other resources and assets of organisations such as financial resources, human resources and so on. Therefore, evaluating IS to make sure using a right system for organisations is the very important (ATMTech, 2014). Moreover, different types of organisations may have different outcomes to evaluate. For example, a hospital may evaluate a system based on how much time can save per patient, whereas a bank would be concerned with saving or making money. Therefore, when evaluating the IS, whether the model or approach can account for the outcomes need to be considered (Hamilton and Chervany, 1981). However, there has no specific standard of evaluation because both internal and external factors such as social, political and legal issues, user acceptance of IS and so on will influence it. For example, one of the internal factors, user satisfaction, is considered a useful assessment of system effectiveness (Conlin and Stirrat, 2008). It is one of the most frequently used criteria and may be seen as a measure of both system quality and user acceptance. Technology Acceptance Model is usually used to measure the user acceptance. Although the system works as normal, if it is not being used or user feeling bad for the system, then obviously there is something wrong (Davis, 1993). Moreover, one of the external factors is cultural issue. Currently, human behaviour is destroyed by day because of the limitations created by the advancement in technology. Introduction of mobile phones, internet and the likes have determined people to seize meeting up and discuss issue face to face in details. For example, mobile society reduces the need for face-to-face contact. Nowadays, most conversations are concluded online or on the phone. It is a bad behaviour since less conversations expressed are appealing (Conlin and Stirrat, 2008). The growing of using mobile phone leads to a culture disconnect due to class concentration which in turn may lead to failure (Sharples, 2002). The culture of reading and writing letters has also reduced with great margin since emails and mobile texts are used on the daily basis because of the fast delivery and instant answers compared to letters that take time (Wiederhold, G. (1992). It has led to very poor handwriting of many people due to gadget use than pens. However, this does not mean the technology has done a lot of harm to the society, but there should be away of balancing the way for using technology to reduce too much dependency in case when things go wrong (Dwivedi et al., 2012). Current IT Trends and Advances The IS or IT and the application company management have evolved constantly, and these have been continuously adapted to the market requirement and to the new emerging technologies. Initially these programs were designed for running accounting applications, but currently the ERP is used to manage all the information that the company needs, even internally as externally (Davis, 2003). In addition, recently there is a trend to the use of free software, in other words, open and gratuitous code in different scopes (Davis, 2003). This trend is starting to take part in the IS for company management. These days it is possible to find in Spain a software called Openbravo which has these characteristics and it is starting to get an important portion of the market (Openbravo, 2014). However, this trend is still very weak and it seems not to be strong enough. Nevertheless, this tendency must be carefully considered. It is important not to forget the recent closing of Encarta the digital encyclopaedia of Microsoft, due to the appearance of a free software encyclopaedia which won the battle (Conlin and Stirrat, 2008). Moreover, another technical trend, the Hosting of Applications Service Providers, is a common activity nowadays. Since there are several companies which are dedicated to lodge management systems, it can be used by companies through high speed internet or some network connections. (Worthen and Schmitz, 1997). These supplier companies cover both the software and the hardware maintenance, updates, security of the applications. All this allows companies to reduce the work load and some costs from different natures and it also allows firms to focus in the Core Business. For the advances, there has a good example of mobile technology which is currently centred in the sales area. However, the applications of mobile technology need to cover others areas. In other words, it must allow customers to carry out and manage the orders through mobile devices. There are many applications for mobile technologies in the processes of the industrial sector. For instance, sales through m-commerce could be treated as an entry to the IS, what is later managed as has made traditionally with faxes or e-mails, but with the advantages that the mobility offers to customers (Worthen and Schmitz, 1997). The mobility that represents the m-commerce could be extended to others areas in the company, what could generate the logical evolution towards the m-business, such as it happened with the e-commerce and the e-business (Tandon, et al. 2003). Recommendations For all the IS evaluation types, it’s recommend to start with planning the conditions. Planning the conditions for the evaluation means deciding upon the scope, the level of ambition, time, cost and resources. It also includes identifying the assigner, the performer, the data providers and the client of the result (Wiederhold, G. (1992). Planning the conditions also mean to describe the business processes and an overview of the IS functionality. After the conditions are planned, the next step is to describe an overview of the functionality of the IS. It’s suggested that the evaluator walks through the IS in an explorative way and describes its functionality briefly in order to be acquainted with the IS (Tandon, R. et al. 2003). It is important if the evaluator even in these types can describe a user profile since the evaluation of the functionality of the Information System is related to the users pre-knowledge. The users should be described by the users’ tasks, roles, experience, education and maturity. Information about users can be found in job descriptions or by inter viewing managers and/or the system owner (Worthen, B. R. and Schmitz, C. C. 1997). In order to evaluate the IS, the evaluator chooses appropriate criteria. Which criteria and the number of criteria to be chosen depend on the conditions of the evaluation context. Furthermore, the evaluator should approach the IS with an open-mind and with no explicit criteria or business goals that govern the evaluation (Tandon, R. et al. 2003). The users should also be observed when they are working with real tasks in a natural setting since the evaluation can result in four outcomes the functionality supports, the business goals, the functionality counteracts the business goals, there is a missing functionality for a business goal or there is functionality that is not related to any business goal (Wiederhold, G. (1992). Finally, it’s recommended for each evaluation type to end with conclusions. The conclusions should consist of key problems and key strengths. Key problems are identified through a problem analysis. This means that identified problems are related to each other in terms of causes and effects. Conclusion According to the above analysis, it can be find that evaluation of IS is important for organisation to modify the structure or achieve more profits. There is no one size that fits all solution for the evaluation process across different sectors (ATMTech, 2014). Simply imitating the things others do or have done without grasping and understanding the framework within which IS evaluation needs to occur will not give practicable and satisfactory results (Petter, 2008). DeLone and McLean IS success Model is a key approach of evaluating IS. Moreover, there also has no principle to judge the evaluation is correct or not because both internal and external factors will influence it. Reference 1. AIMTech. 2014. Evaluation of information systems. [Online]. [Accessed 6 December 2014]. Available from: http://lubswww.leeds.ac.uk/aimtech/research-areas/evaluation-of-information-systems/ 2. AIMTech. 2014. Information behaviour: Seeking, use and practices. [Online]. [Accessed 6 December 2014]. Available from: http://lubswww.leeds.ac.uk/aimtech/research-areas/information-behaviour/ 3. Anderson, J.G. and Aydin, C. E. 2005. Evaluating the Organisational Impact of Healthcare Information Systems. [Online]. New York: Springer Science + Business Media, Inc. [Accessed 6 December 2014]. Available from: http://link.springer.com/book/10.1007/0-387-30329-4 4. Anderson, J. G. et al. 1993. Evaluating health care information systems: approaches and applications. Thousand Oaks, California: Sage. 5. Altman, D. G. and Lyman, G. H. 1998. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast cancer research and treatment. 52(1-3), pp.289-303. 6. Barnes, G. 1988. A comparative evaluation framework for cadastre-based land information systems (CLIS) in developing countries. University of Wisconsin —Madison. 7. Bellahsène, Z. et al. 2002. Object-oriented information systems 8th international conference, OOIS 2002, Montpellier, France, September 2-5, 2002: proceedings. [Online]. Berlin [etc.]: Springer. [Accessed 6 December 2014]. Available from: http://books.google.com.hk/books?id=3X2_KkH3wmIC&printsec=frontcover&hl=zh-CN#v=onepage&q&f=false 8. Bosselaers, A. and Preneel, B. 1995. Integrity primitives for secure information systems final report of RACE integrity primitive’s evaluation RIPE-RACE 1040. Berlin [etc.]: Springer. 9. Cronholm, S. and Goldkuhl, G. 2003. Strategies of Information Systems Evaluation. Journal of Information Systems Evaluation. 6(2), pp.65-74. 10. Conlin, S. and Stirrat, R. L. 2008. Current challenges in development evaluation. Evaluation. 14(2), pp.193-208. 11. Dařena, F. 2011. Information System Evaluation Criteria Based on Attitudes of Graduate Students. Journal on Efficiency and Responsibility in Education and Science. [Online]. 4(1). pp.46-56. [Accessed 6 December 2014]. Available from: http://www.eriesjournal.com/index.php?idScript=11&idArticle=133 12. Davis, C. K. 2003. Technologies & methodologies for evaluating information technology in business. Hershey, PA: IRM Press. 13. Davis, F. D. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioural impacts. International Journal of Man-Machine Studies. [Online]. 38(3), pp.475-487. [Accessed 6 December 2014]. Available from: http://www.sciencedirect.com/science/article/pii/S0020737383710229 14. DeLone, W. H. and McLean, E. R. 2003. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems. 19(4), pp.9-30. 15. Dwivedi, Y. K. et al. 2012. Information systems theory: Explaining and predicting our digital society. New York: Springer. 16. Goodhue, D. L. 1995. Understanding user evaluations of information systems. Management science. 41(12), pp.1827-1844. 17. Hamilton, S. and Chervany, N. L. 1981. Evaluating Information System Effectiveness - Part I: Comparing Evaluation Approaches. MIS Quarterly. [Online]. 5(3), pp.55-69. [Accessed 6 December 2014]. Available from: http://www.jstor.org/discover/10.2307/249291?uid=3738032&uid=2134&uid=2487570093&uid=2&uid=70&uid=3&uid=60&uid=2487570083&purchase-type=article&accessType=none&sid=21105417013853&showMyJstorPss=false&seq=1&showAccess=false 18. Harmon, K. C. 2005. U.S. Patent No. 6,856,928. Washington, DC: U.S. Patent and Trademark Office. 19. McNurlin, B. C., Sprague, R. H., & Bui, T. X. (1989). Information systems management in practice. Prentice-Hall International. 20. Noetzel, M. J. and Baker, R. P. 1984. Shunt fluid examination: risks and benefits in the evaluation of shunt malfunction and infection. Journal of neurosurgery. 61(2), pp.328-332. 21. Openbravo. 2014. Products. [Online]. [Accessed 6 December 2014]. Available from: http://www.openbravo.com/products/ 22. Petter, S. et al. 2008. Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems. [Online]. 17, pp.236–263. [Accessed 6 December 2014]. Available from:http://www.palgrave-journals.com/ejis/journal/v17/n3/full/ejis200815a.html 23. Saastamoinen, H. 2005. Exception-Based Approach for Information Systems Evaluation: The Method and its Benefits to Information Systems Management. The Electronic Journal of Information Systems Evaluation. 8(1), pp.51-60. 24. Sharples, M. 2002. Disruptive devices: mobile technology for conversational learning. International Journal of Continuing Engineering Education and Life Long Learning. 12(5), pp.504-520 25. Smithson, S. and Hirschheim, R. 1998. Analysing information systems evaluation: another look at an old problem. European Journal of Information Systems. 7(3), pp.158-174. 26. Symons, V. J. 1991. A review of information systems evaluation: content, context and process. European Journal of Information Systems. 1(3), pp.205-212. 27. Tandon, R. et al. 2003. M-Commerce-Issues and Challenges. 28. Wang, J. 2011. Information systems and new applications in the service sector: Models and methods. Hershey: Business Science Reference. 29. Wiederhold, G. (1992). Mediators in the architecture of future information systems. Computer, 25(3), 38-49. 30. Worthen, B. R. and Schmitz, C. C. 1997. Conceptual challenges confronting cluster evaluation. Evaluation. 3(3), pp.300-319. Read More