Survey of Agile and Waterfall Integration in Financial Services - Thesis Proposal Example

SURVEY OF AGILE AND WATERFALL INTEGRATION IN FINANCIAL SERVICES CHAPTER INTRODUCTION Software is a basic part to computer system as they are the main features of programming. Distinctions in the purpose and use of computers systems is often dictated or influenced by the software configuration of the system as different software are required for performing different tasks. It is however worth noting that computer software are dynamic and keeps developing in order to handle complex and more confiscated tasks always. The development of software therefore requires continuous improvement in technology and the processes involved. Computer software developers are therefore individuals who specialize in the knowledge and field of computer software development. They adopt and use strategic systems and processes in developing the software as would be necessary depending on the type and purpose of the software being developed. It is however, worth noting that there is disparities that would be identified in the processes and technologies used in software development over years. For instance, the advancement of technology has seen the evolution of skills and expertise in the field to have advanced the traditional technologies to the more efficient and effective technologies of software development as we have them today. Future projection also points towards improved systems and processes to be devised for the process of software development in the future. It is therefore basic and necessary to be aware that the process of software development is very dynamic and keep evolving. In order to ensure that the continuous changing nature of software development does not result in a haphazard approach to developing software, software developers have often used specific modules of software of development, prominent among which is the use of the systems development life cycle (SDLC). According Larman (2004), system development life cycle is a form of application development life-cycle that is used for achieving integrated purposes in software development, including the purposes of planning, creating, testing and deploying information systems within organizations. SDLC has largely been used in direct association with software development. This notwithstanding, Leffingwell and Widrig (2010), actually emphasized that as a system development model, system development life-cycle is good for hardware configuration, as much as it is good for software configuration. Larman (2004) also wrote about the numerous benefits that SDLC offers software developers, emphasizing that it is integrative, meaning that it guarantees the fulfillment of several aspects of a single software development program. It is not surprising that SDLC is made up of four key phases, each of which seeks to achieve a different purpose for the software developer. The four phases are noted to be the planning, analysis, design, and maintenance phases (Leffingwell and Widrig, 2010). Background The Standish Chaos Report gives sufficient evidence of how software projects have over the years become highly infamous for issues such as lateness, higher cost, lack of originality, and failed functionality (Galorath, 2012). According to a report by Krigsman (2011), an estimated 37% of all the software development and improvement projects by organizations fail. He revealed five main reasons of the failures, which are requirements, resources, schedules, planning, as well as risks. Two clear evidence of such failed software projects are the SAP ERP systems of FoxMeyer Corp and W. W. Grainger Inc (Agile Alliance, 2013). Different reasons have been given as to why such software projects take place. In the case cited, three clear causes to the failures have been named as breakdown in communication, lack of planning and poor quality control (Schuh, 2012). More to the above, it would be noted that right scheduling and planning also play a critical role in ensuring the success of any software development project within an organization. Through planning, software developers are able to identify and manage risks that a software project may face in the course of the project’s development. The necessity of proper planning as well as estimation concerning costs involved in development and improvement of software cannot be over emphasized with the current trends of failures showing that the failures results to losses of billions of money. Galorath (2007) therefore emphasizes on this need in his report where he ascertains that with improved estimation as well as planning, such huge losses could be averted. It was part of the reasons of overcoming the challenges of software development that software architects and developers came up with a systematic and logical process to be used in developing and maintaining software (Leau et al.,2012). The process came to be known at the software development life cycle (SDLC). As explained earlier, the SDLC is very systematic and integrative and thus comes in phases for practical application. These phases are noted to be the planning, analysis, design, and maintenance phases. According to Awad (2005), the planning phase of the SDLC sets off the development process where decisions are made on what will be included in the project and what will not be part of the project. This decision making process is considered very important as it serves as the scheme for the completed project itself. Especially as the Chaos Report blames most of the issues of software failure on cost, timing and functionality, it can be expected that planning is a very important phase of SDLC because it is at this phase that the projections and forecasting of cost, time and functionality are made. Once the projections are miscalculated, it can be expected that the resulting software will not be meet requirements and will be faced with most of the challenges expressed in the Chaos Report (Agile Alliance, 2013). Davis (2012) admonishes the use of requirement analysis at the planning phase to overcome most of the problems faced at later parts of the study. The second phase, which is the analysis phase, builds directly on the planning phase. This is because out of the requirement analysis that is done, system analysis is performed to identify where problems are so that the problems will be fixed at the paperwork level of the project (Davis, 2012). There are so many ways in which system analysis may effectively be carried out. To ensure thoroughness, the need to break down the system into pieces according to different project goals have been recommended (Awad (2005). The design phase of the SDLC comes as the first point where actual construction of the design takes place. In some cases, this phase is differentiated from the implementation phase. Once this is done, the design phase will be placed in between the paperwork stage and the actualization stage. Commenting on the input of the design phase however, Flahiff and Project Management Institute (2011) note that much of the work of the designer is based on concentrating on the construction of screen layouts, process diagrams, business rules and different forms of documentations. For a very successful design phase, it is necessary to ensure that the construction that is done is in line with what has been stated in the approved requirements document. This is because the functionality of the software is dependent on the outcome of this phase (Griffiths, 2012). The fourth phase is the maintenance phase. As it will be elaborated later in the literature review, maintenance is a very important phase that guarantees the durability of the software. Part of the problem being looked at in the study has to do with software development that does not live up to their expected standards and functionality. Meanwhile, Flahiff and Project Management Institute (2011) stated that even if there are inadequacies at the pre-completion stage, maintenance can be used to ensure overhauling of the software so that it functions in a manner that is desirable. While the SDLC provided a logical list various steps and deliverables, there are many different systems development methodologies. The original structured methodology was Waterfall. Each step followed in sequence from one to the next, until verification of the required specifications was achieved (Okoli & Carillo, 2010). The stage wise development was important to the developers because they were able to deal with all bugs as they occurred hence leading to production of high quality software. After the design stage, the implementation stage that purely comprised of coding needed no changes. This is because every bit of the software and its specifications had been identified in the initial stages. Waterfall, however, did not satisfy everyone in handling changes in technology and the ever-growing differences in business needs. For example, waterfall needed to be restarted every time there was a slight change in business or technology. This delayed projects and increased costs as significant rework was needed. Such shortcomings brought forth development of new and more interactive software methodologies. Agile aimed at achieving software agility and adaptability, which was the opposite of the rigid and bureaucratic waterfall development. Agile models comprised of numerous iterations in the schedules that went through the complete SDLC stages of planning, analysis, design, coding and testing. The iterations focused on improving the output with every test made after iteration. The iterations emerged from customer feedbacks that assist in arriving at conclusive solutions. This means that the design was open to change and welcomed all constructive views and criticisms that were necessary for ensuring a successful software development. According to Leau et al. (2012), agile development methods comprise of crystal methodology, featured-driven development, dynamic software development method, scrum, lean software development and extreme programming. All the mentioned methods have five common practices. They all promote interaction rather than documentation, they have frequent but rapid iterations, have frequent short meetings, distributed decision making and they have a close interaction with customers. According to agile proponents, these practices translate to important business benefits. These benefits notwithstanding, there are some few criticisms against the use of the agile software development. For example, Mall (2009) mentioned that agile development is largely inefficient with large organizations, where the developmental approach is sequential. What is more, the agile development has been criticized for being overly dependence on other development models, making it work best in a hybrid situation (Griffiths, 2012). Finally, Schuh (2012) bemoaned the ways agile development emphasize on method against results, creating the likelihood that methods that may not yield desirable results may be used all the time. Which method is more popular? According to Gartner’s IT key metric data, waterfall methods were employed on 52% of development efforts in 2011, with iterative methods used in 29% and agile in 19%. These numbers remained roughly stable during the 2009 through 2011 period, although agile usage increased from 15% to 19% in 2011. Gartner (2014) continues to emphasize that only 56.6% of the projects were completed on time, and only 67% finished on budget is this agile or waterfall. The average schedule variance was 20%, and the average budget variance was approximately 14% (Gartner, 2014). Problem statement From the Chaos Report, Galorath (2012) lamented about the numerous cases and instances of software project failures and how these cost organizations and software developers millions of dollars each year. In a direct relation to this, Mall (2009) also noted that these challenges arise because not much has been done by way of finding the right life-cycle modalities that meet the collective needs of software development. Instead of using integrative life-cycle modalities such as SDLC that ensures the achievement of all key phase of software development, most software developers have only focused on specific phase such as the implementation phase. Once this happens, research into the software projects becomes handicapped and exposing the project to future failures. Enterprise software development involves much more than writing code. It involves activities such as requirements management, software configuration management and release engineering. These functions are grouped under the discipline of the SDLC. The stronger the SDLC process, the higher the success of a software project in meeting time, budget and quality requirements. Sadly, this process is not as simple as it sounds. Using a qualitative research design, this chapter therefore emphasizes on the problem of software project failures and seeks to probe the question of how integrating waterfall and agile methodologies in SDLC can help in solving the continuing problem of software failures. Significance of the study The integration of waterfall with agile practices is worth studying for two reasons. First, the study will help in bridging gaps in literature on the efficacy of a hybrid system, as most literature currently views the hybrid system as a weakness rather than strength for the agile development model. This study will therefore contribute to the literature of compatibility of these methodologies. Secondly, there is a practical professional significance as this study will establish the link between these attributes of software technology as would be used for effectiveness and efficiencies in developing more improved organizational software. This study will also contribute to the existing literature on methodologies used in software development and provide recommendation(s) towards future studies in the theme. Agile technologies as are seen today show advancement although are criticized on equally having specific shortcomings just as is the case with the previous methodologies of waterfall. Historical descriptions are the main characteristic features identified in the waterfall technology literatures while the agile technologies has literature on best practices, case studies as well as on improved concepts. Integration of agile technologies have in a number of instances been identified as having potential in improving the waterfall methodologies used and hence proposed as potential effective solution to the challenges faced in software development. This study is therefore very instrumental in affirming such claims and identifying the particular attributes of agile methodologies that would be adopted and incorporated in the waterfall technologies for effectiveness and efficiency. Besides, by the study adopting the investigative approach, it will test the hypothesis that incorporating agile technologies in the waterfall technologies would produce better results that would be advantageous to trading organizations and the players within the field of software development. The purpose of the study will be therefore directed more on improvement of existing methodologies as against development of new technologies. The main objective and aim for this study is identifying the main attributes of the agile technologies that would be adopted and incorporated in the rather traditional waterfall methodologies in order to produce better and improved technologies to be used in software development for business purposes. This study therefore evaluates the agile practices such as the crystal methodology, the dynamic software development, extreme programming, scrum, failure driven development as well as lean software development. By identifying the capacity to integrate the various technologies in development of a better technology of software development, the study will have contributed greatly towards overall software development expertise and knowledge. Research Question The problem being studied in the research is the poor outcome of software development. As a means to addressing this problem, the following research questions will be considered. 1. In what ways can agile technology attributes be incorporated within the waterfall technology in the development of software? 2. What are the advantages of having a hybrid technology made up of agile technology and waterfall technology in the SDLC? 3. What challenges will the hybrid development approach bring? Methodology To answer the research question, this researcher adopted a qualitative research design. Qualitative data collection and analysis that can be utilized to develop a fuller and richer understanding of this complex, multi-dimensional phenomenon. The main advantage appreciated in the adoption of qualitative methodology in this study is that it incorporates only a few assumptions while compared to quantitative method of analysis. This therefore increases the likelihood of having accurate and more reliable findings from the study. Besides, the method is less time consuming for the reason that little procedures in documentation and analysis are involved. The main steps therefore to be observed within this study are data collection, its analysis as well as the interpretation of the findings. Data collection took place via the use of the phase approach. This means that the researcher divided the data collection process into smaller stages where something unique was done at each stage towards the achievement of the research goal. In the first phase, the researcher conducted a comprehensive review of literature. This means that the first phase was dedicated to secondary data collection. There were specific tasks such as the designing of a literature log. The literature log contained exclusion and inclusion criteria on what should be included in the study and what should be out. All variable that needed to be tested through secondary research were also presented on the literature log. The second phase was the primary data collection phase. The primary data collection was dominated with the development of a survey, where questions pertaining to the SDLC model they use in software development were posed and how respondents envisioned the outcome of a combined model. In the third phase, this researcher distributed the survey to his existing employer: CITI, which is a large US based financial corporation, which is known of incorporating both the agile and the waterfall technologies in developing their software. The organization has numerous over 30 software projects that are underway and hence the choice of using it and the data collected there (Aylward and Clarke, 2005). The survey targeted the project manager and his project team, who total 25. The reason for selecting these people is that they are the ones directly involved in software project development in the organization. Emailed surveys are the main tools of data collection for ease of convenience as well as in order to cut on costs implications that would be resultant in the use of otherwise manual methods of administering the surveys. However, written consent from the respondents in the intentional participation within the study will be mandatory with the criteria of choosing the sample respondents being a mandatory two years of experience with the organizational software development. Limitations One critical limitation that the researcher faced in the course of the study was the fact that respondents were sampled from one organization. Because of this, the generalization of results became very difficult for the researcher. Meanwhile, for a study to be considered as adequately reliable, it is expected that the results can be the same when the study is repeated in another setting with all variables remaining relatively the same. The other limitation was that most of the respondents were not very experienced, as the best of them had only 2 years experience. Summary This chapter was written to give the reader an overview of what is contained in the entire study and how the remaining aspects of the study shall be approached. The reader is therefore expected to remember what is expected in the remaining chapters. The next chapter investigates the past studies and such other study literatures that exist under the theme of the methodologies used in software development. The chapter will therefore contribute immensely in guiding the study through methodologies used in past studies while at the same time providing qualitative information, which would also be used as secondary references. References Aylward D. & Clarke R. (2005). Research models and methodologies. HDR seminar series. Faculty of commerce spring session 2005. Retrieved from:< http://www.uow.edu.au/content/groups/public/@web/@commerce/documents/doc/uow012042.pdf> Galorath D., (2007). “Software Project Failure Costs Billions.. Better Estimation & Planning Can Help” Retrieved from < http://www.galorath.com/wp/software-project-failure-costs-billions-better-estimation-planning-can-help.php> Key J. P., (1997). “Qualitative research” Retrieved from Krigsman M., (2011). “CIO analysis: Why 37 percent of projects fail” Retrieved from Maxwell J. A., (2008). “Designing a Qualitative Study” Retrieved from Read More