Conceptual Gain and Successful Problem-solving in Primary School Mathematics - Case Study Example

 Conceptual Gain and Successful Problem-solving in Primary School Mathematics INTRODUCTION Pamela Davenport and Christine Howe researched the relation between collaborative problem-solving in primary school mathematics and conceptual gain. Their study specifically investigated: i) the effect of collaborative work on mathematical problem-solving among primary school pupils, ii) the extent to which the benefits derived from collaborative work depend on gender and academic abilities of pupils, and iii) the relations between benefits derived from collaborative work and generation of good explanations during group work. The authors used pre- and post-test experiment to investigate the interaction between research variables. Seventy-seven Primary 6 pupils in two schools were sampled to form experimental and control groups. Existing classroom sizes and combinations in two Primary Schools in the same locality were adopted. One of the two parallel classes in the first Primary School (with 27 pupils) was together with another class in the second school (with 25 pupils) assigned as experiment groups. The second class in the first Primary School was used as the only control group. The two experimental groups with 52 pupils were used to test the problem-solving intervention in comparison with 25 pupils in the control group. Pupils in the experiment group were divided, first, into groups of four and, secondly, paired to test the effect of the problem-solving intervention using different collaborative tasks. Materials used for pre- and post-tests comprised contextualized single- and multiple answer problems on addition and subtraction. The problems in the materials contained different topics covered in the Primary 6 textbook and contained in the school curriculum. The groups of four were given different number problems to solve collaboratively and to enable them execute another task of teaching their paired colleague, whose group did not solve the same problem. Instruction sheets containing problem-solving instructions were given to groups of four and pairs, and adults were assigned to each group to assist pupils in clarifying instructions. Pupils in the control group solved the same problems individually. All pupils in the study completed the pre-test. The pre-test was in two stages. First, the whole class attempted problem-solving and, secondly, each pupil was interviewed individually by the authors on the mathematical operations and strategies employed in problem-solving. The problem-solving intervention was administered on the experiment classes in group and pair cycles stretching for a period of three weeks. Two days in a week were devoted to group work and another two for pairs. Pupils in the control group attempted the same problem-solving exercise at the same time. Upon completion of the intervention phase, all pupils completed a post-test using the same procedure as the pre-test except additional question on experiential learning asked when pupils in the experimental group were interviewed. Pupils’ performances during pre- and post-test and their dialogue during group work were measured and coded for analysis. Three different aspects of pupils’ performance were measured. These were: i) the number of problems solve by each pupil (score), ii) the strategy used in solving problems (strategy) and iii) the ‘execution’ performance—appropriateness of calculations and solutions (execution). Pre-test performance coded measures were subtracted from post-test measures to measure changes in the 3 aspects of pupils’ performance. In addition, the pre- and post-test interview responses from 41 subjects with reference to strategy and execution were independently coded by two judges who were blind to conditions to test inter-rata reliabilities. Dialogue during group work was measured and coded by rating individual pupil’s contribution during group work. The authors compared the three main pre-test outcome measures of, first, two experiment classes, and, secondly, same measures of the experiment and control classes. There were no significant differences in both cases. Furthermore, they used ANOVAs to measure effect of condition (experimental and control), gender (male or female) and ability (below average, average and above average) on performance changes (score, strategy and execution) among pupils. The results indicated i) pupils working in group can cognitively gain in mathematical problem-solving than pupils working individually in the traditional method, ii) female and low ability pupils working collaboratively in groups gained more than male and higher ability pupils, iii) pupils’ ability to generate explanations when working collaboratively in group deteriorated. The authors attributed the unanticipated finding (deterioration in execution) to the weakness of their approach which failed to marry conceptual and procedural aspects of mathematical problem-solving. CRITICAL ANALYSIS Theoretical grounding of variables, tools and findings The structure of this study, as an empirical study, was derived from previous researches. Literature reviewed sufficiently defined boundaries of argument. Areas of convergent and divergent findings were highlighted, and the authors structured their research problems according to domain-specific theories. Also the authors successfully established theoretical foundations for the content and structure of the learning intervention and materials used to administer the intervention. The findings of the authors were also theoretically linked to previous works. There findings on the effect of combination of group and teaching in pairs on pupils’ performance were in line with the conclusions of Bargh & Schul (1980) and Thomson (1993). On the effect of gender on pupils’ performance, the authors’ findings stopped at only corroborating previous findings despite the acknowledgement of some inconsistencies. This shortcoming is attributable to inappropriateness of the assignment of participants to groups of four. Gender of participants was not given due consideration. The research was not exploratory but the authors successfully highlighted areas for future research and in some cases even hypothesized research problems. A unresolved issue clearly identified is the “problem of linking conceptual and procedural components of mathematical problem-solving” (73). In all, the article clearly revealed the theoretical plot-lines used by the authors and provided theoretical insight into the researched problem: general approach, intervention, materials used, outcome measures and findings. Research Design The authors used classroom (factorial) experiment design, which involves multiple groups and multiple waves of measurement. The use of experiment design is appropriate for research in a domain-specific learning process like this. Experimentation enabled the authors to establish cause-effect relations between dependent and independent variables. The control group provided the authors reference point for comparing outcome measures of experiment group. As it is well established, experiment research design is the strongest in terms of internal validity. In other words, it brings out the causal effectiveness between research variables. Thus the research design formulated is most appropriate to investigating the effectiveness of new learning forms—collaborative problem-solving involving group work and teaching. However, the choice of two classes in one school and one in another did not sufficiently establish probabilistic equivalence that reduces the effect of extraneous variables such as quality of teachers, institutional settings and the teaching and learning environment in the two schools selected for the research. In fact, the authors used the socio-economic factor to equate the two schools selected. This factor may be strong enough to cause variations in pupils’ academic performance. The authors ought to have selected another school with two parallel Primary 6 classes to approximate natural conditions. By choosing experiment design, the authors inescapably introduced biases due to treatment-attribute and treatment-setting interactions (reactive effects of experimental arrangements). Both biases reduce external validity of findings by increasing the chances of ‘John Henry effect’—pupils performing exceptionally because they are aware their performances will be compared with those of control group. However, experiment design is conjecture-driven. Statistical tests at different levels of analysis can be used to iteratively develop, test, refute, refine, regenerate and/or retest different conjectures. This enhances the explanatory power of the design thus making it very useful in generalizing hypothesized outcomes as well as accounting for ancillary outcomes. Study sample and composition The choice of Primary 6 pupils is justifiably fits this kind of research. Administering treatment used in this study requires good understanding of instructions and problem-solving materials. Primary 6 pupils have the best interpretive and explanatory abilities to read and digest both instructional sheets and problem-solving materials as well as respond to questions asked during interviews. The sample sizes (though note equal) used in the study were appropriate for this kind of study. Classroom experiment design generally provides a good approximation of natural learning setting. Using existing pupils in a class is like randomizing assignment of members of experiment and control groups. Random assignment generally reduces threat to external validity. However, the assignment of one class in one school as against two in the other reduces the external validity of the study. The authors quite acknowledge the fact that “gender differences undoubtedly exist” (58) in academic achievement of pupils working in groups and that previous works have highlighted several issues to be resolved. Yet they failed to use the appropriate sample design to enable them attempt to resolve these issues. The authors noted “[g]ender was not considered in the grouping but it was noted” (61-2). They did not provide sufficient reasons for not given the necessary consideration it deserve, being an important research variable. This sampling shortfall has reduced the theoretical value of their work and has resulted in aberrant findings. Their work has only succeeded in corroborating findings on the gender dimension of the benefits of collaborative work. It failed to essentially, as the authors circuitously set out to, “address some of the inconsistencies which have arisen from the results of previous works carried out in the area” (58). With the appropriate sample that considers qualities of groups such as balanced and unbalanced or same sex and mixed sex, the authors would have inferentially addressed the inconsistencies between McCaskin et al (1984) and Webb’s (1994) findings. Treatment Intervention The treatment intervention used by the authors drew on previous research experiences. The design captured major components of the learning domain, mathematical problem-solving. Procedures for the administration of the treatment intervention are easily comprehensible. Also, the success of this kind of research depends largely on the ability of participants to follow procedures thought out in the design of the treatment intervention. In this case instructional sheets were provided and these materials are digestible for primary school pupils. Instructions contained in the sheet aptly guide participants through the new experiment learning setting. In addition, the authors also contingently used adults to assist participants clarify instructions not well understood. Importantly too, piloting enabled authors to have an insight into participants’ understanding and interpretation of instructional and problem-solving materials as well as those classroom norms that may induce extraneous effects during the experimented learning process. With this, the authors can make desirable adjustments. The problem-solving materials used were appropriate for learning mathematical problem-solving. They were covered in textbooks thus contained in the curriculum. The most basic forms of mathematical problems, addition and subtraction, were used to allow maximum intuitive reasoning and manipulative dexterity considering different academic abilities of participants. The authors contextualized the materials using well registered language to aide students to graspingly attempt problems without wasting time and energy in understanding instructions. Of interest to note, the authors were quick to note the weakness of their intervention approach in facilitating understanding about procedural aspect (in addition to conceptual aspect) of mathematical problem-solving. They did so in their discussion of an unanticipated finding (deterioration in execution of correct calculation). Their acknowledgement of weakness of their approach suggests the use of better approached by future researchers. Data and Data Collection The authors used longitudinal data collected over a period of three weeks. The data was a mixture of quantitative and quantitative data as well as primary and secondary data. Primary data was generated through interviews, review of audio-visuals and observations of pupils’ characteristic interaction. The secondary data used was largely previous literature. The quantitative data used in the analysis was the pupils’ scores in number problem-solving while the qualitative data includes responses to interview questions and observations from recorded audio-visuals. Qualitative data was collected through interview and review of audio-visuals was appropriate. The audio-visuals specifically provided ample data backup, which authors could fall back to in carrying out retrospective analysis to explain unanticipated conjectures. The authors conducted all the nested levels of activities in the collection of the data that requires direct contact with participants and interpretive activities. Research assistants are left with functions (such as recording audio-visuals and transcription) that have no direct effect on responses or reactions of participants. The authors, having designed the research, had the expertise to conduct the research and carry out retrospective analysis at any point. This reduced chances of errors due to lack of strict adherence to protocol and procedures by research assistants, and the possibilities of research assistants’ incorrect recording, fudging and/or unintentional expectancy effects. In essence the reliability and validity of data was enhanced by the research design. Piloting also enabled the authors to perfect techniques and materials thus reducing tendencies for biases and pitfalls in protocols and procedures and errors in data collection. Outcome Measures The outcome measures used by the authors match their research questions and precisely quantified research variables. These measures were readily quantifiable using simply ratings and categorization of responses and observations. The score measure was purely numerical while the strategy, execution and dialogues measures were qualitatively sourced but measured quantitatively. In essences, outcome measures precisely captured research variables and reliably formed basis for statistical analysis. An obvious bias that has affected result is the “research-study design effect”. As noted above, the authors’ quite acknowledged the inadequacies of their approach that might have led to unanticipated outcome in generation of explanation. Data analysis The authors used the pre-test outcomes of two inter-group mean analyses (no significant differences between means of two experiment classes, and between means of experimental classes and the control class) to limit observable changes in outcome measures to the effect of intervention (i.e. precluding difference between experiment groups and control group). The authors thus assumed regression to mean (group means being substantially equal). This assumption is logical in reducing the confounding effect of extraneous variables on dependent variables but it more or less increased the threat of ‘statistical regression’ to internal validity. Nonetheless, the use of ANOVAs to test pre- and post-test differences allowed authors to conveniently statistically test the interaction between multiple factors—condition (experiment or control group), gender (male or female) and ability (above average, average and below average) and subjects’ (pupils’) outcome measures (scores, strategies and execution). This increased the power of test of significance by showing the closeness of test statistic of pre- and post-test variables. This has enabled authors to detect, for instance, a two-way interaction of “strategy change by gender and ability” (65). Also results were presented statistically and substantively in the article, often represented in tables. The authors clearly differentiated where result approach ‘significant main effect’ from where there is ‘significant main effect’ or ‘no main effect”. Where no significant main effect was observed, authors conducted appropriate post-mortem analyses to investigate relations among sub-level variables. Interesting results (two way interactions) were, for instance, found when the authors further treated sub-levels of gender after the initial test statistic revealed no significant main effect of gender on strategy change of pupils. Discussion and conclusion In the discussion of findings, the authors based their line of reasoning on empirical data and linked findings with findings of previous research. They systematically used findings of previous researches to support hypothesized conjectures as well as refined conjectures. Importantly too, they delineated points of departure from findings of previous researches. They also highlighted issues that they were unable to resolve in their studies. Although, experiment research design in generally regarded weak in external validity, the authors used theoretical linking of findings to substantively improve the generalizability of their findings. SUMMARY The design approach was most appropriate for this kind of research, domain-specific education research. The research was theoretically grounded. The authors legitimately answered their research questions. Classroom experiment design allowed authors to approximate natural learning setting and also control the particular learning settings that provided basis for comparing effect of intervention in the learning process. The intervention used was appropriate except its weakness in combining conceptual and procedural aspects of mathematical problem-solving. Multiple data sources provided ample backup for authors to conduct retrospective analyses when explaining refined conjectures. Outcome measures match research questions and provide precisely quantified research variables. The authors’ use of statistical tests (ANOVAs) increased the power of significance test and allowed post-mortem analyses at different levels of analysis. The authors optimized empirical and theoretical data in their discussion of results to logically explain different hypothesized as well as evolving conjectures. In all, the authors successfully underscored causal inference in their study by using the experiment research design. This increased the internal validity of their study. Though experiment design in generally considered weak in external validity, the authors’ randomization of participants into experiment and control groups (by adopting existing classroom sizes, combinations and settings) and the systematic grounding of the findings in theory increased the generalizability of the study. Read More