A new book has been published today: „The nature of problem solving. Using research to inspire 21st centure learning“, edited by Benö Csapó and Joachim Funke. It presents work done in the context of PISA 2012, an activity that I was involved from 2009 until 2014 as Chairman of the International Expert Group on Problem Solving that prepared this enterprise (see my former blog-entry). In his Foreword, Andreas Schleicher (the „father“ of PISA assessments) writes enthusiastically about our book: „… while problem solving is a fairly intuitive and all-pervasive concept, what has been missing so far is a strong conceptual and methodological basis for the definition, operationalisation and measurement of such skills. This book fills that gap. It explores the structure of the problem-solving domain, examines the conceptual underpinning of the PISA assessment of problem solving and studies empirical results. Equally important, it lays out methodological avenues for a deeper analysis of the assessment results, including the study of specific problem-solving strategies through log-file data.“
Here is a summary of the content:
Part I („Problem solving – Overview of the domain“) presents our view of problem solving and its assessment. In Chapter 1, Benö Csapó and Joachim Funke highlight the relevance of problem solving within a world that relies less and less on routine behaviour and increasingly requires non-routine, problem-solving behaviour. They underline the need for innovative assessments to understand how improved school practices contribute to the development of problem-solving skills in learners. In Chapter 2, Jens Fleischer and colleagues use data from PISA 2003 to demonstrate the importance of analytical problem solving as a crosscurricular competency. Starting with the observation that German students’ analytical problem solving skills are above the OECD average but their maths skills are only average, they develop the cognitive potential exploitation hypothesis: good problem-solving skills could be harnessed to develop mathematics skills. In Chapter 3, Magda Osman argues that complex problems can be represented as decision-making tasks under conditions of uncertainty. This shift in emphasis offers a better description of the skills that underpin effective problem solving, with a focus on the subjective judgments people make about the controllability of the problem-solving context. In Chapter 4, John Dossey describes the mathematical perspective and points to the strong relation between mathematics and problem solving. He emphasises the important role of metacognition and self-regulation in both solving real problems and in the practice of mathematics.
Part II („Dynamic problem solving as a new perspective“) introduces dynamic problem solving as a new perspective within PISA 2012. In Chapter 5, Dara Ramalingam and colleagues present the PISA 2012 definition of problem solving, outline the development of the assessment framework and discuss its key organising elements, presenting examples of static and interactive problems from this domain. In Chapter 6, Samuel Greiff and Joachim Funke describe the core concept of interactive problem solving in PISA 2012 as the interplay of knowledge acquisition and knowledge application to reach a given goal state. Interactive problem solving differs from static problem solving because some of the information needed to solve the problem has to be found during interaction processes. In Chapter 7, Andreas Fischer and colleagues describe typical human strategies and shortcomings in coping with complex problems, summarise some of the most influential theories on cognitive aspects of complex problem solving, and present experimental and psychometric research that led to a shift in focus from static to dynamic problem solving.
Part III („Empirical results“) deals with data from different studies presented here because they influenced the thinking around the PISA 2012 problem-solving assessment. In Chapter 8, Gyöngyvér Molnár and colleagues report results from a Hungarian study of students from primary and secondary schools who worked with static and interactive problem scenarios. They found that students’ ability to solve these dynamic problems appears to increase with age and grade level, thus providing evidence that education can influence the development of these skills. At the same time, the measured construct is psychometrically sound and stable between cohorts. In Chapter 9 Ray Philpot and colleagues describe analyses of item characteristics based on the responses of students to PISA 2012 problem solving items. They identified four factors that made problems more or less difficult, which could be helpful for test developers but also researchers and educators interested in developing learners from novice to expert problem-solvers in particular domains. In Chapter 10, Philipp Sonnleitner and colleagues discuss the challenges and opportunities of computer-based complex problem solving in the classroom through the example of Genetics Lab, a newly developed and psychometrically sound computer-based microworld that emphasises usability and acceptance amongst students.
Part IV („New indicators“) presents ideas arising from the new computer-based presentation format used in PISA 2012. In Chapter 11, Nathan Zoanetti and Patrick Griffin explore the use of the data from log files, which were not available from paper-and-pencil tests and offer additional insights into students’ procedures and strategies during their work on a problem. They allow researchers to assess not just the final result of problem solving but also the problem-solving process. In Chapter 12, Krisztina Tóth and colleauges show the long road from log-file data to knowledge about individuals’ problem solving activities. They present examples of the usefulness of clustering and visualising process data. In Chapter 13, David Tobinski and Annemarie Fritz present a new assessment tool EcoSphere, which is a simulation framework for testing and training human behavior in complex systems. Its speciality is the explicit assessment of previously acquired content knowledge.
Part V („Future issues: Collaborative problem solving“) deals with issues that came after PISA 2012. Three years after PISA 2012 focused on individual problemsolving competencies, PISA 2015 moved into the new and innovative domain of collaborative problem solving, defined as “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution.” In Chapter 14, Esther Care and Patrick Griffin present an alternative assessment of collaborative problem solving that inspired the PISA assessment while being clearly distinct from it. They deal with human-tohuman collaboration, in contrast to the human-to-computer design eventually used. In Chapter 15, Arthur Graesser and colleagues, explore the use of “conversational agents” (intelligent computerbased systems that interact with a human) and how dialogues and even trialogues with two agents can be used for new types of assessment.
A final „Epilogue“ from Benö Csapó and Joachim Funke summarizes the work that has been done, reflects terminological issues, and looks into the future.
For me, a long story now comes to a good end! We started the book project 5 years ago. In the meantime, there were quiet phases as well as hectic ones, and until the end of 2016 I was not sure if our book project would come up as a success. Julia Karl and Marion Lammarsch helped us in a critical layout phase with the LaTeX documents – thanks!
Beno has written this sentence in our epilogue: „As we finish the work of editing this book and look back at the process behind that effort, we have the feeling that it has simultaneously been one of the most inspiring experiences and one of the most challenging undertakings of our professional careers.“ Thanks to the OECD staff (Rachel, Sophie, Sylvie) that helped to finalize our projects, thanks to Francesco Avvisati for accompanying our book and thanks to our authors for staying with us the whole time!
Csapó, B., & Funke, J. (eds.) (2017). The nature of problem solving. Using research to inspire 21st century learning. Paris: OECD Publishing. http://doi.org/10.1787/9789264273955-en (or try here)
Here is a nice statement about our book from Dirk van Damme, Head of the Innovation and Measuring Progress Division, Directorate for Education and Skills, OECD.
Here is a Press Release in German: http://www.uni-heidelberg.de/presse/news2017/pm20170607_was-ist-problemloesekompetenz-und-wie-laesst-sie-sich-messen.html