Conceptualization.

With regard to how levels of processing were conceptualized in these reviews we found that ten of the reviews explicitly discussed levels of processing, while five did not. Of the five reviews that did not discuss levels of processing (*Afflerbach, Pearson, & Paris, 2008; *Alexander & Judy, 1988; *Ashcraft, 1990; *Paris, 1988; Paris, Lipson, & Wixson, 1983), two of these reviews (*Afflerbach et al., 2008; *Alexander & Judy, 1988) were concerned with the definition of a strategy. For instance, Afflerbach and colleagues addressed the confusion between the terms skill and strategy making the claim that confusion between these two terms could result in less effective instruction for children and adolescents.

Of the reviews that did address levels of processing there were a variety of frameworks from which these levels were addressed. Four of the reviews addressed levels of processing from the perspective of the development of expertise. These perspectives have been forwarded by Alexander and colleagues (Alexander, Grossnickle, Dumas, & Hattan, 2018; *Dinsmore, 2017; *Dinsmore & Alexander, 2012; *Dinsmore, Hattan, & List, 2018). In each of these reviews, conceptualizations of deep- and surface-level processing (strategies to understand the problem versus transforming them respectively) is informed by Alexander’s Model of Domain Learning (MDL; Alexander, 1997, 2004). In the MDL, surface-level strategies are those strategies designed to better understand and solve a problem, whereas deep-level strategies are those strategies designed to transform a particular problem. The MDL predicts that those in acclimation (i.e., novices) would rely primarily on surface-level strategies, whereas experts would increasingly rely on deep-level strategies and less on surface-level strategies (cf. *Dinsmore et al., 2018). Thus, the MDL does not predict that quantity of strategies should relate directly to performance. Rather, the level or type of strategy could be explained by the individual’s development of expertise in a particular domain – such as mathematics – and that that use of the appropriate strategy for that level of expertise should better predict performance in that domain. For example, Dinsmore and Alexander (2016) empirically tested this notion by examining how levels of processing influenced performance on an astronomy task. Those who had low prior knowledge (one of the hallmarks of being a novice) did not perform well using primarily deep-level strategies, whereas those with more prior knowledge performed better on the outcome task using more deep-level strategies. For instance, participants who tried to use elaborative strategies (using one’s own prior knowledge to add information in addition to what the author wrote) while reading the text passage in the study only comprehended that passage better when they possessed higher levels of background knowledge. In other words, in these cases, elaborating on a topic when you have little or no prior knowledge – or worse, inaccurate knowledge – can make comprehension more difficult.

Two reviews relied instead on the Learning Patterns framework (*Asikainen & Gijbels, 2017; *Vermunt & Donche, 2017). The Learning Patterns framework has evolved quite a bit over time (Richardson, 2015) but began with quasi-experimental investigations by Marton and Säljö (1976a, 1976b). These investigations examined the role that expected assessments changed how individuals processed information for studying. For instance, if the task assessment for a text passage was to memorize important details of the text, individuals would be expected to use surface-level strategies such as rehearsal. Those individuals who were asked to interpret what the text meant would be expected to use deep-level strategies such as making inferences about the message the author is trying to convey. Although Marton and Säljö were examining these effects at the task level, much of the current research using SAL examines students’ processing at the course or even semester level. This is evident in *Vermunt & Donche’s (2017) review in which he examined a popular instrument used to measure levels of processing in SAL – the Inventory of Learning Styles (ILS; Vermunt, 1998). It should be noted that the levels in this theory go beyond simply surface and deep level with Biggs (1987) adding an achieving level as well. Similarly, *Asikainen and Gijbels (2017) also examined self-report instrument yet expanded beyond the ILS and included a wider variety of self-report instruments.

Three reviews did not examine levels of processing with regard to deep and surface; rather these reviews examined cognitive versus metacognitive and self-regulatory ­levels (*Alexander, Graham, & Harris, 1998; *Najmaei & Sadeghinejad, 2016; Pintrich, 1999). *Alexander et al. (1998) and *Najmaei and Sadeghinejad (2016) relied primarily on Flavell’s (1976) conceptualization of metacognition to frame the differences between cognitive and metacognitive strategies. Pintrich (1999), on the other hand, used his self-regulated learning (SRL) framework (Pintrich, 2000) which encapsulated cognitive, metacognitive, self-regulatory, and affective strategy use during performance. Although no such current review of SRL strategies exists to our knowledge, the use of SRL to investigate strategy use is typified by the work of Azevedo, Greene, and colleagues (Greene & Azevedo, 2009; Taub, Azevedo, Bouchet, & Khosravifar, 2014). For example, Deekens, Greene, and Lobczowski (2018) used an SRL framework to investigate individuals’ self-regulatory strategy use (and the levels they defined within that framework) across two academic domains – history and science. Differences between the metacognitive and self-regulatory levels are explored in depth in Dinsmore, ­Alexander, and Loughlin’s (2008) systematic review of those constructs.

An outlier to reviews within the three frameworks previously mentioned was *Hattie and Donoghue’s (2016) meta-analysis, which was based on and refined from Hattie’s Visible Learning framework (Hattie, 2008). Hattie’s visible learning is the perspective that students learn best when they become their own teachers – through, among other ideas, better constructed feedback for students to use (e.g., Hattie & Clarke, 2018). Although this framework is not as tightly constructed as the MDL or SAL – it does not contain the specific mechanisms of how deep and surface strategies influence learning – *Hattie and Donoghue (2016) meta-analysis draws primarily from information-processing views of learning (e.g., Klahr & Wallace, 1976).

Operationalization.

Six of the tabled reviews specifically examined the measurement of levels of strategic processing. Two of these reviews were focused solely on retrospective self-report measures (*Asikainen & Gijbels, 2017; *Vermunt & Donche, 2017). Retrospective self-report refers to measures that survey the use of strategies after the task or activity has taken place. As the use of retrospective self-report has been typical in the SAL literature over the past few decades, the prevalence of these retrospective self-reports is not surprising. Given the examination of processing over longer periods of time – such as a course or a semester – the use of retrospective self-reports is easier and less time intensive than some of the concurrent self-report instruments used elsewhere. For example, Vermunt’s ILS (Vermunt, 1998) asks how often students are, “Relating elements of the subject matter to each other and to prior knowledge; structuring these elements into a whole.”

Three of the reviews examined measurement of levels of processing beyond retrospective self-report (*Alexander et al., 2018; *Dinsmore & Alexander, 2012; *Dinsmore et al., 2018). Across these three reviews it is apparent that retrospective self-report remains the dominant measure of strategy use with *Dinsmore and Alexander (2012) reporting that almost half (48%) of the studies they reviewed used ­retrospective self-report, with a higher percentage of retrospective self-report (71%) in their review of studies solely using the MDL. Other methods of measurement included concurrent self-report. Concurrent self-report refers to measurements that collect data about strategic processing during a task, rather than after a task. Concurrent measurements of strategy use were primarily the use of the think-aloud protocol, eye tracking, and neurophysiological measures such as functional magnetic resonance imaging. Think-aloud protocols refer to the process of asking individuals to verbally report their strategy use as they are engaged in a task (cf., Ericsson, 2006; e.g., Parkinson & Dinsmore, 2018). Eye tracking measurements are those that examine how movement of the eye relative to a task (typically a text) relates to their processing (cf. Rayner, Chace, ­Slattery, & Ashby, 2006; e.g., Catrysse et al., 2018). Finally, neurobiological measures, such as functional magnetic resonance imaging (fMRI) or functional near infrared spectroscopy (fNIRS) relates the hemoglobin response (i.e., blood flow) of certain regions of the brain to individuals’ processing (cf. Kotz, 2009; e.g., Dinsmore, Macyczko, Greene, & Hooper, 2019).

Further, one review examined different facets of strategy use and levels of processing more specifically. In his review, *Dinsmore (2017) examined measures of the quantity (i.e., how often a strategy was used), quality (i.e., how well a strategy was used), and conditional use (i.e., when a strategy was used) to investigate whether these measures and the facets of strategy use better related to performance outcomes – a topic discussed in a subsequent section. In this review, Dinsmore found that 94% of the studies contained some measure of quantity, while only 24% and 19% of those studies contained some measure of quality and conditional use respectively. Since most of the studies reviewed were self-report, capturing the quantity of that strategy use is fairly straightforward. However, capturing the quality and conditional use of strategies requires more time and labor-intensive measures, such as think aloud protocols (TAPs).

Discussion.

At issue in the previous two subsections were the conceptualization and operationalization of levels of strategy use. Taken together, findings from these reviews indicate that issues of conceptualization and operationalization have plagued the educational and psychological literature. With regard to the conceptualization of levels of processing, it is clear that how these levels are conceptualized are at worst not explicitly defined (*Dinsmore & Alexander, 2012), and at best researchers in this area have been using competing frameworks with little impetus to collaborate across these frameworks – with some exception (Dinsmore et al., this volume; Gijbels & Fryer, 2017). As Loughlin and Alexander (2012) pointed out, without conceptual clarity, interpreting the findings of these studies – and their accompanying reviews – becomes difficult.

Exacerbating these conceptual issues are measurement issues. The heavy reliance on retrospective self-report has been highly problematic in related areas of the literature such as metacognition and SRL (Dinsmore et al., 2008; Veenman, Van Hout-Wolters, & Afflerbach, 2006). However, the more time and labor-intensive measurements such as TAP are likely not suitable for large, generalizable, longitudinal studies that leverage larger sample sizes over repeated instances across a semester or year of study. Therefore, there has been – and remains – a difficulty with accurately and practically assessing levels of cognitive processing. This issue has left us with either measurements that are quite practical in collecting data from hundreds, even thousands, of students that may or may not accurately reflect their strategic or cognitive processing (i.e., retrospective self-report scales), or measurements that may be more accurate but are difficult to collect and analyze at any large scale.

Discussion.

With regard to the last point – that the link between levels of processing (and strategies more generally) and performance are not of tantamount importance – we disagree. Although we do agree that the ability to engage in self-regulatory strategies is important, this importance is limited if it does not lead to better performance or learning gains. For example, in studies of reading comprehension, there are readers who are termed effortful (Alexander, 2005; Dinsmore, Parkinson, Fox & Bilgili, 2019) who employ many strategies and do quite well in terms of performance outcomes. However, they are very inefficient – their reading times a quite a bit higher than readers in other categories, even those that were deemed to be highly competent readers. Thus, the relation between regulatory competence – which the highly competent readers possess – and reading outcomes is not so straightforward.

Fortunately, evidence supports that strategies – self-regulatory included – should improve performance for a wide range of tasks. Indeed, the MDL in particular proposes that metacognitive and self-regulatory strategies – along with a mix of surface-level and deep-level strategies – are necessary for performance in a domain except for someone who has advanced beyond the novice stage (i.e., in competence or expertise in the MDL). However, we also acknowledge that for some tasks, such as those that are routine or quite straightforward, the degree to which self-regulatory strategies are utilized or required are of much lower import.

Additionally, as a result of these reviews, empirical work that we have conducted, and our practical experience in classrooms with children, adolescents, and adults, we are skeptical of the more targeted view of *Hattie and Donoghue (2016) and *Vermunt and Donche (2017). In the previously mentioned reviews, contextual and individual factors play a key role. We certainly agree with Hattie and Donoghue that some strategies are better than others, but we would posit that the degree to which these strategies are better or worse is far more conditional than they state in their meta-analysis. We dissect this issue now as we turn to the contextual and individual factors that might mediate or moderate the relation between levels of strategy use and performance.

Individual Factors.

Eleven of the reviews in the pool specifically discussed individual factors that might influence the relation between level of processing and performance. The largest category of individual differences discussed across these reviews was motivation. Six of these reviews discussed some facet or multiple facets of motivation. For example, *Vermunt & Donche (2017) discussed goal orientations, attributions, effort, and self-efficacy, while Pintrich (1999) discussed self-efficacy, task value, and goal orientation. In general, across both of these reviews, autonomous motivation and positive conceptions of learning (e.g., higher self-efficacy) are more likely to lead to deeper-level strategies employed.

The next largest category was the discussion of prior knowledge or prior performance. Four reviews specifically addressed the importance of prior knowledge on the use of strategies: *Alexander and Judy (1988), *Alexander et al. (1998), *Dinsmore et al. (2018), and Paris et al. (1983). *Asikainen and Gijbels (2017), *Paris (1988), and *Vermunt and Donche (2017) discussed the role of prior performance – or patterns of performance in the case of Asikainen and Gijbels – in the use of strategies and how that influenced performance. As indicated previously, all these reviews support the notion that higher levels of prior knowledge lead to better deep-level processing.

The next set of factors that were mentioned in a few reviews were of epistemic beliefs and emotions. Two studies, *Alexander Grossnickle, Dumas, and Hattan (2018) and *Vermunt and Donche (2017), both discussed how more sophisticated epistemic beliefs might lead to advantageous differences in strategic processing – particularly the use of deeper-level strategies. *Hattie and Donoghue (2016) focused on a related construct, understanding criteria for success in their review. Finally, *Alexander et al. (2018) discussed learner emotions, while *Vermunt and Donche (2017) also addressed personality, age, and gender in his discussion. For Alexander et al. regulation of learner emotions was considered to be beneficial for strategy use, while for Vermunt and Donche they found that the personality factors of openness and conscientiousness were related to patterns of learning they describe as deep or analytic.

Contextual Factors.

The scope of the reviews with regard to contextual factors was also wide ranging. Some of the reviews focused on specific domains such as reading (*Afflerbach et al., 2008; Paris et al., 1983), mathematics (*Ashcraft, 1990), and business (*Najmaei & Sadeghinejad, 2016). Other reviews, however, focused on how domain-general versus domain-specific investigations of strategic processing influenced performance (*Dinsmore, 2017; *Dinsmore & Alexander, 2012; *Dinsmore et al., 2018; *Vermunt & Donche, 2017). Overall, there seems to be consensus among these reviews that while there are some strategies that can be considered domain general, there is certainly quite a bit of evidence to suggest that being conscious of domain when examining strategic processing is important (e.g., Deekens et al., 2018).

Other reviews focused more on the setting in which these investigations of levels of processing took place. For example, *Asikainen and Gijbels (2017) focused solely on students enrolled in higher education, while *Alexander et al. (2018) addressed strategies that students might employ in online versus face-to-face courses. *Alexander et al. (1998) and *Vermunt and Donche (2017) addressed the role of conducive learning environments regarding levels of processing, which is a major area of research in its own right (cf. Gijbels & Loyens, 2008).

Discussion.

For us, the discussion in these reviews of the individual and contextual factors that influence the levels of strategic processing and its relation to performance provide additional evidence that a less sophisticated model that posits more deep processing will lead to better performance should indeed be a historical notion. The degree to which there are interrelated constructs such as motivation and epistemic beliefs underscore the interconnected relations between levels of strategic processing and performance. Although many of the individual difference factors have been extensively systematically reviewed, the degree to which contextual factors have been reviewed in this regard is much less extensive.