PART V. Statistically Describing and Combining Study Outcomes. Bayesian Meta-Analysis Rebecca M. Turner and Julian P. Higgins pp. Aloe pp. Data Diagnostics. Pigott pp. Publication Bias Jack L. Vevea, Kathleen Coburn, and Alexander Sutton pp.
Data Interpretation. Interpreting Effect Sizes Jeffrey C. Valentine, Ariel M. On the other hand, research synthesists can uncover a wide array of operationalizations of the same construct within the same research area.
The variety of operationalizations that a research synthe- sist may uncover in the literature can be both a curse and. For instance, if we are gathering and integrating studies about how stu- dent perceptions of instructors in high school and college correlate with course achievement we might discover that.
Then, it might not be appropriate for us to label the conceptual variable as student perceptions of instructors. When such. Otherwise, our conclu- sions might appear to apply more generally, to more operations, than warranted by the evidence. We would then face the choice of either broadening the allowable measures of intelligence, and therefore perhaps the.
Thus, it is not unusual for a dialogue to take place be- tween research synthesists and the research literature. The relevance of the. If this test of intelligence produces results different from other tests, then the reason for the discrepancy should be explored as part of the research synthesis, not used as a rationale for the exclusion of MAT studies. Eugene Webb and his colleagues presented strong arguments for the value of multiple operations Multiple operationism has positive consequences because.
If a proposition can. Webb et al. However, multiple operations do not ensure concept-to-operation correspondence if all or most of the operations lack a minimal correspondence to the concept. For example, studies of teacher expectation ef- fects may manipulate expectations in a variety of ways— by providing teachers with information on students in the form of tests scores, by reports from psychologists—and may measure intelligence in a variety of ways.
This means that patterns of irrelevant variation in operations are different for different manipulations of expectations and measures of intelligence.
Thus we are much more likely to begin a search by crossing the keyword teacher expectations with intelligence than we are by crossing Test of Intellectual Growth Potential a made-up test that might have been used to manipulate teacher expectations with Stanford-Binet and Wechsler. It is the abstractness of the keywords that allows unex- pected operationalizations to get caught in the search net.
By extending the keywords even further, searchers may uncover research that has been cast in conceptual frameworks different from their own but which include manipulations or measures relevant to the concepts they have in mind.
Even though these con- cepts are labeled differently the operations used in the. In fact, similar operations generated by different disciplinary traditions often do not share other features of research design—for example, they may draw participants from different populations— and therefore can be used to demonstrate the robustness of results across methodological irrelevancies.
Multiple operations do more than introduce the potential for more robust inferences about relationships between conceptual variables. They are an important source of variability in the conclusions of different syntheses meant to address the same topic.
A variety of operations can af- fect synthesis outcomes in two ways. Some synthesists pay careful attention to study opera- tions. They identify meticulously the operational distinc- tions among retrieved studies and test for whether results are robust across these variations.
Other synthesists pay less attention to these details. I pointed out that entering da- tabases with abstract terms as keywords will permit the serendipitous discovery of aligned areas of research, more so than will entering with operational descriptions. Now I. While reading study abstracts, they should use the tag potentially relevant as liberally as possible. However, in the early stages, when conceptual and operational boundaries may still be a bit fuzzy, the synthesist should err on the overly inclusive side, just as primary researchers collect some data that might not later be used in analysis.
Had we started the search with IQ, we might never have known that others considered the SAT and GRE tests of intelligence in teacher expectation research. Third, a broad conceptual search allows the synthesis to be carried out with greater operational detail.
For ex- ample, searching for and including IQ tests only permits us to examine variations in operations such as whether the test was group versus individually administered and timed versus untimed. If so, are there plausible explanations for this?
What does it mean for the. Often, these analyses produce the most interesting results in a research synthesis. Does the problem relate simply to the prevalence or level of a phenomenon in a population? Does it suggest that the variables of interest relate as a simple association or as a causal connection? Does the problem or hypothe- sis refer to a process that operates within an individual unit 1 —how it changes over time—or to general tenden- cies between groups of units?
The distinctions embodied in these questions have crit- ical implications for the choice of appropriate research designs. I assume the reader is familiar with research de- signs and their implications for drawing inferences. Still, a brief discussion of the critical features of research prob- lems as they relate to research design is needed in order to understand the role of design variations in research synthesis. Researchers need to answer three questions about their research problems or hypotheses to be able to determine the appropriateness of different research designs for ad- dressing them for a fuller treatment of these questions, see Cooper :.
However, the gathering and integration of narrative, or qualitative, research is an important area of methodology and scholarship. The reader is referred to. Such problems typically ask what is happening. Thus, we might ask. From this, we might draw an inference about how well-liked instructors generally are on college campuses.
A second type of problem might be what events hap- pen together. Here, researchers ask whether characteris- tics of events or phenomena co-occur with one another. Then, the two variables could be correlated to answer the question of.
The third research problem seeks an explanation for an event. In this case, a study is conducted to isolate and draw a direct productive link between one event the cause and another the effect. In our example, we might ask whether increasing the liking students have for their instructors causes them to get higher grades in class.
Three classes of research designs are used most often. It takes simple correlational research a step further by examining co-occurrence in a multivari- ate framework. For example, if we wish to know whether liking an instructor causes students to get higher grades, we might construct a multiple regression equation or structural equation model that attempts to provide an exhaustive description of a network of relational link- ages Kline This model would attempt to account for, or rule out, all other co-occurring phenomena that might explain away the relationship of interest.
Studies that compare groups of partici- pants to one another—such as men and women or differ- ent ethnic groups—and control for other co-occurring. The second class involves quasi-experimental research Shadish, Cook, and Campbell Here, the research- ers or some other external agent control the introduc- tion of an event, often called an intervention in applied research or a manipulation in basic research, but they do not control precisely who may be exposed to it.
Instead, the researchers use some statistical control in an attempt to equate the groups receiving and not receiving the inter- vention. For example, we might be able to train college instructors to teach classes using either a friendly or a distant instructional approach. However, we might not be able to assign students to classes so we might match students in each class on prior grades and ignore students who did not have a good match. Finally, in experimental research both the introduction of the event for example, instructional approach and who is exposed to it the students in each class are con- trolled by the researcher or other external agent.
The re- searcher uses a random procedure to assign students to conditions, leaving the assignment to chance Boruch Of course, there are numerous other aspects of the design that must be attended to for this strong inference to be made—for example, standardizing testing conditions across instructors and ensuring that instructors are un- aware of the experimental hypothesis. For our purposes, however, focusing on the characteristics of researcher control over the conditions of the experiment and assign- ment of participants to conditions captures what is needed to proceed with the discussion.
Other research problems relate to the average differences in and variability of a characteristic between groups of units. This latter problem is best addressed using the designs just discussed. The former problem—the problem of change within a unit— would best be studied using the various forms of time se- ries designs, a research design in which units are tested at different times, typically equal intervals, during the course of a study.
For example, we might ask a student in. A concomitant time series design would then reveal the association between change in liking and homework grades. Or, the change from distant to friendly instruction might involve having the instructor add a different friendly teaching technique with each class session, for example, leaving time for questions during session 2, adding re- vealing personal information during session 3, adding telling jokes during session 4.
Of course, for this type of design to allow strong causal inferences other experimental controls would be needed— for example, the random withdrawal and reintroduction of teaching techniques. How- ever, because different students might have found differ- ent approaches appealing, averaged across the students in the class, the friendliness manipulations might be causing gradual improvements in the average homework grade.
So, the group- averaged data can be used to explain improvement in grades only at the group level; it is not descriptive of the process happening at the individual level, nor visa versa. It is important to note that the designation of whether an individual or group effect is of interest depends on the research question being asked. So, the group performance. Thus, the key to proper inference is that the unit of analysis at which the data from the study is being analyzed corresponds to the unit of interest in the problem or hypothesis motivating the primary research or research synthesis.
If not, an erroneous conclusion can be drawn. Floyd Fowler suggests that two types of descriptive questions are most appropriately answered using quantitative ap- proaches For example, we might want to know how many college students receive A grades in their classes.
The second descriptive question involves collecting information about attitudes, opinions, or per- ceptions. For example, our synthesis concerning liking of instructors might be used to answer how well on average college students like their instructors.
Gathering and integrating research on the question in- volving frequency of grades would lead us to collect from each relevant study the number of students receiving A grades and the total number of students in each study. The question regarding opinions would lead us to collect the average response of students on a question about liking their instructor.
The primary objective of the studies this data might be collected from might not have been to answer our descriptive questions. This evidence could have been re- ported as part of a study examining the two-variable as- sociation between liking of the instructor and grades.
However, it is not unusual for such studies to report, along with the correlation of interest, the distribution of grades in the classes under study and the average liking of the instructors.
But there are still two com- plications. If instructors use different grading schemes, getting an A might mean very different things in different classes and different studies. Of course, if different studies report dif- ferent frequencies of A grades, we can explore whether other features of studies for example, the year in which the study was conducted covary with the grading curve used in their participating classrooms.
Second, it might be that the aggregation of frequencies across studies is undertaken to estimate the frequency of an event in a population. So, our motivating question might be what the frequency is that the grade of A is as- signed in American college classes.
The value of our ag- gregate estimate from studies will then depend on how well the studies represented American college classes. It would be rare for classes chosen because they are conve- nient to be fortuitously representative of the nation as a whole. But, it might be possible to apply some sampling weights that would improve our approximation of a pop- ulation value. Aggregating attitudes, opinions, and perceptions across studies is even more challenging.
Suppose we wanted to aggregate across stud- ies the reported levels of liking of the instructor. Scales with the same numeric gradations might also differ in the anchors they used for the liking dimen- sion.
Clearly, we cannot simply average these measures. One solution would be to aggregate results only across studies using the identical scales and than we report results separately for each scale type. Another solution would be to report simply the percentage of respondents using one side of the scale or the other.
The num- ber can be derived from the raw rating frequencies, if they are given, or from the mean and variance of the ratings for an example, see Cooper et al.
It is rare to see research syntheses in social science that seek to aggregate evidence across studies to answer de- scriptive questions quantitatively. When this occurs, how- ever, it is critical that the synthesists pay careful attention to whether the measures being aggregated are commen- surate.
Combining incommensurate measures will result in gibberish. In primary research, a researcher selects the most appro- priate research design for investigating the problem or hypothesis.
It is rare for a single study to implement more than one design to answer the same research question, al- though there may be multiple operationalizations of the constructs of interest. More typical would be instances in which primary researchers intended to carry out an opti- mal design for investigating their problem but, due to un- foreseen circumstances, ended up implementing a less- than-optimal design. By the end of the semester, it is clear that the students remaining in the sections were not equivalent, on average, when the study began.
To rescue the effort, we might use post hoc matching or statistical controls to. Thus, the experiment has become a quasi-experiment. So, a study search- ing for a causal mechanism has become a study of associ- ation. The legitimate inferences allowed by the study began as strong causal inference, degraded to weak causal inference, and ended as simple association. In contrast, research synthesists are likely to come across a wide variety of research designs that relate the concepts of interest to one another.
A search for stud- ies using the term instructor likeability and examining them for measures of achievement should identify studies that resulted in simple correlations, multiple regressions, perhaps a few structural equation models, some quasi- experiments, a few experiments, and maybe even some time series involving particular students or class averages.
What should the synthesists do with this variety of re- search designs? Thus, it seems that we would be well advised to include all the research designs in our synthesis of evi- dence on association. The issue is more complex when our research question deals with causality: Does increasing the liking students have for their instructors cause higher student grades?
Here, the different designs produce evidence with differ- ent capabilities for drawing strong inferences about our problem. As such, if this were the only research design found in the literature, it would be appropriate for us to assert that the question remained untested. When an association is found, multiple regressions statistically control for some alternative explanations for the relationship, but probably not all.
Structural equation models relate to the. Well-conducted quasi-experiments may permit weak causal inferences, made stronger through multiple and varied replications. Experiments using random assign- ment of participants to groups permit the strongest infer- ences about causality.
How should synthesists interested in problems of cau- sality treat the various designs? At one extreme, they can discard all studies but those using true experimental de- signs. This approach applies the logic that these are the only studies that directly test the question of interest.
All other designs either address association only or do not permit strong inferences. The other extreme would be to include all the research evidence while carefully qualify- ing inferences as the evidence for causality moves farther from the ideal.
A less extreme approach would be to in- clude some but perhaps not all designs while again care- fully qualifying inferences. There are arguments for each of these approaches. In research areas where strong experimental designs are relatively easy to conduct and plentiful—for example re- search on the impact of drug treatments on the social ad- justment of children with attention disorders Ottenbacher and Cooper —I have been persuaded that including only designs that permit relatively strong causal infer- ences was an appropriate approach to the evidence.
Here, the synthesists are forced to make a choice between two philosophically different approaches to evidence. If the synthesists be- lieve that discretion is the better part of valor, then they might opt for including only the few experimental studies or stating simply that no credible evidence on the causal relationship exists.
Alternatively, if they believe that any evidence is better than no evidence at all then they might proceed to summarize the less-than-optimal studies, with the appropriate precautions, of course.
Generally speaking, when experimental evidence on causal questions is lacking or sparse I think a more inclu- sive approach is called for, assuming that the synthesists pay careful and continuous attention to the impact of re- search design on the conclusions that they draw. Returning to our example of instructor.
Grades were operationalized as scores on homework as- signments turned in after each lecture. These studies might have demonstrated that more likeable guest lectur- ers produced higher student grades on homework. Thus, in order to carry out the manipulation it was necessary to study likeability in short-term instructor-student interac- tions. Could it be that over time—more similar to the real-world relationship that develops between instructors and students—likeability becomes less important and perceived competence becomes more important?
These issues, related to construct and external validity, go unaddressed if only the experimental evidence is per- mitted into our synthesis of instructor likeability and homework grades. The quasi-experiments found in the literature might use end-of-term class grades as outcome measures. The structural equation models might use large, nationally- representative samples of students and relate ratings of liking of instructors in general to broad measures of achievement, such as cumulative GPAs and SAT scores.
If the different types of evidence are inconsistent, it should be viewed as a cau- tion to generalization. Designs appropriate to gather data on one type of problem may or may not. Typically, in primary research only a single research design can be employed in each study. However, in research synthesis, a variety of research de- signs relating the variables of interest are likely to be found.
When the relation of interest concerns an associa- tion between variables, designs that seek to rule out al- ternative explanations or establish causal links are still relevant. Single-case research has developed its own array of research designs and data analysis strate- gies for an exposition of single-case research designs, see Barlow and Hersen ; for a discussion of data analysis issues, Thomas Kratochwill and Joel Levin The issues I have discussed regarding descriptive, associational, and causal inferences and how they relate to different research designs play themselves out in the single-case arena in a manner similar to that in group- level research.
Different time series designs are associated with different types of research problems and hypotheses. Much like variations in group de- signs, variations in these interrupted time series would result in stronger or weaker inferences about the causal relationship of interest. The mechanics of quantitative synthesis of single-case designs requires its own unique toolbox for examples of some approaches, see Busk and Serlin ; Faith, Alison,.
However, the logic of synthesizing single-case research is identical to that of synthesizing group-level research. It would not be unusual for synthesists to uncover a variety. Appropriate inferences drawn from each of these designs correspond more or less closely with our causal question. The synthesists may choose to focus on the most correspondent designs, or to include designs that are less correspondent but that do provide some information.
And, of course, regardless of the decision rule adopted concerning the inclusion of research designs, synthesists are obligated to carefully delimit their inferences based on the strengths and weaknesses of the included designs. At their broadest level, the problems and hypotheses that motivate most research syntheses involve main effects, relations between two variables.
Does liking of the in- structor cause higher class grades? Recognized as the definitive resource for research synthesis when published in , updated in and again in , this work remains the first book many people turn to.
The book is arranged as an encyclopedia, with each chapter written by experts in a specific aspect of research synthesis. However, the editors also arranged for the chapter authors to use the same datasets and style throughout, so the book flows naturally from one section to the next, and offers an excellent overview of the field for statisticians and researchers alike.
In the third edition, the editors present updated versions of classic chapters and add new sections that evaluate cutting-edge developments in the field. This edition of the Handbook provides comprehensive instruction in the skills necessary to conduct research syntheses and represents the premier text on research synthesis.
Where "The Handbook" covers many aspects of research synthesis, this volume focuses almost exclusively on meta-analysis the statistical component of research synthesis , and does so from a mathematical perspective. Order Online. Julian P. Page, Vivian A. Welch co-eds. Hoboken, NJ: Wiley-Blackwell. This book is the essential manual for all those preparing, maintaining and reading Cochrane reviews of the effects of health interventions. The fully updated second edition contains extensive new material on systematic review methods addressing a wide-range of topics including network meta-analysis, equity, complex interventions, narrative synthesis, and automation.
Also new to this edition, integrated throughout the Handbook, is the set of standards Cochrane expects its reviews to meet. Designed to be an accessible resource, the Handbook will also be of interest to anyone undertaking systematic reviews of interventions outside Cochrane, and many of the principles and methods presented are appropriate for systematic reviews addressing research questions other than effects of interventions. This book focuses on that part of the research synthesis that is not the meta-analysis: How to formulate the research question, locate the relevant studies, code the data, interpret and present the analysis.
This book puts the meta-analysis endeavor in context, and is an invaluable and very readable resource, both for researchers and as a text in courses on meta-analysis. Written in plain language with four running examples drawn from psychology, education, and health science, this book offers practical advice on how to conduct a synthesis of research in the social, behavioral, and health sciences. With ample coverage of literature searching and the technical aspects of meta-analysis, this one-of-a-kind book applies the basic principles of sound data gathering to the task of producing a comprehensive assessment of existing research.
The book offers a practical, hands-on approach and as such is probably the best general purpose introduction to meta-analysis. It offers an overview of all aspects of research synthesis, and a non-technical introduction to the statistical aspects of meta-analysis.
DOI: Hedges and Jeffrey C. Hedges , Jeffrey C. Valentine Published Psychology. Distilling a vast technical literature and many informal sources, the Handbook provides a portfolio of the most effective solutions to the problems of A century of research on conscientiousness at work PNAS A Brief History of Research Synthesis and Meta-Analysis In , Karl Pearson conducted what is believed to be the first meta-analysis.
Traditionally, the main purpose of meta-analyses has been to estimate the combined effect size of multiple replications. However, there is more to meta-analysis than merely obtaining an overall effect size estimate.
To resolve this inconclusiveness in an empirical way, we conducted a meta—analysis on 17 laboratory studies and assessed effect sizes unbiased d, r and r 2 of the epo—induced improvements in aerobic exercise capacity measured by maximal oxygen uptake Effectiveness and optimal dosage of resistance training Feb 24, Dixon-Woods and her colleagues reviewed potential methods for synthesizing qualitative research with the guiding notion that syntheses should aim to develop concepts and theory, rather than simply aggregate data.
We performed a systematic scoping review of published methodological recommendations on how to systematically review and meta-analyse observational studies. We searched online databases and websites and contacted experts in the field to locate Marc J. Lajeunesse Publications The Handbook of Research Synthesis and Meta-Analysis is an illuminating compilation of practical instruction, theory, and problem solving.
The use of the term meta-synthesis to describe any synthesis of qualitative research has been criticised on the grounds that it A meta-analysis and synthesis of public transport customer As such, meta-analysis provides a basis for a more rational approach to decision making, especially decisions involving highly emotional issues such as the education of special-needs students.
Three meta-analyses in the educational literature address the issue of the most effective setting for the education of special-needs students Baker Open Access Journals This respectability, when combined with the slight hint of mystique that sometimes surrounds meta-analysis, ensures that results of studies that use it are treated with the respect they deserve.
The Handbook of Research Synthesis is one of the most important publications in this subject both as a definitive reference book and a practical. Since the term and modern approaches to research synthesis were first introduced in the s, meta-analysis has Chapter Systematic Reviews Meta Analysis and In research synthesis, clinical and public health profes-sionals examine the scientific literature on a certain topic from multiple years and perspectives.
Meta-analysts dis-play the literature in one publication in different figures, including forest plots, funnel plots, and chronological cu-mulative meta-analysis. Comparisons of average effect sizes from sample studies indicated focused second language instruction results in large target-oriented gains, explicit types of instruction are more effective than implicit types, and focus on form and focus on forms interventions Overview of Systematic Review and Research Synthesis The Handbook of Research Synthesis and Meta-Analysis also provides a rich treatment of the non-statistical aspects of research synthesis.
Society for Research Synthesis Methodology - Home synthesis is a better descriptor of the process for qualitative research and the term meta-synthesis is used to distinguish this from quantitative meta-analysis. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. Using qualitative synthesis to explore heterogeneity of complex interventions.
Hedges Editor ; Jeffrey C. Valentine Editor Praise for the first edition: "The Handbook is a comprehensive treatment of literature synthesis and provides practical advice for anyone deep in the throes of, just teetering on the brink of, or attempting to The Handbook of Research Synthesis and Meta-Analysis The Handbook of Research Synthesis and Meta-Analysis draws upon groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right.
The editors and leading scholars guide the reader through every stage of the research synthesis. Place of Meta-analysis among other Methods of research The Handbook of Research Synthesis and Meta-Analysis draws upon groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right. The editors and leading scholars guide the reader through every stage of the research synthesis process—problem formulation, literature search and evaluation, statistical integration, and report The SAGE Handbook of Applied Social Research Methods Sep 12, Describes the steps of synthesizing sources for a literature review.
These approaches include meta-study, meta-summary, grounded formal theory, meta-ethnography, and qualitative meta-synthesis. Data Extraction Search this Guide Search. A Guide to Evidence Synthesis Whether you plan to perform a meta-analysis or not, you will need to establish a regimented approach to extracting data.
Green Eds. Meta-analysis of gender and science research: Synthesis Report Jun 15, Read Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis A meta-analysis of plant facilitation in coastal dune What is a meta-analysis In , Glass coined the term meta-analysis Meta-analysis refers to the analysis of analyses the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings. Glass, , p3 Meta-analysis techniques are needed because only Course Description May 30, Cochrane Handbook, part 2, chapter 9: Summarizing study characteristics and preparing for synthesis This chapter provides recommendations for health science authors on data synthesis.
Methods for research synthesis - 28Nov Research weaving might assist a meta-analysis in divorcing itself from poor practices, because research weaving makes more researchers aware of the importance of a systematic-review approach, enforcing transparency in every step of the process.
Many studies have iden. Download for offline reading, highlight, bookmark or take notes while you read The Handbook of Research Synthesis and Meta-Analysis. When the first edition of The Handbook of Research Synthesis and Meta-Analysis was published in , it quickly became the definitive reference for conducting meta-analyses Meta-Analysis and Meta-Synthesis Methodologies: Rigorously The Fifth Edition of Harris Cooper's bestselling text offers practical advice on how to conduct a synthesis of research in the social, behavioral, and health sciences.
Harris Cooper and Larry V. Hedges; eds. New York: Russell Sage Foundation; Meta-analysis has evolved in response to the need to synthesize the burgeoning literature for the advancement of knowledge. Research synthesis and dissemination as a bridge to The Handbook of Research Synthesis and Meta-Analysis draws upon groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right.
The Fifth Edition of Harris Cooper's bestselling text offers practical advice on how to conduct a synthesis of research in the social, behavioral, and health sciences. With ample coverage of literature searching and the technical aspects of meta-analysis, this one-of-a-kindOTseeker Jun 18, The purpose of this article is to illustrate structured review or meta-synthesis procedures for qualitative research, as well as, novel meta-analysis procedures for the kinds of multiple treatment.
Compare and contrast two of the five types of research Methodological synthesis in quantitative L2 research: A review of reviews and a case study of exploratory factor analysis.
Language Learning, 65 , Supp. Norris, S. Schoonen , Any research study can only be fully appreciated once it is situated in relation to existing work. This is no mean feat, however, given the sheer quantity and variety of publications to date.
Given the expanding application and importance of literature synthesis A methodological synthesis and meta-analysis of judgment Providing researchers with a practical and accessible advice, the Fourth Edition of the lauded "Research Synthesis and Meta-Analysis" offers thoroughly updated information. Research synthesis and meta-analysis - PhD course guide 1 Place of Meta-analysis among other Methods of research synthesis Julia Koricheva and Jessica Gurevitch in the terms, meta-moSt general analysis is one method of research synthesis.
Research syn-thesis may be defined as a review of primary research on a given topic with the purpose of "Meta Interpretation": A Method for the Interpretive The Handbook of Research Synthesis and Meta-Analysis also provides a rich treatment of the non-statistical aspects of research synthesis.