Comparative analysis of meta-analysis methods: When to use which?

Background: Several meta-analysis methods can be used to quantitatively combine the results of a group of experiments, including the weighted mean difference, statistical vote counting, the parametric response ratio and the non-parametric response ratio. The software engineering community has focuse...

Descripción completa

Detalles Bibliográficos
Autores principales: Dieste, Oscar, Fernández, Enrique, García Martínez, Ramón, Juristo, Natalia
Formato: Objeto de conferencia
Lenguaje:Inglés
Publicado: 2011
Materias:
Ciencias Informáticas
Meta-analysis
Reliability
statistical power
effect size
weighted mean difference (WMD)
response ratio (RR)
vote counting
Acceso en línea:http://sedici.unlp.edu.ar/handle/10915/129014
Aporte de:
SEDICI (UNLP) de Universidad Nacional de La Plata
id I19-R120-10915-129014
record_format dspace
institution Universidad Nacional de La Plata
institution_str I-19
repository_str R-120
collection SEDICI (UNLP)
language Inglés
topic Ciencias Informáticas
Meta-analysis
Reliability
statistical power
effect size
weighted mean difference (WMD)
response ratio (RR)
vote counting
spellingShingle Ciencias Informáticas
Meta-analysis
Reliability
statistical power
effect size
weighted mean difference (WMD)
response ratio (RR)
vote counting
Dieste, Oscar
Fernández, Enrique
García Martínez, Ramón
Juristo, Natalia
Comparative analysis of meta-analysis methods: When to use which?
topic_facet Ciencias Informáticas
Meta-analysis
Reliability
statistical power
effect size
weighted mean difference (WMD)
response ratio (RR)
vote counting
description Background: Several meta-analysis methods can be used to quantitatively combine the results of a group of experiments, including the weighted mean difference, statistical vote counting, the parametric response ratio and the non-parametric response ratio. The software engineering community has focused on the weighted mean difference method. However, other meta-analysis methods have distinct strengths, such as being able to be used when variances are not reported. There are as yet no guidelines to indicate which method is best for use in each case Aim: Compile a set of rules that SE researchers can use to ascertain which aggregation method is best for use in the synthesis phase of a systematic review. Method: Monte Carlo simulation varying the number of experiments in the meta analyses, the number of subjects that they include, their variance and effect size. We empirically calculated the reliability and statistical power in each case Results: WMD is generally reliable if the variance is low, whereas its power depends on the effect size and number of subjects per meta-analysis; the reliability of RR is generally unaffected by changes in variance, but it does require more subjects than WMD to be powerful; NPRR is the most reliable method, but it is not very powerful; SVC behaves well when the effect size is moderate, but is less reliable with other effect sizes. Detailed tables of results are annexed. Conclusions: Before undertaking statistical aggregation in software engineering, it is worthwhile checking whether there is any appreciable difference in the reliability and power of the methods. If there is, software engineers should select the method that optimizes both parameters.
format Objeto de conferencia
Objeto de conferencia
author Dieste, Oscar
Fernández, Enrique
García Martínez, Ramón
Juristo, Natalia
author_facet Dieste, Oscar
Fernández, Enrique
García Martínez, Ramón
Juristo, Natalia
author_sort Dieste, Oscar
title Comparative analysis of meta-analysis methods: When to use which?
title_short Comparative analysis of meta-analysis methods: When to use which?
title_full Comparative analysis of meta-analysis methods: When to use which?
title_fullStr Comparative analysis of meta-analysis methods: When to use which?
title_full_unstemmed Comparative analysis of meta-analysis methods: When to use which?
title_sort comparative analysis of meta-analysis methods: when to use which?
publishDate 2011
url http://sedici.unlp.edu.ar/handle/10915/129014
work_keys_str_mv AT diesteoscar comparativeanalysisofmetaanalysismethodswhentousewhich
AT fernandezenrique comparativeanalysisofmetaanalysismethodswhentousewhich
AT garciamartinezramon comparativeanalysisofmetaanalysismethodswhentousewhich
AT juristonatalia comparativeanalysisofmetaanalysismethodswhentousewhich
bdutipo_str Repositorios
_version_ 1764820451774693376

Ejemplares similares