RSL con PRISMA: métodos para el proceso de validación de exámenes de ingreso a la universidad desde el enfoque basado en argumentos
Contenido principal del artículo
Resumen
En México, la selección de candidatos para la educación superior es un desafío debido al limitado número de cupos disponibles en las universidades. Para asegurar la calidad de los estudiantes admitidos, es crucial evaluar la validez de los exámenes de ingreso. Uno de los enfoques más revisados para la validación de pruebas educativas, en los últimos veinte años, es el Enfoque Basado en Argumentos propuesto por Michael Kane, sin embargo, no ha quedado claro el método, métodos o enfoques para poder evaluar la validez de las pruebas. Por ende, se realizó una Revisión Sistemática de la Literatura a través del modelo Prisma, con el fin de contribuir a esclarecerlo. Así, se encontró que en todos estudios que cubren este enfoque citan a Kane para que forme parte de su marco teórico y, en general, desarrollan métodos basados en argumentos sin esclarecerlos puntualmente. Asimismo, se encontró que el Enfoque Basado en Argumentos se ha extendido a enfoques de investigación mixta y cualitativas. Lo que demuestra, en conclusión, que el Enfoque Basado en Argumentos depende del contexto, la situación y las especificaciones de la prueba, aportando flexibilidad y estructura a la investigación en este campo.
DESCARGAS
Descargas
Métricas
Detalles del artículo
Referencias
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2018). Estándares para pruebas educativas y psicológicas (M. Lieve, Trans.). American Educational Research Association.
Asociación Nacional de Universidades e Instituciones de Educación Superior. (2023, 23 de abril). Anuarios estadísticos de educación superior. Ciclo escolar 2021-2022 y 2022-2023 de ANUIES. http://www.anuies.mx/informacion-y-servicios/informacion-estadistica-de-educacion-superior/anuario-estadistico-de-educacion-superior
Aviad-Levitzky, T., Laufer, B., & Goldstein, Z. (2019). The new computer adaptive test of size and strength (CATSS): Development and validation. Language Assessment Quarterly, 1–24. https://doi.org/10.1080/15434303.2019.1649409
Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford University Press.
Barrios, G., Santamaría, W., Navarro, C., & Berlanga, K. (2015). Validez del Exani-II en estudiantes de Ingeniería Industrial del Instituto Tecnológico de Ciudad Valles. Revista TECTZAPIC, 1(2). http://www.eumed.net/rev/tectzapic/2015/02/exani-II.html
Beigman Klebanov, B., Ramineni, C., Kaufer, D., Yeoh, P., & Ishizaki, S. (2017). Advancing the validity argument for standardized writing tests using quantitative rhetorical analysis. Language Testing, 026553221774075. https://doi.org/10.1177/0265532217740752
Chapelle, C., Enright, M., & Jamieson, J. (2008). Building a validity argument for the Test of English as a Foreign Language. Routledge.
Chappelle, C., Enright, M., & Jamieson, J. (2010). Does an argument-based approach to validity make a difference? Educational Measurement: Issues and Practice, 29(1), 3-13.
Chapelle, C. (2012). Validity argument for language assessment: The framework is simple…. Language Testing, 29(1), 19–27.
Chuang, P., & Yan, X. (2022). An investigation of the relationship between argument structure and essay quality in assessed writing.
Cizek, G. J., Kosh, A. E., & Toutkoushian, E. K. (2018). Gathering and evaluating validity evidence: The generalized assessment alignment tool. Journal of Educational Measurement, 55(4), 477–512. https://doi.org/10.1111/jedm.12189
Covay Minor, E., Porter, A. C., Murphy, J., Goldring, E. B., Cravens, X., & Elloitt, S. N. (2014). A known group analysis validity study of the Vanderbilt Assessment of Leadership in Education in US elementary and secondary schools. Educational Assessment, Evaluation and Accountability, 26(1), 29–48. https://doi.org/10.1007/s11092-013-9180-z
Esfandiari, M., Riasati, M., Vaezian, H., & Rahimi, F. (2018). A quantitative analysis of TOEFL iBT using an interpretive model of test validity. Language Testing in Asia, 8(1).
Fan, J. (2014). Chinese test takers’ attitudes towards the Versant English Test: A mixed-methods approach. Language Testing in Asia, 4(1). https://doi.org/10.1186/s40468-014-0006-9
Fechter, T., Dai, T., Cromley, J. G., Nelson, F. E., Van Boekel, M., & Du, Y. (2021). Developing a validity argument for an inference-making and reasoning measure for use in higher education. Frontiers in Education, 6(1). https://doi.org/10.3389/feduc.2021.727539
García-Peñalvo, F. J. (2022). Desarrollo de estados de la cuestión robustos: Revisiones Sistemáticas de Literatura. Education in the Knowledge Society (EKS), 23, e28600. https://doi.org/10.14201/eks.28600
Ginther, A., & Elder, C. (2014). A comparative investigation into understandings and uses of the "TOEFL iBT"® test, the International English Language Testing Service (Academic) test, and the Pearson Test of English for graduate admissions in the United States and Australia: A case study of two university contexts. "TOEFL iBT"® Research Report. TOEFL iBT-24. ETS Research Report. RR-14-44. 1-39. https://doi.org/10.1002/ets2.12037
Huang, B. H., Bailey, A. L., Sass, D. A., & Shawn Chang, Y. (2020). An investigation of the validity of a speaking assessment for adolescent English language learners. Language Testing, 026553222092573. https://doi.org/10.1177/0265532220925731
Im, G.-H., Shin, D., & Cheng, L. (2019). Critical review of validation models and practices in language testing: Their limitations and future directions for validation research. Lang. Test. Asia, 9(14). https://doi.org/10.1186/s40468-019-0089-4
Jawhar, S., Al, M., Alhawsawi, S., & Alkushi, A. (2021). Validating English language entrance test at a Saudi university for health sciences. Arab World English Journal (AWEJ), 12(2), 49-71. https://dx.doi.org/10.24093/awej/vol12no2.4
Jiménez, Ó. (2021). Los exámenes de ingreso a la Educación Superior Mexicana, un obstáculo a la aspiración de educación inclusiva para estudiantes pobres y de pueblos originarios. Espacio I+D, Innovación más desarrollo, 10(25). https://doi.org/10.31644/IMASD.26.2021.a07
Jorion, N., Gane, B. D., James, K., Schroeder, L., DiBello, L. V., & Pellegrino, J. W. (2015). An analytic framework for evaluating the validity of concept inventory claims. Journal of Engineering Education, 104(4), 454–496.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319-342.
Kane, M. (2002). Validating high stakes testing program. Educational Measurement: Issues and Practice, 21(1), 31–41.
Kane, M. (2006 a). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319-342. https://doi.org/10.1111/j.1745-3984.2001.tb01130.x
Kane, M. T. (2006 b). Validation. In R. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Westport, CT: American Council on Education/Praeger.
Kane, M. (2011). Validating score interpretations and uses. Language Testing, 29(1), 3–17. https://doi.org/10.1177/0265532211417210
Kane, M. (2012). Validating score interpretations and uses. Language Testing, 29(1), 3–17.
Kane, M. (2013). The argument-based approach to validation. School Psychology Review, 42(4), 448–457.
Kane, M. (2016). Validity arguments for educational measurement. Routledge.
Kerlinger, F., & Lee, H. (2001). Investigación del comportamiento: métodos de investigación en ciencias sociales. McGraw Hill.
LaFlair, G. T., & Staples, S. (2017). Using corpus linguistics to examine the extrapolation inference in the validity argument for a high stakes speaking assessment. Language Testing, 34(4), 451–475. https://doi.org/10.1177/0265532217713951
Lane, S. (2014). Validity evidence based on testing consequences. Psicothema, 26(1), 127–135.
Lavery, M., Bostic, J., Kruse, L., Krupa, E., & Carney, M. (2020). Argumentation surrounding argument-based validation: A systematic review of validation methodology in peer-reviewed articles. Educational Measurement: Issues and Practice. https://doi.org/10.1111/emip.12378
Li, S. (2018). Developing a test of L2 Chinese pragmatic comprehension ability. Language Testing in Asia, 8(1). https://doi.org/10.1186/s40468-018-0054-7
Messick, S. (1989). Validity. En R. L. Linn (Ed.), Educational measurement (3.ª ed., pp. 13–103). Macmillan.
Mislevy, R., Steinberg, L., & Almond, R. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–62.
Morales, R., Barrera, A., & Garnett, E. (2015). Validez predictiva y concurrente del EXANI-II en la Universidad Autónoma del Estado de México. X Congreso Nacional de Investigación Educativa, sujetos de la educación. https://www.comie.org.mx/congreso/memoriaelectronica/v10/pdf/area_tematica_16/ponencias/0701-F.pdf
Newton, P., & Shaw, S. (2014). Validity in educational & psychological assessment. SAGE.
Padilla, J.-L., & Benítez, I. (2014). Validity evidence based on response processes. Psicothema, 26(1), 136–144.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., ... Moher, D. (2021). Declaración PRISMA 2020: Una guía actualizada para la publicación de revisiones sistemáticas. Revista Española de Cardiología, 74(9), 790–799. https://doi.org/10.1016/j.recesp.2021.06.016
Reeves, T., & Marbach-Ad, G. (2016). Contemporary test validity in theory and practice: A primer for discipline-based education researchers. CBE—Life Sciences Education, 15(1).
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J., & PRISMA-S Group. (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10(39). https://doi.org/10.1186/s13643-020-01542-z
Rix, S. (2012). Construct validity in formative assessment: Purpose and practices. Journal on English Language Teaching, 2(2), 1–6. https://eric.ed.gov/?id=EJ1070202
Rodríguez, B. (2004). El modelo argumentativo de Toulmin en la escritura de artículos de investigación educativa. Revista Digital Universitaria, 5(1), 2-18. https://www.revista.unam.mx/vol.5/num1/art2/ene_art2.pdf
Saldaña, J. M. (2015). The coding manual for qualitative researchers (3.ª ed.). SAGE Publications.
Schilling, S. G. (2004). Conceptualizing the Validity Argument: An Alternative Approach. Measurement: Interdisciplinary Research and Perspectives, 2(3), 178–182.
Seipel, B., Kennedy, P. C., Carlson, S. E., Clinton-Lisell, V., & Davison, M. L. (2023). MOCCA-College: Preliminary Validity Evidence of a Cognitive Diagnostic Reading Comprehension Assessment. Journal of Learning Disabilities, 56(1), 58–71. https://doi.org/10.1177/00222194221121340
ShayesteFar, P. (2020). A model of interplay between student English achievement and the joint affective factors in a high-stakes test change context: model construction and validity. Educational Assessment, Evaluation and Accountability. https://doi.org/10.1007/s11092-020-09326-8
Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practices, 16(2), 5–24.
Sireci, S. (2007). On validity theory and test validation. Educational Researcher, 36, 477-481.
Sireci, S., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26(1), 100–107.
Staples, S., Laflair, G. T., & Egbert, J. (2017). Comparing Language Use in Oral Proficiency Interviews to Target Domains: Conversational, Academic, and Professional Discourse. The Modern Language Journal, 101(1), 194–213. https://doi.org/10.1111/modl.12385
Yan, X., & Staples, S. (2019). Fitting MD analysis in an argument-based validity framework for writing assessment: Explanation and generalization inferences for the ECPE. Language Testing. https://doi.org/10.1177/0265532219876226
Barra lateral del artículo
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.
