Generative AI Efficacy
“This project sparked my imagination on pedagogical innovation. I began to see the power of generative AI in transforming pedagogical methodologies as well as classroom settings.” –Dongpeng Huang, doctoral student in the College of Communication
ChatGPT and other generative artificial intelligence (GenAI) tools have emerged as potentially transformative factors in education, received with a mix of excitement and apprehension. Educators at BU are particularly interested in how this emerging advanced technology can be leveraged in the classroom to create enriching and engaging learning experiences.
In this Future of Learning: AI Grant (FLAG) project, Dongpeng Huang, a doctoral candidate in the Emerging Media Studies program in the College of Communication, explored the potential of integrating ChatGPT into the teaching and learning process of her course COM CM321 Communication Research Method, with an enrollment of 25 students. This course focuses on enhancing students' understanding of abstract concepts, statistical analysis, and conducting group research projects to apply research methods in practice. By incorporating ChatGPT into various classroom activities, students had access to powerful academic assistance tools that supported and augmented their academic tasks, fostered their critical thinking skills, and promoted collaborative learning through group research projects. The project also enabled Huang to gather feedback from students, which will guide her in refining the integration of ChatGPT into future teaching pedagogies. 
To accomplish this, Huang created three class activities designed to leverage the capabilities of ChatGPT while ensuring active student participation and academic growth:
- Individual formative e-assessments with ChatGPT to examine the impact of GenAI on student learning outcomes.
- Group Project - Writing a Research Proposal with ChatGPT to facilitate collaborative learning and the development of critical thinking and research skills through group work.
- Group Project - Data Analysis with ChatGPT to leverage ChatGPT's capabilities in data analysis to enhance students' understanding of research findings and develop data-driven insights.
The AI-integrated activities enhanced student’s learning experiences by improving their critical thinking and problem-solving skills and increasing academic collaboration and communication. The student-ChatGPT group project explored ways for human-AI collaboration, where students learned to effectively communicate their ideas and benefit from AI technology.
“When working with RStudio for data analysis, ChatGPT provided step-by-step instructions which were much faster than traditional search methods,” according to undergraduate student, Hanyun Deng. “This immediate assistance kept me more involved in the learning process and encouraged me to take on more complex tasks with more confidence.”
For the group collaboration, students were placed in groups of 3 or 4 and instructed to find a research topic and question. Students first brainstormed with group members and ChatGPT to find interesting issues to explore and then used ChatGPT to refine their project title or survey questions. The assignment prompts were designed to let students report their activities with GPT, where they were required to cite their work with GPT and explain what they completed vs. what ChatGPT completed. ChatGPT was also a useful tool for proofreading, logic, and clarity in the proposal development phase for non-native English speakers enrolled in the course.
Overall, students developed critical thinking skills by evaluating the information and perspectives that ChatGPT provided, and making their own evidence-based decisions. Huang employed Likert scales to assess trustworthiness of ChatGPT, social presence, usage intention, and student attitudes toward the technology. She also recorded student interaction with ChatGPT while they worked on assessment questions and their research proposal. In collaboration with Lisa Burgess, Huang conducted a student focus group within the class to analyze their perceptions of academic cheating with GenAI. She found that while students clearly recognized overt cheating, opinions varied regarding more subtle forms of dishonesty and the effectiveness of formal deterrents. Peer influence and personal ethics were identified as strong factors shaping cheating behaviors. Class policies enforced by instructors appeared to have a greater influence on student cheating behavior with generative AI compared to broader institutional policies. Moreover, student trustworthiness varied depending on the task, where students had lower trust toward ChatGPT for assessment than for group work. Overall, Huang believes that group collaboration is a scenario that will benefit from AI-incorporation for students and teaching applications.
The integration of ChatGPT into the research methods course has far-reaching implications beyond the immediate classroom transformation. Its inclusion as an educational tool establishes a precedent for innovative teaching methodologies in the Emerging Media Studies (EMS) program, and aligns with EMS's objective of staying up-to-date with emerging technologies, which offers valuable insights into leveraging such technologies to enhance learning outcomes and student engagement. This, in turn, can foster a culture of innovation and promote the adoption of transformative teaching practices throughout the institution.
“Attitudes significantly predict usage intention of taking a course with generative AI as a teaching assistant,” according to Huang’s research. “Integrating AI in the classroom is inevitable. To prepare, we need to define and regulate the use of ChatGPT.”
For her future vision, Huang is considering using GenAI again in future courses with consideration for trying out more types of GenAI technology.
“There’s a huge innovation in the field right now, with image, text, video, text to video, and text to image,” said Huang. “I think there should be more research along this line in terms of how generative AI can help you achieve pedagogical goals and help with curriculum development.”
Conferences, News & Publications
- Huang, D., Huang, Y., & Cummings, J. J. (2024). Exploring the integration and utilisation of generative AI in formative e-assessments: A case study in higher education. Australasian Journal of Educational Technology, 40 (4), 1–19. https://doi.org/10.14742/ajet.9467.
- Huang, D., Hash, N., Cummings, J., and Prena, K. (2024, August) “Academic Cheating with Generative AI: Exploring a Moral Extension of the Theory of Planned Behavior.” AEJMC2024: The 107th Annual Conference of the Association for Education in Journalism and Mass Communication, Philadelphia, Pennsylvania, USA.
Project Lead
Dongpeng Huang is a Ph.D. student at Boston University’s Emerging Media Studies program. Her research interests include artificial intelligence (AI) in education. In 2023, she was awarded a Shipley Center Accelerating Classroom Transformation (ACT) grant and conducted quantitative and qualitative research on AI integration in…
