Mentored Teaching Project

As part of the completion of the Certification in College Teaching offered at Michigan State University, I have compiled my teaching portfolio on this webpage.
Course Description

This project was conducted in CEM 355, which is the first organic chemistry laboratory that students majoring in chemistry, biochemistry, and chemical engineering take. These students are usually in their sophomore year of study. CEM 355 seeks to familiarize students with common processes and analytical instrumentation used in organic chemistry. Therefore, much of students’ time is dedicated to honing organic synthesis and purification skills. Sections of approximately 21 students meet twice each week for three hours. CEM 355 is offered in the spring semester, but recently summer courses have been added. The completion of this project occurred during the second portion of the 2018 summer semester under the mentorship of Prof. Kevin D. Walker of the Department of Chemistry.

Course Objectives
  •  Students will relate concepts from organic chemistry lecture courses to experiments in lab.
  •  Students will practice common organic chemistry laboratory techniques, including isolation, purification, synthesis, and analysis.
  •  Students will interpret data, draw conclusions, and recommend next steps for an experiment.
  •  Students will organize background information, experimental methods, results, and conclusions into written laboratory reports.

Problem

This project seeks to increase student engagement in critical thinking in CEM 355 and emphasize the importance of data analysis and presentation as components of the scientific method.

Currently, the laboratory reports and grading rubrics for this course are heavily focused on successful completion of the experiment. Therefore, for certain experiments, a student who completes the experiment but does not obtain a high yield or pure product will receive a low score, even if they have worked to determine the source of their errors. Further, laboratory reports have previously been due at the end of the laboratory period in which the experiment is completed. This limits the opportunity students have to analyze their data, consider errors, and draw conclusions.

Research Question

The research question guiding this project is “What skills do students gain with the addition of questions to guide their analysis and preparation of laboratory reports?”

Artifacts and Rationales

The Worksheet for Developing a Mentored Teaching Project Plan and the proposal I submitted for the mentored teaching project is included, where the background, hypothesis, and methods of the project are described.

For the purposes of this project, I modified the course syllabus to include pre-lab reading assignments, post-lab guiding questions, and a description of the formal laboratory report assignments.

To assist students in preparing their laboratory reports, I disseminated a template and a description of laboratory report expectations.

I modeled the rubrics on those used in the CEM415 laboratory at Michigan State University, splitting the distribution of points in order to assess both the products of the laboratory and spectral characterization, which is how CEM355 had been graded, as well as responses to the guiding questions.

The project evaluation disseminated to students in Spring 2018 (SS2018, control group) and Summer 2018 (US2018, experimental group) asked students to rank their confidence in laboratory skills and agreement with statements from 1 (low confidence or disagreement) to 5 (high confidence or agreement).

Results

CEM 355 is offered in spring semesters, with 11 sections of approximately 20 students during SS2018. US2018 was the second time the course was offered during a summer semester, with 2 sections of 20 students (Figure 1); the first being US2017, when only one section was offered. Despite a significant variation in class size between the two semesters studied, the response rates were comparable, 84% and 78%, respectively for SS2018 and US2018. Course surveys were disseminated to students by teaching assistants during the last two weeks of each semester.

Figure 1. SS2018 respondents by course section: 183 respondents of 217 enrolled students from 11 course sections (84% of students responding); US2018 respondents by course section:  31 respondents of 40 enrolled students from 2 course sections (78% of students responding)

The way students used course materials did not change much between SS2018 and US2018. Over 60% of students in both semesters indicated the use of the supplementary resources, while less than 40% read the textbook (Figure 2). With the change in grading scheme, US2018 students were awarded five points for turning in pre-lab summaries at the beginning of class, which explains the 40% increase in students reporting that they do that before class. During SS2018 and previous sessions of CEM355, students are instructed to write pre-lab summaries which are turned in with their reports following the completion of the experiments. Bearing in mind the connectedness of modern students, Prof. Walker encouraged US2018 students to watch YouTube videos explaining and demonstrating laboratory techniques. Anecdotally, several students reported to the teaching assistants that they found the YouTube videos helpful in preparing them for what glassware and processes they would be doing in lab.

Figure 2. Pre-Lab Preparation, multiple responses accepted

When surveyed on the thoroughness of the course materials, namely the supplementary information and text book, 71% and 43% of SS2018 students reported a 4 or 5, indicating satisfaction with the supplementary content and textbook, respectively (Figure 3). Responses from US2018 students on these same questions were generally consistent with those from SS2018. 65% of students reported satisfaction with the supplementary course materials, and 48% were satisfied with the text book.

Figure 3. Thoroughness of Course Materials

Undergraduate laboratory instruction is performed by graduate teaching assistants who receive little formal instruction in how to interact with students. The majority of SS2018 and US2018 CEM355 students reported that their instructors encourage critical thinking and help them learn to make data-based conclusions (Figure 4).

Figure 4. Instructor Effectiveness

Despite the difference in the number of enrolled students in the two semesters studied,  self-reported confidence in laboratory skills varied between the pre- and post-intervention groups (Figure 5). Students reported higher confidence in laboratory skills that were repeated several times throughout the semester (melting point, distillation, extraction, and recrystallization) compared to thin layer chromatography (TLC) and column chromatography, both of which students performed only once (Figure 5a). It is  noteworthy that, in general, US2018 students reporting low confidence in skills used lower values to do so than SS2018 students. This trend is more evident in Figure 5b, where students reported confidence in spectroscopy and data analysis skills. The 20% decrease in students reporting confidence values of 4 or 5 for NMR during US2018 may have been due to personnel shortages; student comments indicated that low confidence in this area was related to guidance using the IR and NMR. SS2018 students were trained in the NMR facility outside of class, and a teaching assistant was always present at the facility while students were using the instruments. Since only two teaching assistants were employed during US2018 and both sections took place at the same time, the teaching assistants trained students during down time in the laboratory and took turns checking on students while they were using the instrumentation.

Figure 5. a) Laboratory Techniques, b) Spectroscopy and Analysis

Similar to other skills, student confidence in data analysis and ability to draw conclusions decreased from SS2018 to US2018 (Figure 5b). Considering the aim of this project was to enhance students’ critical thinking ability, these results are contrary to our hypothesis. Generally, 58% of US2018 students reported a confidence value of 5 (Figure 6), compared with 54% of SS2018 students. The percentage of students reporting a confidence value of 4 decreased from SS208 to US2018 by 14%. Middle to low range confidence values of 2 and 3 remained much the same, with 1% to 2% more of US2018 students reporting these levels. Interestingly, 0.3% of SS2018 students (6) reported the lowest level of confidence when it came to laboratory and analysis skills, compared to 6% of US2018 students (19).

Figure 6. Stratification of Responses to Confidence in Laboratory Skills

The increased stratification of responses from US2018 students compared to SS2018 students continues in critical thinking and conclusion inquiries (Figure 7, Q1 and Q2). US2018 students expressed less confidence in these skills than SS2018 students.

Initially, these data indicate that the interventions administered to US2018 students decreased their self-efficacy. While this may be true, there was no significant difference in the final grade distribution from SS2018 to US2018, and from the perspective of the instructor, the discussion and conclusion sections of most US2018 reports were well-written.

Another lens through which to interpret these data considers that the decrease in confident students extends from a more thorough understanding of what is needed to succeed in CEM355. It could be argued that increased proficiency in critical thinking could result in students’ increased scrutiny of their own self-efficacy. The US2018 students responded to guiding questions and were aware of the grading scheme from the beginning, whereas there was less guidance for reports during SS2018 and students’ grades were based on purity, yield, and spectral analysis.

Figure 7. Critical Analysis

Figure 8. Percentage of positive and negative student comments on common themes from SS2018 (128 comments) and US2018 (17 comments)

Despite the increase in stratification from SS2018 to US2018, many student comments were positive, indicating that students were satisfied by the course and assignments (Figure 8, Figure 9). Student comments were profiled as positive and negative and grouped into six main categories (Figure 8). 70% (128) of SS2018 students and 55% (17) of US2018 students who returned their survey made comments. The majority of student comments from both semesters indicated dissatisfaction with course materials, including the text book and syllabus containing experimental details. Students also expressed dissatisfaction with group work (more group work was desired, or repercussions for partners who did not attend consistently), assignments, time available to complete experiments, and the course overall. A greater portion of US2018 students were satisfied with the course materials, teaching assistants, and the course overall compared with SS2018 students.

Several SS2018 student comments echoed course modifications that were planned for US2018, including updates to the syllabus, more structured pre-lab assignments, instrument training closer to experiments where they will be used, and providing guidance for conclusions.

Figure 9. Selected student comments from SS2018 and US2018

US2018 students provided valuable feedback in their comments as well (Figure 9), indicating that while modifications were made to the syllabus, there was still discrepancy between the experimental description and what was performed in lab. Though students were trained on the NMR and IR instrumentation closer to using it for experiments, students expressed low confidence in using the instrumentation and analyzing the results. As the course stands, NMR training is performed outside of class time and students feel rushed. In the future, an experiment focusing on the use of NMR and IR and analysis of the resulting spectra may be beneficial. Follow these links to documentation of all student comments from SS2018 and US2018.

Conclusions

Contrary to our hypothesis, the percentage of students’ reporting low confidence in laboratory skills and data analysis increased from SS2018 to US2018. While it is possible that more students felt less confident, final grades in the course were consistent with previous semesters. This response may represent students’ understanding of the subject area’s complexities despite low confidence. The intervention may have increased students’ awareness of their own deficiencies and areas needing improvement, “knowing what they don’t know,” so to say.

A variety of factors must be taken into account with these conclusions, foremost being the large difference in number of students enrolled in SS2018 compared to US2018. The discrepancy results from the common sequence of courses within the Chemistry Department, students typically take CEM355 in spring semesters. Students who enroll in summer semesters indicated in conversation that they were catching up in coursework, working on campus over the summer, or trying to get ahead in the curriculum. The level of motivation of summer students compared to students who follow the typical course sequence was not studied. Lastly, the expedited timeline of summer semesters adds stress to students and their workload, which certainly may contribute to low self-efficacy.

Should this study be continued in the future, it should be performed during the spring semester to observe a larger population of students consistent with the control group. Additionally, rubrics should be modified to enhance detail and assess students’ ability to draw conclusions from data. These assessments, when compared to students’ self-assessments, could provide greater insight to the relationship between skill proficiency and self-efficacy.

Reflection

As an experienced teaching assistant familiar with CEM355, I have spent a lot of time thinking about what I would do as the instructor to enhance student learning in my future classroom. This teaching as research project provided a platform upon which I could practice and receive mentoring toward that end. My experience planning a teaching as research project for the CIRTL MOOC helped me as I planned this project. I used backward design to determine learning objectives, form my research question and hypothesis, and plan my assessments.  I received some feedback from Dr. Joyce Parker on my project plan, and met several times during SS2018 with Dr. Kevin Walker to discuss the course materials and assessments I prepared.

In the process of completing the project, I realized that however much work I put into the planning process, the project could have benefitted from more work and consultation with experienced education researchers. I found that, while improved from SS2018 rubrics, the US2018 rubrics I constructed would’ve benefitted from greater detail. Nevertheless, I was impressed by the quality of students’ laboratory reports and saw improvement in their ability to discuss results and draw conclusions compared to my previous semesters teaching CEM355.

One consideration for the future is the format of laboratory reports. For the purposes of this project, students typed full laboratory reports instead of reports composed in their laboratory notebooks. In my opinion, the extended deadline and formality of the reports resulted in students taking them more seriously than in previous semesters, but added a lot of time to students’ workload in an already time-intensive summer session. In the future, a balance between writing up relevant data, analysis, and conclusions in notebooks and the added structure of this project would benefit students while not significantly increasing their workload outside of class.

Assessment

Evaluation of the Mentored Teaching Project by my mentor for the project, Dr. Kevin Walker.

Acknowledgements

I extend my gratitude to Professor Ardeshir Azadnia, Professor Kevin D. Walker, Professor Joyce Parker, Dr. Yu-Ling Lien, Dr. Timothy Shannon, Dr. David A. Olson, and Mr. Benjamin Appiagyei for their contributions to the planning and execution of this study. Additionally, Mr. David Voss and Ms. Dianne Karsten provided technical support throughout the semester. My primary research advisor, Professor Milton R. Smith, III has been very supportive of  my professional development pursuits. Professor Melanie Cooper provided stimulating discussion and important considerations regarding the results of this study. Lastly, this study would not have been possible without the SS2018 and US2018 CEM355 students who participated in the laboratory and assessments.

OMC Fall 2018