Studies in Analytical Chemistry and Chemical Education. Part 1: Characterization of Complex Organics By Raman Spectroscopy and Gas Chromatography. Part 2: Differential Item Functioning on Multiple-choice General Chemistry Assessments
Date of Award
Doctor of Philosophy
Joseph H. Aldstadt, III
Kristen L. Murphy
Joseph H. Aldstadt, III, Kristen L. Murphy, Mark L. Dietz, Andrew Pacheco, April Zenisky
Artistic Media, Assessment, Chemed, DIF, Raman Spectroscopy, Varnish Ageing
PART 1: CHARACTERIZATION OF COMPLEX ORGANICS BY RAMAN SPECTROSCOPY AND GAS CHROMATOGRAPHY.
The analytical chemistry component of this thesis focused on instrumentation and methods to address challenges in art conservation, particularly the identification, quantitation, and reactivity of a set of representative varnishes and their degradation products. Methods for characterizing varnishes are of great interest to art conservators to restore art work more accurately. A database was created as a means to identify and quantify the composition of aged varnishes. Fourier Transform (FT)-Raman Spectroscopy was used to study common organic acids found in varnishes. The database included nine short-chain carboxylic acids, four di-carboxylic acids, and six medium-to-long-chain fatty acids. Four varnish samples (Linseed Oil, Tung Oil, Dammar, and Mastic) were studied as well. Through visual comparison and fingerprinting analysis comparison, identification of components in the Raman Spectral Database were recognized as components of the varnish samples. Singular Value Decomposition (SVD) was conducted to determine how well the database represented the unknown varnish samples. SVD was applied to the 19 standards collected in building the database. To reduce the amount of data, seven singular values were chosen. The seven singular values were then used to model several unknowns - Linseed Oil, Tung Oil, Dammar, and Mastic. The root-mean square (RMS) error for the unknowns were 0.08, 0.13, 0.21, and 0.21 Raman Intensity units, for Linseed Oil, Tung Oil, Dammar, and Mastic, respectively. If those values are compared to the largest peak in the unknown spectra, the % relative RMS errors are 1.7%, 1.7%, 4.9%, and 6.4%, respectively.
A method based upon Gas Chromatography (GC) was developed to characterize carboxylic acids formed as a result of varnish degradation. In this method, a headspace solid-phase microextraction (SPME) approach was optimized in which a 75 µm carboxen-polydimethylsiloxane (CAR/PDMS) SPME fiber was used to analyze mono carboxylic acids. For quantitative determinations, the injection port was in the splitless mode and held at 250°C for 1.0 min for the desorption of the analytes from the SPME fiber. After the initial minute, the injector was switched to a 1:100 split ratio. The temperature program consisted of the oven being initially set to a temperature of 30°C and held for 1 min, and then ramped at 25°C/min to 200°C, where the temperature was held for 1 min, thereby resulting in a total run time of 8.80 min. The PFPD was held at 200 °C for the entire run with a 0.5 ms gate delay, and the gate width was set to 20.0 ms. The mono carboxylic acids that were studied were Formic, Acetic, Propanoic, Butyric, Valeric, and Caproic Acid. A linear relationship was observed between the number of carbons in the carboxylic acid and the retention time (y = 0.75x + 1.55, R2=0.95). Quantitation of Acetic Acid was done by calibration using a first-order regression fit. The model yielded: y = 0.29x + 0.92 (R2=0.95). Using a second-order model, a better fit was found: y = 0.0025x2 - 0.0016x + 5.9 (R2=0.99).
An ageing chamber was designed, fabricated, and tested as a means for better understanding the decomposition of varnishes over time as a function of temperature, humidity, and ultraviolet light. The goal in the development of the ageing chamber was to demonstrate that it may be possible to create Standard Reference Materials (SRMs) artificially that resemble authentically aged varnishes. This is possible by the use of the ageing chamber that was built because it is directly incorporated into a GC oven where temperature, where UV radiation, humidity levels, and pollutants can be precisely controlled and carefully monitored. The GC method for carboxylic acids described above was developed to aid in the measurement of carboxylic acid fragments that could arise from the ageing process. There are promising results of the Raman Intensity increasing as the sample aged.
PART 2: DIFFERENTIAL ITEM FUNCTIONING ON MULTIPLE-CHOICE GENERAL CHEMISTRY ASSESSMENTS.
Over the past 30 years, there have been a plethora of studies on gender differences. Some of the earlier studies found that male students typically outperform female students in visual-spatial and quantitative abilities, whereas female students outperform male students in verbal abilities. In later studies it was reinforced that female students still tended to outperform male students in verbal abilities while the gap in science and mathematics (the latter as an extension of visual-spatial and quantitative abilities) closed greatly. During this same time, more female students entered the science, technology, engineering, and mathematics (STEM) fields. In 1966, only 25% of all STEM bachelor's degrees were obtained by female students, whereas in 2010 that percentage had grown to 50%. Specifically in chemistry, 49.9% of the bachelor's degrees were earned by women compared to the 18.5% in 1966.1 With assessments as a large source of the student's overall course grade, it is imperative that those assessments be valid and unbiased. One way to determine this is to use Differential Item Functioning (DIF). DIF occurs when subgroups of equal abilities perform statistically different on an item on an assessment where typically students that are matched with equivalent ability would have an equivalent possibility of answering the question on the assessment correctly. Because of the difficulty in determining students' ability often times the subgroups are matched on their proficiency or the score they received on an assessment.
This dissertation focused on four main questions. The first question focused on identifying items that exhibited DIF. The second question was to determine if DIF was real, i.e. did it persist no matter the set of students or the matching criteria used? The third question focused on determining the causes of DIF by cloning the items by content and construct (format). Lastly, it was hypothesized that one of the reasons behind why DIF is happening was due to the students' problem-solving process and examining these through the use of incorrect heuristics.
Data for the first part of the study was collected from two American Chemical Society‐Examinations Institute (ACS‐EI) trial tests (Form A and Form B) that were given to students who had completed one term of general chemistry. This data was analyzed using the Mantel‐Haenszel statistic to determine which items exhibited possible DIF. Along with the Mantel‐Haenzel statistic a two stage DIF analysis2 was conducted. Out of the 140 items, 33 exhibited DIF. On Form A there were 14 items which exhibited DIF, seven that favored male students and seven that favored female students. On Form B there were 19 items which exhibited DIF, 11 that favored female students and eight that favored male students. Those items that exhibited the highest probability of DIF were cloned and included on hourly examinations. These items were examined for DIF persistence against both stages of the two-stage analysis and other relevant measures of proficiency. As more results were collected, patterns emerged for persistent DIF items. On the 24 hourly examinations that were included in this analysis, there were a total of 687 items: 33 (5%) had a significant value using the Mantel-Haenszel statistic, thereby exhibiting persistent DIF. Of those 33 items, 15 were flagged with persistent DIF that favored female students and 18 were flagged with persistent DIF that favored male students. On the three standardized examinations, there were a total of 140 items; 19 (14%) had a significant value using the Mantel-Haenszel statistic, thereby exhibiting persistent DIF. Of those 19 items, two of the items that were flagged with persistent DIF favored female students and 17 of the items that were flagged with persistent DIF favored male students.
Along with these items, certain content areas and formats of the items were found to favor one gender. Over six semesters of testing, the content areas that consistently showed DIF that favored male students were measurement (density), greatest/least number of atoms, limiting reagents, ideal gas equation, and crystal structures; the content areas that favored female students were nomenclature and molecular orbital theory. The formats that tended to favor male students were visual-spatial, reasoning, and computation; the format that favored female students was specific chemical knowledge. By cloning these items, it was found that some of the possible causes of persistent DIF for certain items were the content and/or the format.
Lastly semi-structured interviews were conducted and it was found that for seven items the possible reason why DIF was happening was due to one subgroup using an incorrect heuristic. These items were in the specific content areas of measurement (density), greatest/least number of atoms, stoichiometry-general, and crystal structures. Additionally, the format inclusions of visual-spatial, reasoning, and computation for these items could also be contributing factors to the observed results.
1. S&E Degrees: 1966-2010: National Center for Science and Engineering Statistics. http://www.nsf.gov/statistics/nsf11316/content.cfm?pub_id=4062&id=2 (accessed May 26).
2. Zenisky, A. L.; Hambleton, R. K., Detection of Differential Item Functioning in Large-Scale State Assessments: A Study Evaluating a Two-Stage Approach. Educational and Psychological Measurement 2003a, 63 (1), 51-64.
Kendhammer, Lisa Kay, "Studies in Analytical Chemistry and Chemical Education. Part 1: Characterization of Complex Organics By Raman Spectroscopy and Gas Chromatography. Part 2: Differential Item Functioning on Multiple-choice General Chemistry Assessments" (2013). Theses and Dissertations. 253.