Retrospective on Engineering Admission Policy

Research-InformedPolicyChange: ARetrospectiveonEngineeringAdmissions

BethM.Holloway,a TeriReed,b P. K.Imbrie,b and KenReidc

aPurdueUniversity,bTexasA&MUniversity, cOhioNorthernUniversity

Abstract Background Few studies have investigated how engineering education admission policies contribute to the underrepresentation of specific groups. Transforming these policies may sig- nificantly affect who becomes an engineer. This article reports the outcome of using re- search results to inform change in admission policy at a Midwestern public university.

Purpose There were three research questions: Is there statistically significant evidence of admis- sion decision gender bias for engineering applicants? Do affective and cognitive factors predic- tive of engineering student success differ between men and women? Can a differ- ence in the resulting admitted class be confirmed when such factors inform admission policy?

Design/Method Admissions records were examined for differences in cognitive metrics between men and women. Student records were analyzed before and after the policy change. Neural network modeling of student records predicted the cognitive and affective measures most important for success in retention and graduation.

Results Statistical analysis indicated a gender bias in the admission process results, which was traced back to the policy. Success factor modeling suggested a different set of criteria could mitigate this bias. After admission criteria were changed, statistical analysis confirmed the gender bias against women was mitigated.

Conclusions The application of research and the change process described shows the im- portant role of research in motivating and informing policy change. This work highlights the contribution of institutional bias in admission policy to the underrepresentation of groups in engineering education, especially if admission is limited to a minimum standar- dized math test score.

Keywords gender equity; research-informed policy change; success factor modeling

Overview One of the early steps a student makes along the pathway toward becoming an engineer is apply- ing and being admitted to an accredited institution. While an abundance of literature describes the virtues of precollege and recruiting programs and their potential effect on increasing the number of applications to engineering schools, little to no research informs institutional policy on important factors to consider in the engineering admissions process. This article describes a research-to-practice effort at a U.S. Midwestern public university which began by studying the results of admission practices and ended by changing admissions policy. The genesis of this

Journal of Engineering Education VC 2014 ASEE. http://wileyonlinelibrary.com/journal/jee April 2014, Vol. 103, No. 2, pp. 274–301 DOI 10.1002/jee.20046

 

 

effort was when the authors identified that the number of engineering applications from women increased by 46% over a five-year time period while the number of women admitted during that same time period increased by only 23%. This disparity between application and admissions gains led to an investigation of the university’s engineering admissions process and ultimately policy. When a gender bias was confirmed by statistical analysis, the authors used research-based success modeling to identify key admission factors that could produce a differ- ent result from the university’s engineering admission policy. While most research stops here, the authors were able to use these research findings to influence a process and policy for which they had no direct responsibility. By promoting and reprioritizing researched admissions fac- tors, the number of women admitted to the College of Engineering increased, and mitigation of gender bias was confirmed.

The following is a retrospective of this four-year journey toward research-informed policy change. While this time period seems long, a goal of the authors’ is to encourage others to consider the extra step of using research findings to affect change and improve policies of the engineering education system. In much the same manner as Jamieson and Lohmann (2009) demonstrated the importance of linking research and educational practices, this article demonstrates the possibilities for change when linking research and policy. Using research to inform engineering educational policy could significantly improve the higher education system, given administrators who under- stand the power of applied research and researchers who value and understand the potential of how research-informed policies can positively affect system change.

Introduction The National Academy of Engineering’s (NAE) Changing the Conversation (2008) created an awareness of the public’s perception of engineering in general and of teens’ perceptions of engi- neering more specifically. These perceptions or misperceptions suggest changes needed so that high school students can be more effectively recruited to the engineering field. The national conversation regarding the education of engineers was sparked and re-energized by NAE’s The Engineer of 2020 (2004) and Educating the Engineer of 2020 (2005). But no such similar national conversations have focused on the role of admission policy and how it is either a barrier to or enhances being able to study to become an engineer. Though admission policies can and do vary by institution, admission is a gateway through which all eventual engineers must pass. Changing admission policy, then, may have significant implications for who becomes an engineer. It may also be valuable to modify admission policy to align better with producing the type of engineer who will be successful in the future. However, to know what changes to admission policies can align with producing the types of engineers needed in the future, an understanding of the out- comes of the current policies is needed, as well as an understanding of how changes might impact admission results, and how policy changes can be made.

Viewing the admission process within the larger context of the progression of a potential engineering student through to graduation, as in Figure 1, demonstrates the relationships be- tween applicants, admitted students, and yielded students who enroll in an undergraduate engi- neering education program. From this systems perspective, it is clear that an applicant cannot enroll and be retained through to graduation unless first admitted. From this standpoint, admis- sion policy is situated squarely between recruiting and retention.

If, then, admission policies have a significant role in who does or does not become an engi- neer, changing such policies may play a role in increasing the representation of groups such as women and underrepresented minorities in the engineering education system.

Retrospective on Engineering Admission Policy Change 275

 

 

At a large U.S. Midwestern public university, institutional data indicated that from 2006 through 2010 the number of applications received from women for engineering increased by 46%, a result of increased efforts in recruiting. However, over that same time period, the num- ber of women who were admitted into engineering increased by only 24%, and the number of women who enrolled in engineering increased by only 20%. A mismatch in the growth rates of women applicants and women admitted was unexpected because the university’s College of Engineering had for many years set goals for increasing the number and percentage of women studying engineering. In fact, this college was the first in the United States to create a women in engineering program, which demonstrated its longstanding commitment to increas- ing women’s representation and success in the field. The disparity between application gains and admission gains raised questions about equity within the admission process and the poten- tial effects of admission policy on the underrepresentation of women in engineering, both at this university and in general.

There are many theories of change and of the factors that either promote or impede the change process. The research-to-practice project described in this article illustrates Weick and Quinn’s (1999) theory of episodic change, which relies on Kurt Lewin’s (1951) classic framework of organizational development: unfreeze, change, and refreeze. These three stages were used to frame a retrospective review of the request from the College of Engineering to

Figure 1 Progression of an engineer- ing student from prospect to graduate.

276 Holloway, Reed, Imbrie, & Reid

 

 

the University admissions office to change engineering admission policy that was shown to be biased in favor of men. The communication channels of the change process are particu- larly emphasized; since the change was a policy recommendation, communication was the primary means of effecting change. Although this project was done at a single university, many facets of the process are transferable to other institutions of higher education.

LiteratureReview Ongoing research that seeks to identify reasons for the persistent underrepresentation of women in engineering has focused primarily on two broad areas: recruitment, specifically precollege preparation, exposure, and experience; and retention, specifically higher education experience. A significant body of literature describes programs and practices that have been implemented, and which have incorporated findings from both recruitment and retention research (for example, Bogue & Cady, 2010). Unfortunately, the prevalence of this informa- tion has not significantly increased the number of women engineering graduates in the last 20 years. The lack of change suggests that other factors may be influencing that outcome.

There is a dearth of research about the university admission process and its related policies that could guide the understanding of the degree to which this process and related policies are or are not subject to gender bias. Few studies have specifically investigated the variance that admission policy contributes to the underrepresentation of women in engineering. Margolis and Fisher (2002) found that changes in the admission process and evaluation criteria increased the number of women studying computer science. Unfortunately, that research was limited in scope and did not generalize its findings to engineering education. A review of the literature revealed there is a significant lack of research that critically evaluates engineering admission processes and policies in three areas: gender bias when admission decisions principally depend on typical high school metrics (i.e., standardized test score, high school grade point average [GPA], and high school class rank); gender bias regarding the types of factors (i.e., cognitive and psycho-social or affective and attitudinal factors) used to make admission decisions; and the role of systematic research to inform policy creation or modification. One exception is a study by Leonard and Jiang (1999) that indicated a systemic gender bias against women when SAT (a U.S.-based national standardized test used in college admissions) scores were used to admit students to the University of California, Berkeley in all fields except engineering. They did find, however, that within engineering, those women on the margin of admission accord- ing to their SAT scores outperformed similarly scoring men with respect to their college grades. This general lack of literature suggests engineering admission processes, policy, and cri- teria are closely held by institutions, presumably for competitive reasons. Unfortunately, the lack of scholarly work on engineering admissions promotes keeping these processes and poli- cies unchanged rather than modifying them in an informed way. Indeed, Camara and Kimmel (2005) point out that “most admissions decisions are made using tools that have been around for 50 years or more” (p. viii).

As stated earlier, admission decisions are generally based on the premise that the selection cri- teria used (e.g., standardized test scores, high school rank) will yield the highest degree of student success, where student success is typically operationalized in terms of first-year retention and ulti- mately graduation. However, evidence is emerging from the literature on the affective and attitu- dinal factors that shape our understanding of student success (Artelt, Baumert, Julius-McElvany, & Peschar, 2001; Atkin, Black, & Coffey, 2001; Schreiber, 2002). These affective and attitudi- nal factors have been shown to be positively correlated with college student success, but have not

Retrospective on Engineering Admission Policy Change 277

 

 

been made a substantial part of admissions decisions, although researchers have identified their addition as a possible way to ameliorate the underrepresentation issue (Sedlacek, 2005).

Substantiating a broader set of admission criteria that can lead to improved retention and graduation requires a more thorough understanding of the relationships between attributes and outcomes of first-year students. To date, many studies look in isolation at the individual contri- butions of affective and attitudinal student attributes to student success; these include Biggs, Kember, and Leung (2001), French and Oakes (2001), Hackett, Betz, Casas, and Rocha-Singh (1992), O’Neil and Abedi (1996), Pajares (1996), and K. Taylor and Betz (1983). Numerous articles, such as Moller-Wong and Eide (1997), identify the relationship between critical cogni- tive variables and student success. Unfortunately, little research has used hybrid models that combine both cognitive measures and affective and attitudinal measures. Such models have the potential to provide more insight into factors influencing student success.

Guided by this literature about the contributions of affective and attitudinal attributes to student success, a 161-item instrument was developed that used psychometrically tested affec- tive and attitudinal indicators of student success. The instrument is referred to as the Student Attitudinal Success Instrument, or SASI (Immekus, Maller, Imbrie, Wu, & McDermott, 2005; Reid, 2009; Reid & Imbrie, 2008). Combining the results of the SASI with typical high school cognitive metrics (e.g., standardized test scores, high school GPA, class rank, etc.) forms a hybrid model of student success. The hybrid modeling of student success used in this project provided a tool that could optimize the factors considered during the admission process and set admission policy tailored to engineering students at the studied university.

This project offered the possibility to better understand the admissions process and the results of the policy that guide the process, especially as a potential partial solution to bolster the rep- resentation of women and other minorities in engineering. The following research questions guided this study:

To what extent is there statistically significant evidence of admission decision gender bias for engineering applicants when considering standardized test scores, high school GPA, and class rank?

Do affective and cognitive factors used to predict engineering student success (opera- tionalized as first-year retention and graduation) differ between men and women?

When such factors are used to inform admission processes and policy, can a difference in the resulting admitted and enrolled class demographics be confirmed?

This article also describes the process by which the findings from the first two research ques- tions were used to inform and change engineering admission policy at a Midwestern university.

TheResearch UniversityAdmissionProcess To better understand the context of this study, a brief review of the admission process at the Midwestern university is needed. The university has stated on its Web site that many general factors are used to make a decision on a student’s application, and that all factors are taken into consideration in a holistic manner. These factors are:

subject matter expectations (the number of semesters of math, science, English, social studies, and foreign language that each student is required to have taken in high school)

278 Holloway, Reed, Imbrie, & Reid

 

 

overall high school grade point average (GPA)

core high school GPA (English, math, science, foreign language, and social studies classes)

high school class rank

scores from U.S.-based national standardized tests for college readiness (SAT or ACT)

overall grades in academic coursework

grades related to intended major

strength of student’s overall high school curriculum

trends in achievement

ability to be successful in intended major

personal background and experiences

time of year the application is received

space availability in intended program

Some of these factors are quantitative, such as standardized test scores, high school GPA, and class rank. Many are not, such as ability to be successful in intended major and personal background and experiences. When this research project began in 2008, the application did not require an essay; an essay was added some years later but before the admission policy was changed. The university’s admissions office has stated that there are no minimum requirements for quantitatively measured metrics; such a policy leads to a system that is nonformulaic and flexible, but also makes the process less transparent and more subjective.

A flowchart of the admission process is presented in Figure 2. Every application is first reviewed for completeness in the data processing office. If an application is incomplete, a file is started and a request for additional information is sent to the applicant. If an application is com- plete, there are no irregularities in the application data, and the metrics of the applicant are clearly outstanding, the applicant is admitted. If there are some irregularities, or the metrics of the appli- cant are not as outstanding, the application is forwarded to one of several designated admission counselors for the College of Engineering. The counselor can then either admit the student or send the application on to the college-specific committee with a recommendation on an admis- sion decision. For each college there is a committee, which is comprised of all the admission counselors designated for that college and a senior admission counselor. The committee then comes to a decision about the application. The committee can admit the student to engineering, offer the student an alternate option at the university, deny the student admission, or request additional information from the applicant.

StatisticalAnalysis This effort began when the authors identified that the number of engineering applications from women increased by 46% over a five-year time period while the number of women admitted during that same time period increased by only 23%. A statistical analysis of the quantitative metric data of the applicants and admitted students to the College of Engineer- ing over the five-year period was performed to investigate potential gender bias in the results of the admission process.

Retrospective on Engineering Admission Policy Change 279

 

 

Methods and Results The admissions office stores applicant information in a database. A new database is created for each admissions cycle. Each database contains the demograph- ics of each applicant (including gender, ethnicity, and residency), the cognitive metrics of each applicant (including standardized test scores, class rank, number of semesters of and grades in core courses, and overall and core high school GPAs), and the admissions decision made for each applicant. This research project used data for the 2006–2010 cohort entry years. The data then were filtered to include only records for the following:

applicants with complete applications (incomplete applications were filtered out)

applicants for fall semesters only

applicants who would be admitted directly from high school and would be first-time college students

applicants to the College of Engineering

The overall demographics of these applicants, disaggregated by gender, are given in Table 1. Not all metric data are available for each applicant. For example, since high schools increas-

ingly do not provide rankings of their students, some students did not have a high school class rank. Some international students do not take standardized tests. Not every student took both the SAT and ACT tests. In order to minimize the amount of missing data, all ACT test scores were converted into equivalent SAT (SATe) Math and Verbal scores using the concordance published by the College Board, the administrator of the SAT (Dorans, 1999). Finally, many students take these tests more than once in an effort to improve their scores. This university’s policy with regard to multiple tests is to use the highest scores from each part of the test for consideration in the admission process, as opposed to just using the latest complete set of test data or the highest overall set of test data. Because of this policy, only the maximum test scores were used in the analyses.

Figure 2 Flowchart of admissions process.

280 Holloway, Reed, Imbrie, & Reid

 

 

Anderson-Darling normality tests were run on each metric distribution to determine if a normal probability distribution was adequate to describe the data; each metric was determined not to be normal. Therefore, a nonparametric, 2-sample Mann-Whitney test at a 95% confi- dence level was used. This test can be used to make inferences about the difference between two population medians based on two independent random samples.

An analysis of the aggregate applicant pool is shown in Table 2 and includes the sample size and median value for each metric as well as the p-value and Cohen’s d for each comparison. The scales for each metric are also presented as a range from minimum to maximum possible value. Analyses were completed for each individual cohort year, and the results were similar each year, as demonstrated through cluster analysis. Cluster analysis showed resultant shapes and patterns to be consistent from cohort to cohort, demonstrating strong repeatability and

Table 1 Demographics of Applicants

to Engineering, 2006–2010 Cohorts

Women (n 5 7,884)

Men (n 5 30,856)

n % n %

Race/Ethnicity Caucasian, non-Hispanic 5,016 72.1 19,996 75.8 African American, non-Hispanic 462 6.6 1,135 4.3 Hispanic American 383 5.5 1,243 4.7 Asian American/Pacific Islander 473 6.8 1,730 6.6 Asian American 301 4.3 1,049 4.0 Native American 37 0.5 160 0.6 Native Hawaiian/Pacific Islander 2 0.0 12 0.0 Other 71 1.0 274 1.0 Two or more races 51 0.7 169 0.6 Unknown 140 2.0 506 1.9 Not reported 19 0.3 118 0.4

Residency Domestic 6,955 88.2 26,392 85.5 International 929 11.8 4464 14.5

Table 2 Metric Medians for Applicants

to Engineering, 2006–2010 Cohorts

Women Men

Metric Scale Median n Median n p Cohen’s d

Overall GPA 0.0–4.0 3.9 7,017 3.7 21,357 0.0000 0.42* Core GPA 0.0–4.0 3.75 7,681 3.52 29,459 0.0000 0.48* Class rank 1st299th 94 4,460 87 17,393 0.0000 0.45*

percentile SATe verbal 200–800 620 7,775 600 30,310 0.0000 0.22 SATe math 200–800 680 7,774 680 30,310 20.08 SATe total 400–1600 1,300 7,775 1,290 30,310 0.0000 0.08

*Moderate effect size.

Retrospective on Engineering Admission Policy Change 281

 

 

stability. Similar taxonomies are indicated by a consistent number of core profiles, similar in magnitude and shape as determined by the cluster homogeneity coefficient > 0.6 (Tyron & Bailey, 1970) and Cattell’s (1978) similarity coefficient (similar: rp > 0.95, dissimilar: |rp| < 0.7 [Freedman & Stumpf, 1978]). Values of Cattell’s coefficient comparing clusters expected to be highly similar are consistently 0.94 < rp < 1.00, while those expected to be dissimilar were |rp|< 0.54. Therefore, only the results from the total combined pools are presented here. Simi- larly, previous research has shown gender-based results were the same when evaluating differ- ences between male and female students each year and if taken in aggregate (Reid, 2009; Reid & Imbrie, 2009). For all analyses, a p-value of 0.05 or less was considered to be statistically sig- nificant. In Tables 2 and 3, a statistical significance in the difference in the medians is denoted by a bolded higher median. Because the size of the pool of applications was large (n > 38,000), most differences in median were found to be statistically significant. Therefore, to determine if the differences are also meaningful, Cohen’s d, was used to determine the effective size of the differences. Cohen (1988) originally defined ranges for effect sizes as small, d 5 0.2; medium, d 5 0.5; and large, d 5 0.8; with the caveat that “there is a certain risk inherent in offering con- ventional operational definitions for those terms for use in power analysis in as diverse a field of inquiry as behavioral science” (p. 25). On the basis of subsequent exploration of effect sizes as they apply to research in the social sciences, Hyde (2005; Hyde & Linn, 2006) defined the ranges as part of the gender similarity hypothesis as near-zero, d�0.10; small, 0.11 < d�0.35; moderate, 0.36 < d�0.65; large, 0.66 < d�1.0; and very large, d > 1.0. In the results tables, moderate effect sizes are indicated by one asterisk.

The data for the overall applicant pool (Table 2) show that the medians of the women’s overall GPA, core GPA, class rank, SATe verbal scores, and SATe total scores are statistically higher than those of the men. In terms of effect sizes, the differences between men’s and wom- en’s overall GPA, core GPA, and class rank are moderate; all others are small or near-zero. These same type of results were found of engineering applicants to a small comprehensive regional university located in New Jersey (Cleary, Riddell, & Hartmann, 2008).

Figure 3 shows boxplots of the distribution of overall GPA and SATe math scores of appli- cants by gender. The box represents the middle 50% of the data, and the horizontal line through the box is the median. The vertical lines, often called whiskers, extending from the top and bottom of the box are 1.5 quartiles in length; the dots denote individual datum points out- side of that range. Note that the men have a much wider data spread and longer tails, especially on the lower end. Figure 3 data clearly indicate that men with lower high school GPAs apply

Table 3 Metric Medians for Students Admitted

to Engineering, 2006–2010 Entry Cohorts

Women Men

Metric Median n Median n p Cohen’s d

Overall GPA 4.0 4,937 3.8 20,131 0.0000 0.38*

Core GPA 3.8 6,763 3.62 22,748 0.0000 0.42*

Class rank 95 3,991 91 12,520 0.0000 0.35*

SATe verbal 640 6,699 630 22,511 0.0000 0.14 SATe math 690 6,699 710 22,511 0.0000 20.28 SATe total 1,320 6,699 1,330 22,511 0.0000 20.07

*Moderate effect size.

282 Holloway, Reed, Imbrie, & Reid

 

 

F ig

u re

3 D

at a

d is

tr ib

u ti

o n

o f

(A )

o ve

ra ll

G P

A an

d (B

) S

A T

e m

at h

sc o re

s fo

r ap

p li

ca n

ts to

en g in

ee ri

n g ,

2 0 0 6 – 2 0 1 0

co h

o rt

s.

Retrospective on Engineering Admission Policy Change 283

 

 

for admission to engineering, whereas similar women do not apply. Data distributions for high school core GPA, high school class rank, SATe verbal scores, and SATe total scores are similar to those shown in Figure 3 and are not presented here.

An analysis of the pool of students admitted to engineering is shown in Table 3. This table includes the sample size and median value for each metric as well as the p-value and Cohen’s d for each comparison. The medians of the women’s overall GPA, core GPA, class rank, and SATe verbal scores are statistically higher than those of the men. The medians of the men’s SATe math and SATe total scores were statistically higher than the median of the women’s. In terms of effect sizes, the differences between men’s and women’s overall GPA, core GPA, and class rank are moderate; all others are small or near-zero.

The boxplots in Figure 4 show data point distributions of overall GPA and SATe math scores for the men and women admitted to engineering. Note that the men have a much wider data spread and longer tails, especially on the lower end for overall GPA. This characteristic of the distribution is also present in the overall application distributions (Figure 3).

Discussion It is not surprising, perhaps, to see gender-based differences in the overall population of applicants to engineering because the university and its admissions office have limited influence over the population of who applies. In an ideal admission process, there should be no expectation that there will be significant differences in the metrics of the men and women, particularly in institutionally defined populations, that is, those that the admis- sions office controls, such as the admitted student population.

This university states that it uses standardized tests as part of its admission criteria. Admis- sion counselors typically consider standardized test scores when estimating the applicant’s like- lihood of academic success in college.

The literature indicates that standardized test scores, however, are not as good at predict- ing student success in college as high school metrics and are gender-biased. Research pub- lished by the College Board, the administrator of the SAT, has indicated that a student’s high school grades and class rank are a better predictor of first-year college grades than a student’s SAT score (Burton & Ramist, 2001; Morgan, 1989). Bowen, Chingos, and McPherson’s (2009) analyses demonstrate that high school grades are “extremely strong predictors of graduation rates even when we cannot (or do not) take account of the characteristics of the high school attended” (p. 123).

A consistent gender bias in standardized tests has been found in 37 studies (Young & Kobrin, 2001). In particular, Wainer and Steinberg (1992) found that men score 35 points higher on the SAT math section than women who earn the same grades in the same college math courses. Sources of test bias are extremely difficult to identify, and once identified, test instruments are typically corrected. Given these sets of research data as a backdrop, one might expect to see no gender differences in the metrics of students’ high school records and class ranks. One might also expect that the math standardized tests scores of women would be lower at the same high school metric level. However, the statistical analysis presented above indicates that, across the board, the women have higher high school metrics than do the men. Without direct knowledge of the thoughts of admission counselors, and without a written policy of the weighting of each of the admissions criterion, possible explanations for the gender differences in the admitted student metrics could include one or more of the following: only the highest ability women are encouraged or self-select to apply to engineering, and men with a much wider range of academic ability are encouraged or self-select to do so; women are held to a higher standard than men with regard to their high school performance; or the admission counselors put more weight on standardized test scores than on high school performance in the admission process.

284 Holloway, Reed, Imbrie, & Reid

 

 

F ig

u re

4 D

at a

d is

tr ib

u ti

o n

o f

(A )

o ve

ra ll

G P

A an

d (B

) S

A T

e m

at h

sc o re

s fo

r ad

m it

te d

st u d

en ts

to en

g in

ee ri

n g ,2

0 0 6 – 2 0 1 0

co h

o rt

s.

Retrospective on Engineering Admission Policy Change 285

 

 

Each of these potential explanations represents a different type of bias. The first explana- tion, that only the highest ability women are encouraged or self-select to apply for engineering, is likely due to a combination of prevailing gender schemas that every person holds about what is appropriate for men and women and the stereotypes of what and who engineers are and are not. Gender schemas are a set of implicit hypotheses about sex differences that shape both men’s and women’s expectations and evaluations of men and women (Valian, 1998, p. 2). Both sexes tend to hold the same gender schemas and expect women to be caring, nurturing, and expressive. Both men and women expect men to be competent and independent. Because men are assumed to be competent (perhaps even beyond what they have demonstrated in high school), and because men tend to better fit the stereotypes of who and what engineers are, they are encouraged to think about engineering as a career choice at a higher rate than do women, who do not fit the engineering stereotype as well as men. Challenging the stereotypes of who engineers are, why people become engineers, and what engineers do is the subject of a recent National Academy of Engineering study (2008), and should be considered in the recruiting process. While challenging these stereotypes in the recruiting process will not affect the admis- sion process, it could certainly affect the overall applicant pool.

The second potential bias explanation, that women are held to a higher standard, is also likely due to the gender schemas that every person holds. The admission counselors at the uni- versity work very hard to make thoughtful admissions decisions. It is assumed that they would not consciously say or think that women need to have demonstrated a greater ability or compe- tency in order to be admitted to engineering. But according to Virginia Valian, “In situations where we evaluate the professional competence of men and women, and where there is much room for interpretation, men will have significant advantage due to unconscious assumptions. Our schema for men is a better fit for professional success, and especially for high-intensity sci- entific and engineering careers” (cited in Sevo & Chubin, 2008). Since the admissions decision process for this university’s College of Engineering is just such a situation, gender schemas could be a reason the women’s high school metrics are higher than men’s.

A third type of bias, institutional bias, results if admission counselors put more weight on standardized test scores than on high school performance in the admission process. If an insti- tution’s policy or tradition (written or otherwise) requires a certain level of achievement on a test that is known to disadvantage a certain group, institutional bias exists (Valian, 1998). Insti- tutional bias can also be unintended in that it is more tradition than policy. This unintentional- ity may be particularly true in institutions that have made concerted efforts to raise the average SAT scores of the incoming classes in recent years, such as at this studied university. In sum, there is statistically significant evidence of admission decision gender bias for engineering applicants when considering standardized test scores, high school GPA, and class rank. This conclusion leads to a study of this article’s second research question.

Modeling A discussion of biased academic admissions practices leads to the question of what are appro- priate indicators of student success. The difficulty of this discussion is that historically the fac- tors used to answer this question have been cognitive, such as standardized test scores and high school GPA. The mention of modeling to discuss admission policies can elicit strong concern that profiling will result and may limit admission to certain populations. Weinstein et al. (2001) acknowledge this modeling controversy and relate it to a misconception that “the role of models is to establish truth rather than to guide clinical and policy decisions” (p. 348); they

286 Holloway, Reed, Imbrie, & Reid

 

 

provide examples of public policy domains from areas such as environmental protection or defense strategy that involve human life and health where models are generally accepted as decision aids. Therefore, this research project extended the same use of modeling in student retention to advise and inform admission policies at the studied university.

Method To model student success, the research project used results of the Student Attitudinal Success Instrument (SASI) instrument combined with typical high school cognitive metrics. The SASI, a 161-item survey assessing nine specific affective and attitudinal constructs was devel- oped largely on the basis of existing instruments (Immekus et al., 2005; Reid, 2009; Reid & Imbrie, 2008) and was shown to be sex invariant (Reid & Imbrie, 2009). The SASI is designed to provide data on affective and attitudinal characteristics for incoming engineering students before their first year and which higher education institutions may influence during students’ first year. Data collected from this instrument may be suitable for use in the development of predictive models of student retention and graduation, which is the definition of success used in this project.

The SASI provides the College of Engineering with the attitudinal and affective character- istics of incoming first-year engineering students, which is information beyond the cognitive characteristics typically available on most college applications. The College of Engineering at the studied university administers it to all incoming engineering students and completion is a requirement prior to entrance advising. Such systematically gathered information helps the college assess the effect of institutional and programmatic decisions aimed at recruitment, ad- mission, retention, and success of all students and, in particular, underrepresented students, in- cluding women and other minorities.

The hybrid SASI model of student success includes nine self-reported affective factors (lead- ership, deep- and surface-learning types, team or individual orientation, academic self-efficacy, motivation, metacognition, expectancy value, and major indecision) and eight academic prepa- ration items from high school, including standardized test results by subarea; average grades in mathematics, science, and English; and the number of semesters completed of mathematics, science, and English. Since males and females in science, technology, engineering, and mathe- matics (STEM) have been reported to differ in terms of their self-beliefs (Brainard, Laurich- McIntyre, & Carlin, 1995), a measure of self-beliefs was included as well. Table 4 provides details of the origins of each factor along with its label used in Figure 5.

Results In prior work by Reid (2009), women were found to be similar to men in the way they answered each attitudinal and affective scale (that is, near-zero or small effect sizes), but when these same factors were used to model success by gender, there were real differences between men and women. Analysis by Lin, Imbrie, Reid, and Wang (2011) further illustrated the difference in the importance of cognitive and affective characteristics in the development of models to predict retention. In the present study, modeling was completed for first-year retention and graduation after four years (eight semesters), five years (10 semesters), and six years (12 semesters) for the cohort of students entering engineering in 2004. Due to factors such as the number of credit hours to obtain an engineering degree or a student’s participation in cooperative learning (one semester in school alternated with one semester in industry), average graduation rates in the United States are just over four years, thus the interest of this project in longer times-to- graduation. Figure 5 shows, by gender, the model’s most important factors used to predict success,

Retrospective on Engineering Admission Policy Change 287

 

 

where success is operationalized as one-year retention, eight-semester graduation, 10-semester graduation, or 12-semester graduation, for one cohort of students entering in 2004 for men (n 5 1,196) and women (n 5 286). Again, the factor data (the independent variables) were col- lected just before students entered the college. The importance of a particular factor towards predicting the outcome variable (success) is indicated by the radial distance from the center of the circle, represented by normalized weighting factors (center 5 low importance; perimeter of the circle 5 high importance). For example, in the “1 Year Retention” radial plot in Figure 5, leadership is an important attribute for women’s success (defined as retention) at the first-year level. For men, the semesters taken of high school mathematics is important. Semesters of sci- ence taken in high school is a factor important to both men’s and women’s success. Since the cohort remains the same for all four plots, the results indicate that factors important to predict success for women and men are not the same and that important factors change with the mea- sure of success. Since the model is used for predictive purposes rather than as an explanatory tool, the authors cannot explain why the important factors differ by gender.

It is important to note that the success factors that best predict the positive retention and graduation of men are those that are traditionally used in admitting students: math standardized test scores and previous coursework in math and science. These factors contrast to unique factors that best predict the positive retention and graduation of women: motivation, propensity toward deep learning, and self-perception of leadership ability. Also, due to the legal environment in the United States, while admission counselors can and should use a holistic mix of factors in directing admissions and scholarship decisions, different factors cannot be used for men and women or majority and minority students, even if these factors were based on known success factor data for each population (American Association for the Advancement of Science & Association of

Table 4 Sources Consulted for

Factors in Development of SASI

Input factors (with references) Label

Affective and attitudinal factors Team vs. Individual Orientation* (McMaster, 1996) TeamInd Academic Self-efficacy* (Bandura, 1986; Pajares, 1996) Efficacy Motivation (French & Oakes, 2001; Pintrich & Schunk, 1996) Motivation Major Indecision (Osipow, 1999) Major Leadership* (Hayden & Holloway, 1985) Leader Surface Learning (Biggs et al., 2001) Surface Deep Learning (Biggs et al., 2001) Deep Metacognition (O’Neil & Abedi, 2000; Pintrich & De Groot, 1990) Meta Expectancy Value (Wigfield & Eccles, 2000) Expect

High school performance factors SAT/ACT verbal score SAT_V SAT/ACT math score SAT_M Semesters of English taken in high school SEM_ENG Average grades of English in high school AVE_ENG Semesters of math taken in high school SEM_MATH Average grades of math in high school AVE_MATH Semesters of science taken in high school SEM_SCI Average grades of science in high school AVE_SCI

*Developed internally based upon the cited reference(s).

288 Holloway, Reed, Imbrie, & Reid

 

 

F ig

u re

5 S

A S

I m

o d

el o f

su cc

es s

fa ct

o rs

b y

g en

d er

fo r

th e

2 0 0 4

co h

o rt

.

Retrospective on Engineering Admission Policy Change 289

 

 

American Universities, 2010). Efforts to apply identical admission criteria to every group can lead to selection criteria benefitting certain population(s); identifying the benefitted popula- tion(s) creates the opportunity for change in the higher education system that may increase access to an engineering education for those who have been traditionally underrepresented.

In summary, these analyses indicate that affective and cognitive factors used to predict engi- neering student success differ between men and women.

ChangeProcess By proposing a change in admission criteria, a change in university admission policy was ini- tiated. The resulting change process can be understood retrospectively in terms of the change process described by Weick and Quinn.

Policy change is often seen as having a slower rate of adoption than a physical innovation (Rogers, 2003, p. 13). Rogers notes that innovations are typically tangible and are often mate- rial objects. However, some important innovations are informational in nature, and these types typically have a lower degree of observability as well as a higher degree of difficulty in traceabil- ity. Educational policy change certainly fits this type of innovation.

Weick and Quinn (1999) metaphorically describe episodic change as infrequent, discontin- uous, and intentional with the pattern or sequence described as unfreeze-transition-freeze. The process through which the admission policy was changed could certainly be described as episodic. Weick and Quinn describe the process:

The presumption is that episodic change occurs during periods of divergence when organizations are moving away from their equilibrium conditions. Divergence is the result of a growing misalignment between an inertial deep structure and perceived environmental demands. This form of change is labeled ‘episodic’ because it tends to occur in distinct periods during which shifts are precipitated by external events such as technology change or internal events such as change in key personnel. (p. 365)

In the admission change process, the divergence could be described as a misalignment between an admission policy that results in lower than expected increases in the number and percentage of women admitted in the presence of ever increasing applications from women and the value that the leadership of the engineering college placed on the diversity of its entering class. The following narrative not only describes the path of this university’s admission policy change, but allows others to consider what path a similar policy change could take within their institution.

Using Weick and Quinn’s (1999) idea of episodic change, which builds on the premise that “change is Lewinian: inertial, linear, progressive, goal seeking, motivated by disequilibrium, and requires outsider intervention” (p. 366), the first stage is to unfreeze the organization. Based on concern noted above that applications of women were increasing at a rate almost dou- ble that of the admissions of women, a subset of the authors initiated the analysis described in the Statistical Analysis section above (Research Question 1). Once bias was confirmed, the research focus moved to what alternate factors could be used in the admission process to miti- gate this bias, leading to the success factor modeling described in the Modeling section above (Research Question 2). As noted by Weick and Quinn (1999), episodic change requires inter- vention from those outside the system. At this university, the results of both the statistical admissions analysis and the modeling analysis were presented in early 2010 to the members of the Diversity Action Committee (DAC), a standing committee within the engineering college dedicated to improving the college’s diversity climate. The committee recommended that these

290 Holloway, Reed, Imbrie, & Reid

 

 

results be brought forward to the dean of engineering, who saw their value and became one of the strongest champions for change in the university’s admission policy, and played the role of change agent for the remainder of the process.

The communication processes, according to Rogers (2003), are very important in the adoption of a new idea. Through the lens of Rogers’s framework, communication was easy and straightfor- ward with the DAC and the dean because the communication was between homophilious groups, that is, groups more alike than different. Inasmuch as all involved in the communication process to that point were dedicated to the goals of increasing the representation of women in engineer- ing, were affiliated with the college, were engineers and researchers, and believed in the power of data interpretation and research to glean new knowledge, these groups were homophilious.

AdvocatingAdmissionPolicyChange With the full support of the dean of engineering, who was the primary change agent, and in her presence, the results of both the statistical admissions analysis and the modeling analysis were then presented to the leadership of the admissions office. These communications, i.e., the pre- sentation of these results, were not as straightforward or as well received as they were internally within the college. Even though the admission counselors and admission leadership had worked with the college for many years, their perspectives on the admission process and informational frameworks were different. They were responsible for balancing admission across the university and were much more familiar with the legal environment in regard to admissions practices than the researchers and administrators in the college. Due to these responsibilities, they were hesi- tant to recognize the possibility that current policy may be biased toward men and were cautious about any modifications or changes. Weick and Quinn (1999) call this stage of episodic change the transition. The cognitive restructuring of admissions factors as they related to engineering students along with implanting a new standard of judgment are characteristics of this stage.

The research was also presented several times to leadership in the provost’s office in an effort to explain to the university’s higher administration the college’s desire to modify the admission policy and practices used for admission to the college of engineering. At the time of these pre- sentations, the key persons in the provost’s office, including the provost, were engineers; thus the process once again benefitted from homophilious communication.

The next stage in Weick and Quinn’s episodic change is refreezing the change, or creating a new normal state. As it applied here, a new normal state would mean using the proposed admission criteria recommendations from the college to guide admission decisions. Unless mechanisms are enacted to prevent reversion, however, change is often only temporarily achieved (Weick & Quinn, 1999). At this point in the change process, the admissions office requested from each of the univer- sity’s colleges a formal identification of what student admission factors were important to each spe- cific college in building their incoming class. This formal request for important admission factors was made at the direction of the provost’s office, in an effort to create more of a college-level voice in the admission process. The researchers, with the dean’s strong encouragement and support, used this opportunity to use the results of the success-factor modeling research to guide the criteria rec- ommendations to the admissions committee. These recommendations placed a higher emphasis on the affective indicators that were shown to be more predictive of success for women. Thus, it was recommended that admission decisions be based on a set of priorities that included a reduced emphasis on standardized math test scores, and a stronger emphasis on the following cognitive fac- tors: verbal or written scores on standardized tests and number of semesters of mathematics, sci- ence, and English taken in high school. The recommendations also included a strong emphasis on affective indicators such as leadership, major indecision, and academic motivation, to the extent

Retrospective on Engineering Admission Policy Change 291

 

 

that these factors could be determined from the application. In addition, based on reports such as the National Academy of Engineering’s Changing the Conversation (2008), social relevance of engi- neering as a discipline was added to the recommended criteria. Figure 6 presents an excerpt from the form that was institutionalized to refreeze the change and is now sent from the admissions office to the leadership in each college each summer. The importance of each factor from the col- lege’s perspective is boxed; comments from the college are also shown.

Moreover, in addition to being episodic, the admission policy change can be defined as a second-order change. That is, the admission policy process shifted from a concentrated power model in which the admissions office made all decisions, to a distributed power struc- ture where each college was asked for input into their own admission criteria.

Three connections between politics and change are highly interrelated (S. Taylor, Rizvi, Lingard, & Henry, 1997) and were seen in this admission policy change process. The first connection is external pressures and the context that drives the perceived need for change. The external pressures at the studied university in part consisted of the national as well as the local continued underrepresentation of women in engineering. The second connection is the inter- nal dynamics of the change, and the role of the leadership and strategies to facilitate the change. At the studied university, the leadership of the dean of the College of Engineering and the strategies suggested by the provost’s office were critical to facilitate admission policy change. The third con- nection is the institutionalization of change as expressed through a dialectic between external pres- sures and internal dynamics (S. Taylor et al., 1997, pp. 162–163). Ultimately, political conditions affect how policy change is implemented, given the structural location of the key players in an organization, the approach taken to implement policies, as well as the processes of resistance, marginalization, and co-option that change frequently invokes (S. Taylor et al., 1997, p. 169).

Figure 6 Excerpt of Admission Criteria Request Form with selections boxed and comments entered.

292 Holloway, Reed, Imbrie, & Reid

 

 

Within episodic change, the role of the change agent is that of the prime mover. It is impor- tant to note that the change agents in this process at this particular institution (the researchers, the DAC, the dean of engineering) were not and are not the decision makers in admission pol- icy. Weick and Quinn (1999) state that the process used by change agents “focuses on inertia and seeks points of central leverage” (p. 366) with the objective of changing meaning systems by communicating alternative schema, such as use of the hybrid success modeling results, while building coordination and commitment, such as the support of the provost’s office.

ConfirmationofChange Implementation of the new admissions criteria affected the admission cycle for the class enter- ing in fall 2011. It is important to note that identical admission criteria were used for all stu- dents evaluated, regardless of gender, based upon advice received from legal counsel. The results of the admission cycle showed that while female applicants to engineering increased 11% over the previous year, the number of women admitted to engineering increased by 19%. For the fall 2011 entering class, 26% of the students enrolled in engineering were female, up from 21% the year previous. The number of women in the first-year engineering class increased by 28% to 466, up from 384 in fall 2010. The percentage of women among the U.S. domestic entering class of students was 29%, while the percentage of women among the international entering class was 17%. This difference is significant because international and domestic admis- sions are managed by two separate offices at this university, and the admission policy recom- mendations were given only to the office that processes domestic admissions.

The admissions criteria recommendations remained unchanged for fall 2012. Although there was not another increase in the number of female admits to engineering, enrollment results from the fall 2012 entering class show that 27% of the class was female, and the num- ber of women in the first-year class increased to 477. The percentage of women among the U.S. domestic entering class was 28%, while the percentage of women among the interna- tional entering class increased to 22%. Interestingly, both the admissions statistical analysis and the success factor modeling research were presented to the office that is responsible for international student admissions during the fall 2012 admissions cycle.

Regarding the change process, confirmation included validating that a change occurred in the admission policy. One way to validate the change was to look at the overall composition of the admitted and enrolled classes, as was discussed above. Another way was to replicate the sta- tistical analysis of the admissions data for the 2011 and 2012 admissions years. Table 5 presents

Table 5 Metric Medians for Applicants to Engineering, 2011–2012 Cohorts

Women Men

Metric Median n Median n p Cohen’s d

Overall GPA 4.0 3,199 3.8 11,032 0.0000 0.38* Core GPA 3.79 3,735 3.67 11,655 0.0000 0.31 Class rank 95 1,857 89 6,160 0.0000 0.44* SATe verbal 630 3,922 620 14,168 0.0000 0.19 SATe math 700 3,922 710 14,169 0.0000 20.13 SATe total 1,300 3,922 1,300 14,168 0.04

*Moderate effect size.

Retrospective on Engineering Admission Policy Change 293

 

 

the results of a statistical analysis of the metric medians for the 2011 and 2012 combined appli- cant pool.

These data for the aggregate applicant pool show, in general, that the applicant pools before and after the policy change are similar. The medians of the women’s overall GPA, core GPA, class rank, and SATe verbal scores are statistically higher than those of the men. In terms of effect sizes, the difference between men’s and women’s overall GPA and class rank are moderate-sized effects, similar to the 2006 through 2010 entry years; all others are small or near-zero.

Figure 7 shows boxplots of the distribution of overall GPA and SATe math scores of appli- cants by gender. Men still have a much wider data spread and longer tails, especially on the lower end. Figure 7 data clearly show that men with a wider spread of high school metrics con- tinue to apply for admission to engineering, whereas similar women do not apply. A median line through the box representing the data distribution for women’s overall GPA is not visible because the median of overall GPA for women applying to engineering is a 4.0 GPA; that is, at least half of the women who applied to engineering had an overall GPA of a 4.0.

An analysis of the pool of students admitted to engineering is shown in Table 6, and includes the sample size for each metric, the median for each metric, and the p-value and Cohen’s d for each comparison. As before, the medians of the women’s overall GPA, core GPA, and class rank are statistically higher than those of the men. The medians of the men’s SATe math and SATe total scores were statistically higher than the median of the women’s. However, note the changes in terms of effect sizes; the only moderate effect size is the difference between the men’s and women’s SATe math score; all others are now small or near-zero effects.

The boxplots in Figure 8 show data point distributions of overall GPA and SATe math scores for the men and women admitted to engineering. The men continue to have a wider data spread and longer tails, especially on the lower end for overall GPA, although the dif- ference is not as apparent as in the 2006 through 2010 entry years. Women have a wider data spread and longer tails for SATe math score.

In summary, the analysis of the admissions data for the 2011 and 2012 entry years as compared to the 2006 through 2010 entry years confirms that there was a change in policy, evidenced by the difference in distributions of data in the admitted pools and the changes in effect sizes. Interest- ingly, the median SATe math score for admitted men is now 40 points higher than the median SATe math for admitted women, while the differences in the medians reflecting high school per- formance (overall GPA, core GPA, and class rank) are now smaller. That 40-point differential is strikingly similar to the amount by which Wainer and Steinberg (1992) found men to be

Table 6 Metric Medians for Students Admitted to Engineering, 2011–2012 Cohorts

Women Men

Metric Median n Median n p Cohen’s d

Overall GPA 4.0 3,199 3.9 11,032 0.0000 0.20 Core GPA 3.8 3,735 3.71 11,655 0.0000 0.25 Class rank 96 1,857 93 6,160 0.0000 0.22 SATe verbal 650 3,922 640 14,168 0.0440 0.02 SATe math 700 3,922 740 14,169 0.0000 20.43* SATe total 1,340 3,922 1,350 14,168 0.0000 0.21

*Moderate effect size.

294 Holloway, Reed, Imbrie, & Reid

 

 

F ig

u re

7 D

at a

d is

tr ib

u ti

o n

o f

(A )

o ve

ra ll

G P

A an

d (B

) S

A T

e m

at h

sc o re

s fo

r ap

p li

ca n

ts to

en g in

ee ri

n g ,

2 0 1 1 – 2 0 1 2

co h

o rt

s.

Retrospective on Engineering Admission Policy Change 295

 

 

F ig

u re

8 D

at a

d is

tr ib

u ti

o n

o f

(A )

o ve

ra ll

G P

A an

d (B

) S

A T

e m

at h

sc o re

s fo

r ad

m it

te d

st u d

en ts

to en

g in

ee ri

n g ,2

0 1 1 – 2 0 1 2

co h

o rt

s.

296 Holloway, Reed, Imbrie, & Reid

 

 

advantaged on the math portion of the SAT (35 points). Taken together with the now-small effect sizes in the differentials of men’s and women’s high school performance metrics, it would seem that the bias towards men in the engineering admission process is now significantly lessened.

ConfoundingFactors Obviously, these changes to admission policy did not occur in isolation. Academia, like most other organic systems, has many factors that change simultaneously and confound analyses. Other notable changes at the university coincided with the request to change the admissions criteria. These additional changes primarily affected the percentage of students admitted who subsequently enroll but not the composition of the admitted class. For example, the opportu- nity to control awarding of scholarships at the college level was a change made for fall 2011 as well. While this opportunity most likely affected the percentage of admitted women who enrolled, it did not affect the composition of those admitted.

FutureWork A study of the differences in the academic performance and self-reported affective measures between the women who enrolled in engineering in fall 2011 and fall 2012 and those who enrolled in previous years will be investigated. It is clear that in addition to knowledge of demo- graphic results, an understanding of the consequences of the change in the admission policy on the characteristics of students enrolled is necessary to fully analyze the impacts of changes made to admissions procedures. One study will be to track the retention of these classes as they pro- gress. The fall 2011 and fall 2012 cohorts have shown a slight increase in first-year retention, which warrants continued future study.

Transferability The retrospective presented here may have limitations of transferability to differing types of institutions. This work was done at a state public university, that is majority white, had a Carnegie classification of RU/VH (research university, very high research activity) and L4/R (large four year, primarily residential) (Carnegie Foundation for the Advancement of Teach- ing, 2013), and is classified as “more selective” by U.S. News & World Report (2013). There- fore, direct transfer of all of the success factors found in this study is not necessarily advisable or possible. Even for institutions where admissions is done by the engineering college itself, some of these same institutional biases found in this study may exist, especially if there is a heavy weighting of SAT math score or its equivalent in admission decisions. Other institutions can consider this retrospective as an example of how understanding one’s own institutional data and implications of related policies may be affecting the admission classes at any particular institution. Success models built on the local setting are ideal and most appropriate, but these types of models take many years of longitudinal data and large sample sizes to be statistically significant, and are thus difficult to build. Therefore, the message is one of data-driven change. Knowing the institution’s data provides insight to making future innovative policy changes. Even in the case of open-access admission institutions, modeling success factors of engineering students could aid in studying either recruiting initiatives or advising policies upon student entrance. Through informal conversations, the authors are aware of three major U.S. colleges of engineering that were inspired by this research and made similar admission policy changes that also resulted in an increased percentage of women in the admitted class. All have the same Carnegie and U.S. News & World Report designations.

Retrospective on Engineering Admission Policy Change 297

 

 

Conclusion The impact of this research on the change of the admission policy at this university is clear and has resulted in the admission of more women to engineering. There are also implica- tions for a broader impact on transforming the system of higher education in engineering by focusing on a part of the system (admissions) which has, for the most part, been overlooked. Statistical processes can be used to probe for the possibility of admission biases toward or against particular populations of interest. In the modeling of affective measures, a means of describing students and focusing on successful attributes has been identified. An appropriate criticism of modeling stems from the concern that historically underrepresented students may be marginalized in the results, because they are present in lower numbers and thus have little to no effect on the outcome of an overall model, which then may be used to inform policy or programmatic decisions. This research project confirms the need to consider histori- cally underrepresented populations individually. Other institutions can use these same tech- niques to address the composition of their own student bodies and create policies and programs for admission, student success, and retention.

More generally, the research and change process described clearly establishes the impor- tance of the role of research in policy change. In much the same manner as Jamieson and Lohmann (2009) demonstrated the importance of linking research and educational practices, this article demonstrates the possibilities for change when linking research and policy. The use of research to inform engineering educational policy could have a significant impact on the higher education system, given administrators who understand the power of applied re- search and researchers who value and understand the potential of how research informed poli- cies can affect system change.

Acknowledgments The researchers wish to acknowledge the support provided by a grant from the National Sci- ence Foundation, Division of Engineering Education and Centers (EEC-0416113). We also would like to acknowledge the significant contributions of Dr. Joe J. J. Lin of Texas A& M University and Dr. Qu Jin of Stanford University.

References American Association for the Advancement of Science & Association of American

Universities. (2010). Handbook on diversity and the law: Navigating a complex landscape to foster greater faculty and student diversity in higher education. Washington, DC: Author.

Artelt, C., Baumert, J., Julius-McElvany, N., & Peschar, J. (2001). Knowledge and skills for life: First results from the OECD Programme for International Student Assessment (PISA) 2000. Retrieved from http://hdl.voced.edu.au/10707/18123

Atkin, J. M., Black, P., & Coffey, J. (Eds.). (2001). Classroom assessment and the national science education standards. Washington, DC: National Academies Press.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Biggs, J., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133–149.

298 Holloway, Reed, Imbrie, & Reid

 

 

Bogue, B., & Cady, E. (Eds.). (2010). Olio: Research on women in science & engineering. Retreied from http://www.lulu.com/shop/barbara-bogue-and-elizabeth-t-cady/olio/ebook/ product-18589224.html

Bowen, W. G., Chingos, M. M., & McPerson, M. S. (2009). Crossing the finish line: Ccom- pleting college at America’s public universities. Princeton, NJ: Princeton University Press.

Brainard, S. G., Laurich-McIntyre, S., & Carlin, L. (1995). Retaining female undergraduate students in engineering and science: 1995 annual report to the Alfred P. Sloan Founda- tion. Journal of Women and Minorities in Science and Engineering, 2(4), 255–267.

Burton, N. W., & Ramist, L. (2001). Predicting success in college: SAT studies of classes gradu- ating since 1980 (College Board Report 2001-2). New York, NY: College Board.

Camara, W. J., & Kimmel, E. W. (Eds.). (2005). Choosing students: Higher education admis- sions tools for the 21st century. Mahwah, NJ: Lawrence Erlbaum.

Carnegie Foundation for the Advancement of Teaching. (2013). The Carnegie Classification of Institutions of Higher Education. Retrieved from www.classifications.carnegiefoundation.org/

Cattell, R. B. (1978). The scientific use of factor analysis in behavioral and life sciences. New York, NY: Plenum Press.

Cleary, D., Riddell, W., & Hartmann, H. (2008). Analysis of applicant data to improve recruitment of female and underrepresented engineering students. Proceedings of the 2008 ASEE Annual Conference and Exposition, Pittsburgh, PA.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Dorans, N. J. (1999). Correspondences between ACT and SAT I scores (College Board Report 99-1). New York, NY: College Board.

Freedman, R., & Stumpf, S. (1978). What can one learn from the learning style inventory? Academy of Management Journal, 21(2), 275–282.

French, B. F., & Oakes, W. (2001). Measuring academic instrinsic motivation: Evidence of reliability and validity of a new instrument. Paper presented at the Annual Meeting of the American Education Research Association, Seattle, WA.

Hackett, G., Betz, N. E., Casas, J. M., & Rocha-Singh, I. A. (1992). Gender, ethnicity, and social cognitive factors predicting the academic achievement of students in engineer- ing. Journal of Counseling Psychology, 39(4), 527– 538.

Hayden, D., & Holloway, E. (1985). A longitudinal study of attrition among engineering students. Journal of Engineering Education, 71(7), 664–668.

Hyde, J. S. (2005). The gender similarity hypothesis. American Psychologist, 60(6), 581–592. Hyde, J. S., & Linn, M. (2006). Gender similarities in mathematics and science. Science,

314(5799), 599–600. doi:10.1126/science.1132154 Immekus, J., Maller, S., Imbrie, P. K., Wu, N., & McDermott, P. (2005). Work in pro-

gress: An analysis of students’ academic success and persistence using pre-college factors. Proceedings of the 35th ASEE/IEEE Frontiers in Education Conference, Indianapolis, IN. doi:10.1109/FIE.2005.1612212

Jamieson, L. H., & Lohmann, J. R. (2009). Creating a culture for scholarly and systematic inno- vation in engineering education: Ensuring U.S. engineering has the right people with the right talent for a global society. Washington, DC: American Society for Engineering Education.

Leonard, D. K., & Jiang, J. (1999). Gender bias and the college predictions of the SATs: A cry of despair. Research in Higher Education, 40(4), 375–407.

Lewin, K. (1951). Regression, retrogression, and development. Field Theory in Human Sci- ence. New York, NY: Harper.

Retrospective on Engineering Admission Policy Change 299

 

 

Lin, J. J., Imbrie, P. K., Reid, K. J., & Wang, J. (2011). Work in progress – Modeling aca- demic success of female and minority engineering students using the Student Attitudinal Success Instrument and pre-college factors. Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference, Rapid City, SD. doi:10.1109/FIE.2011.6143044

Margolis, J., & Fisher, A. (2002). Unlocking the clubhouse: Women in computing. Cambridge, MA: MIT Press.

McMaster, J. (1996). Desired attributes of an engineering graduate. Paper presented at the AIAA Advanced Measurement and Ground Testing Technology Conference, New Orle- ans, LA.

Moller-Wong, C., & Eide, A. (1997). An engineering student retention study. Journal of Engineering Education, 86(1), 7–15.

Morgan, R. (1989). Analysis of the predictive validity of the SAT and high school grades from 1976 to 1985 (College Board Report 89-1). New York, NY: College Board.

National Academy of Engineering (NAE). (2004). The engineer of 2020: Visions of engineer- ing in the new century. Washington, DC: National Academies Press.

National Academy of Engineering (NAE). (2005). Educating the engineer of 2020: Adapting engineering education to the new century. Washington, DC: National Academies Press.

National Academy of Engineering (NAE). (2008). Changing the conversation: Messages for improving public understanding of engineering. Washington, DC: National Academies Press.

O’Neil, H., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory: Potential for alternative assessment. Journal of Educational Research, 89, 234–245.

Osipow, S. (1999). Assessing career indecision. Journal of Vocational Behavior, 55(1), 147– 154.

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543–578.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning compo- nents of classroom academic performance. Journal of Educational Psychology, 82, 33–40.

Pintrich, P. R., & Schunk, D. H. (1996). Motivation in education: Theory, research, and appli- cations. Englewood Cliffs, NJ: Prentice Hall.

Reid, K. J. (2009). Development of the Student Attitudinal Success Instrument: Assessment of first year engineering students including differences by gender (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (AAI 3378850)

Reid, K. J., & Imbrie, P. K. (2008). Noncognitive characteristics of incoming engineering students compared to incoming engineering technology students: A preliminary examina- tion. Proceedings of the 2008 ASEE Annual Conference and Exposition, Pittsburgh, PA.

Reid, K. J., & Imbrie, P. K. (2009). Exploration of differences in male and female students over a four year period: Does the data indicate support for the gender similarity hypothe- sis? Proceedings of the 2009 ASEE Annual Conference and Exposition, Austin, TX.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Simon & Schuster. Schreiber, J. B. (2002). Institutional and student factors and their influence on advanced

mathematics achievement. Journal of Educational Research, 95(5), 274–286. Sedlacek, W. E. (2005). The case for noncognitive measures. In W. J. Camara & E. W.

Kimmel (Eds.), Choosing students: Higher education admissions tools for the 21st century, 177–193. Mahwah, NJ: Lawrence Erlbaum.

300 Holloway, Reed, Imbrie, & Reid

 

 

Sevo, R., & Chubin, D. E. (2008). Bias literacy: A review of concepts and research in discrimi- nation. Retrieved from http:\\momox.org\BiasLiteracy.pdf

Taylor, K. M., & Betz, N. E. (1983). Applications of self-efficacy theory to the understand- ing and treatment of career indecision. Journal of Vocational Behavior, 22(1), 63–81.

Taylor, S., Rizvi, F., Lingard, B., & Henry, M. (1997). Educational policy and the politics of change. London: Routledge.

Tyron, B., & Bailey, D. (1970). Cluster analysis. New York, NY: McGraw Hill. U.S. News & World Report. (2013) Retrieved from www.premium.usnews.com/best-

colleges/ (membership required) Valian, V. (1998). Why so slow? The advancement of women. Cambridge, MA: MIT Press. Wainer, H., & Steinberg, L. (1992). Sex differences in performance on the mathematics

section of the Scholastic Aptitude Test: A bidirectional validity study. Harvard Educa- tional Review, 62(3), 323–337.

Weick, K. E., & Quinn, R. E. (1999). Organizational change and development. Annual Review of Psychology, 50(1), 361–386.

Weinstein, M. C., Toy, E. L., Sandberg, E. A., Neaumann, P. J., Evans, J. S., Kuntz, K. M., . . . Hammitt, J. K. (2001). Modeling for health care and other policy descions: Uses, roles, and validity. Value in Health, 4(5), 348–361.

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81.

Young, J. W., & Kobrin, J. L. (2001). Differential validity, differential prediction, and college admission testing: A comprehensive review and analysis (College Board Report 2001-6). New York, NY: College Board.

Authors Beth M. Holloway is Assistant Dean for Undergraduate Education and Director of the

Women in Engineering Program in the College of Engineering at Purdue University, 701 West Stadium Avenue, West Lafayette, IN, 47907-2045; holloway@purdue.edu.

Teri Reed is Assistant Vice Chancellor for Academic Affairs for the Texas A& M System and an associate professor in the Harold Vance Department of Petroleum Engineering in the Dwight Look College of Engineering at Texas A& M University, 3126 TAMU, College Station, TX, 77843-3126; terireed@tamu.edu.

P. K. Imbrie is Director of Undergraduate Engineering Education and an associate profes- sor in the Dwight Look College of Engineering at Texas A& M University, 3127 TAMU, College Station, TX, 77843-3126; imbrie@tamu.edu.

Ken Reid is Program Director of Engineering Education and Director of First-Year Engineering at Ohio Northern University, 525 S. Main St, Ada, OH 45810; k-reid@onu.edu.

Retrospective on Engineering Admission Policy Change 301

Looking for a similar assignment? Get help from our qualified experts!

"Our Prices Start at $9.99. As Our First Client, Use Coupon Code GET15 to claim 15% Discount This Month!!":

Order a Similar Paper Order a Different Paper