Program Evaluation

Quick Links

Logic Model
Evaluation Plan
Holistic Analysis of Institutional Transformation

Logic Model

Logic Model
Logic Model Graphic

The underlying conceptual framework for the TAMU ADVANCE Program is the American Psychological Association’s Psychologically Healthy Workplace initiative. Psychologically Healthy Workplace (PHW) practices are grouped into 5 categories: 1) Employee Growth and Development, 2) Health and Safety, 3) Employee Involvement, 4) Employee Recognition, and 5) Work-Life Balance. Health and Safety is operationalized in our academic setting as wellbeing & lack of mistreatment (Grawitch et al., 2007). Additionally, each of the 12 activities is overseen by a faculty/staff committee which means that Employee Involvement, one of the more critical PHW categories (Grawitch et al., 2009), is even more broadly practiced than the Logic Model indicates. Raising bias literacy on our campus (implicit bias) is also foundational to our program and is intentionally addressed.

Each of the 12 activities in which the TAMU ADVANCE Program is engaged: a) is aligned with one or more of the 5 PHW practices, b) has a collective (change faculty environment) and/or individual (support faculty) focus, and c) targets either improving Workplace Climate, Recruitment and Retention, or Success Enhancement of women STEM faculty. This approach reflects the TAMU ADVANCE theory of institutional change which assumes that no single intervention will substantially impact progress of women faculty. Instead, a series of interrelated interventions/activities are necessary for institutional transformation and a Psychologically Healthy Workplace.

Evaluation Plan

Activity Evaluation Plan
Activity Evaluation Plan Summary

Each of the 12 ADVANCE activities is being evaluated for efficacy.  The Evaluation Team works closely with the ADVANCE Leadership Team, the Social Science Studies Team, and the activity committees to craft instruments for tracking participation and develop evaluation tools to measure impacts of the program activities.  The data gathered allows for formative evaluations intended to help refine the implementation of the ADVANCE activities.  In order to complement the internal evaluation, the ADVANCE Center has partnered with an evaluator at the University of Colorado to provide an external evaluation of the ADVANCE program (see External Evaluation Reports).

Holistic Analysis of Institutional Transformation

It is anticipated that, at the end of five years, climate and faculty workplace practices at Texas A&M University will be more psychologically healthy than when the ADVANCE program was initiated. To achieve this goal, the Leadership Team developed a series of interrelated interventions (the 12 activities) designed to change the climate, enhance professional success, and improve the recruitment and retention of female faculty. Assessing and evaluating individual interventions (aside from tracking participation and for some interventions, satisfaction) would be nearly impossible because various interventions interact with one another. It would also be counterproductive because efforts focused on individual interventions would distract the Evaluation Team from its efforts to discern the extent to which institutional transformation is being achieved. Furthermore, the primary goal of the ADVANCE Center is organizational change, not scientific investigations of individual interventions.

The Evaluation Team is, therefore, focusing on a more holistic analysis of institutional transformation. This analysis is embedded in the literature on institutional change and based on department-level data on faculty climate, faculty recruitment and retention, salaries, start-up packages, and space allocations. The underlying hypothesis is that departments which have been more engaged with ADVANCE will show greater improvements in climate, retention, recruitment, and resource balance over time. To address the hypothesis, a difference-in-difference approach will be used to evaluate the extent to which engagement in ADVANCE activities can help to explain variations in departmental changes over time. For more information about data being collected for evaluation purposes, see the Activity Evaluation Plan and Activity Evaluation Plan Summary above.

Grawitch, M., Trares, S., and Kohler, J. (2007). Healthy Workplace Practices and Employee Outcomes. International Journal of Stress Management, Vol. 14, No. 3, 275–293.

Grawitch, M. J., Ledford, Jr., G. E., Ballard, D. W., & Barber, L. K. (2009). Leading the healthy workforce: The integral role of employee involvement. Consulting Psychology Journal: Practice and Research, 61, 122-135.