Scroll back to the top

Faculty Climate Surveys


Cropped image from behind showing a man looking at data on a computer screen


Overview

Climate improvement was one of the stated goal of the ADVANCE program. Therefore, during the NSF phase of ADVANCE (2010-2017), the Evaluation Team examined the relationship between ADVANCE and changes in faculty climate (as measured by the 2013 and 2015 climate surveys).  The general methodological approach was difference-in-difference analysis. Basically, the Evaluation Team measured the extent to which treated departments saw larger changes in various measures of climate than did non-treated departments.

The Evaluation Team found evidence that job satisfaction and some dimensions of a psychologically healthy workplace improved for some populations at TAMU.  However, in all but one case, there was no significant difference between treated and non-treated departments. In the one case with a significant difference, the evidence suggests that perceptions of Employee Growth and Development improve more in non-treated departments than in treated departments. Further, differences between the sexes were not significantly different between treated and non-treated departments.

Although disappointing, these null results were not unexpected. Two years is a remarkably short period over which to see statistically significant changes in climate. Other ADVANCE institutions have been able to detect the influence of ADVANCE interventions on climate, but only after much longer periods of time have elapsed.

 

Difference-in-Difference Analysis

ADVANCE activities were not the only factors influencing salaries at TAMU since the implementation of the program. The university instituted a university-wide salary freeze from 2009-2011. An early retirement program was implemented in 2011, and there was also a hiring freeze. Turnover among top administrators created uncertainty. The Vice President for Diversity and the Dean of Faculties continued to sponsor programs that were independent of ADVANCE but might have influenced department head behaviors, faculty climate and retention. These alternative sources of change made it challenging to separate and analyze the effects of TAMU's ADVANCE program. 

To address this challenge, the evaluation team used a difference-in-difference approach to detect evidence of climate change. Given the changes described in the previous paragraph, we cannot simply compare relative wages before and after ADVANCE. That effect would be measured with time bias, i.e. bias caused by changes over time. Another possibility would be to compare relative wages in ADVANCE-treated departments before and after the intervention. However, such a measure would include error because treated departments could be different from non-treated departments. Fortunately, the type of error given by these two approaches comes from different sources. The difference-in-difference approach is able to subtract out the error caused by time-bias with the first approach by comparing treated departments both before and after the treatment to non-treated departments both before and after the treatment. Essentially, one measures differences between groups (treated and non-treated) before the intervention (pre) and again after the intervention (post) to see if those differences have narrowed or widened.

For the purpose of this analysis, the treated departments are all STEM departments plus the non-STEM departments of the College of Agriculture and Life Sciences.  The climate surveys excluded faculty affiliated with TAMU-Qatar and faculty in newly affiliated divisions (e.g. the Law School and the colleges with ties to TAMU's Health Science Center).  Analysis was restricted to tenured or tenure-track faculty members. All models controlled for college fixed effects to allow for the possibility that climate might be systematically bettern in some colleges than in others in both time periods (pre and post).


 

Climate Change Effects

The Evaluation Team examined 10 measures of climate. Seven of the measures explored the impact of ADVANCE on the five dimensions of the psychologically healthy workplace (PHW): Employee Growth and Development; Employee Health and Well-Being (two measures); Employee Involvement; Employee Recognition; and Work-Life Balance (two measures).  Factor analysis was used to distill responses from multiple survey questions into each PHW measure.

 

Items Used to Construct the Psychologically Healthy Workplace (PHW) Factors

Factors

Items

Employee Growth and Development

  • The head/director of my department:
    • provides opportunities for members to develop their abilities to become future leaders.
    • meets with me regularly to provide constructive feedback regarding my performance.
    • encourages my development with respect to establishing and maintaining an effective research program.
    • encourages my development with respect to improving my teaching.

Healthy and Safety (1)

  • During my work, I often feel emotionally drained.
  • After work, I usually feel worn out and weary.

Health and Safety (2)

  • How would you rate your physical health over the past year?

Employee Involvement

  • How satisfied are you with:
    • the opportunity to collaborate with other faculty?
    • the amount of social interaction with members of your unit/department.
    • the level of intellectual stimulation in your day-to-day contacts with faculty colleagues.

Employee Recognition

  • How satisfied are you with the degree to which your professional developments have been recognized?
  • The head/director of my department praises my successes.
  • My colleagues would fail to notice, even if I did the best job possible.

Work-Life Balance (1)

  • To what extent do you agree that each of the following statements represent the philosophy or beliefs of the majority of the members of your department:
    • Individuals who take time off to attend to personal matters are not committed to their work.
    • It is assumed that the most productive employees are those who put their work before their family life.
    • The ideal faculty member is one who is available 24 hours a day.

Work-Life Balance (2)

  • After work, I come home too tired to do some of the things I would like to do.
  • On the job, I have so much work to do that it takes away from my personal interests.
  • My family/friends dislike how often I am preoccupied with my work while I am at home.
  • My work takes up time that I would like to spend with family/friends.
 

Items Used to Construct Other Climate Variables

Scale

Items

Job Satisfaction

  • All things considered, I am satisfied with my job.
  • I would recommend employment at TAMU to a colleague.

Departmental Inclusion

  • The climate for women is good in my department.
  • We have taken steps to enhance the climate for women in my department.
  • We have made an effort to recruit women faculty in my department.

Climate for Diversity

  • Leaders are committed to diversity in my department.
  • Diverse perspectives are valued in my department.
  • People from different backgrounds get along well in my department.
  • Leaders value everyone regardless of their backgrounds in my department.
  • Colleagues value everyone regardless of their backgrounds in my department.
 

Key Findings

The Evaluation Team found that female faculty members at TAMU had lower climate perceptions than male faculty members in the pre-ADVANCE period.  The coefficient on Female was negative and statistically significant at the 5-percent level for all of the factors except Employee Involvement and Employee Recognition.  Among the departments that would be treated by ADVANCE, female faculty members had lower average climate perceptions than male faculty members in 2013 with respect to Employee Growth and Development, Employee Involvement, Climate for Diversity, and Departmental Inclusion.

There was little evidence of change in male perceptions of climate between 2013 and 2015 in treated departments. Males in treated and non-treated departments reported higher levels of job satisfaction in 2015 than in 2013. Males in non-treated departments also reported improvements in Employee Growth and Development and in Climate for Diversity. 

Female faculty members in non-treated departments reported higher job satisfaction in 2015 than in 2013.  However, female faculty members in treated departments  did not report increased levels of job satisfaction. Similar to their male counterparts, female faculty members in non-treated departments reported improvements in the Climate for Diversity between 2013 and 2015. In contrast, neither male nor female faculty members in treated departments reported any significant increase in this climate indicator.

Female faculty members in both treated and non-treated departments reported statistically significant declines in perceived departmental inclusion between 2013 and 2015. In contrast, male faculty members in both treated and non-treated departments reported no change in this climate indicator.

The key variables for the Difference-in-Difference analysis is the three-way interaction between the indicator for a treated department, the indicator for the POST period, and the indicator for female. Across the board, the three-way interaction was statistically insignificant, meaning that one cannot conclude that the difference between the sexes narrowed more in treated departments than it did in non-treated departments.  In short, the climate survey data does not sugest that ADVANCE changed the climate measures between 2013 and 2015.


 

Additional Information

Diversity Survey Menu.  This document includes survey items that are recommended by the ADVANCE Social Science Studies Team for inclusion on any TAMU Diversity survey. The team recommends that all diversity surveys contain at least one sclae from each of the overarching sections: Demographics, Climate Assessment, Inappropriate Behavior, and Outcomes. 

2013 Survey Response Rate

2013 Campus Climate Survey: Executive Summary Report

2013 Campus Climate Survey: Report on Climate

2013 Campus Climate Survey: Report on Job Satisfaction and Turnover Intentions

2015 Faculty Climate Survey Presentation.  This slide presentation explains the importance of the faculty climate survey and how units should use the results. Each college and department received data for their unit.