top of page

Numbers, Numbers and More Numbers: A Look Into Quantitive Research with Response Rates

Updated: Jul 9, 2023

Welcome back to Ed.D Leadership Chronicles where your host Amanda is not sure what is going on each week but manages to figure it out!

This week I wanted to share some quantitative data I pulled and briefly analyze the data.

I analyzed data from 3 articles:


  1. Paradoxical Leadership and Innovation in Work Teams: The Multilevel Mediating Role of Ambidexterity and Leader Vision as a Boundary Condition

  2. Serving while being energized (strained)? A dual-path model linking servant leadership to leader psychological strain and job performance.

  3. Linking servant leadership to follower emotional exhaustion through impression management.


I know, sounds daunting, huh?


The first article, Paradoxical Leadership and Innovation in Work Teams, analyzed paradoxical leadership and its positive effects. Wondering what paradoxical leadership means? It’s okay, don’t stress over it, the point of this post is to pull and analyze a specific data set. Both the second and third articles are over servant leadership. The second article Serving while being energized (strained), researched to understand how and when leaders can benefit from practicing servant leadership. The last article Linking servant leadership to follower emotional exhaustion through impression management, researched to see if servant leadership had a direct effect on emotional exhaustion.

Simple enough? I hope so.


I chose to focus my attention on response rates rather than the conclusion of all the data gathered (you should be thankful). Each article administered surveys to the company in hopes of having high response rates for their research. Luckily, the response rates were good, and one article managed to obtain almost a 97% response rate (which is not easy to obtain).

I wanted to focus on response rates because those are the rates used to tell the rest of the numerical story. The large data we see in charts are mainly based on how many people complete the survey. The more people you have complete a survey, the more likely you are able to generalize an assumption (you can do it without large numbers, but it may be more credible to have a large participant study).


Article 1

In Paradoxical Leadership and Innovation in Work Teams, the authors wanted to sample 760 people. They used a 3 “Times” approach, meaning they wanted to conduct 3 different surveys over a certain amount of time. In this instance, the authors chose a 6-week period; 2 weeks in between each survey. Let’s take a closer look at their response rate from the original 760 people they wanted to ask:



The authors started with 597 people responding to the very first survey. They initially wanted 760. This had a response rate of about 78%.


After two weeks, the authors conducted the second survey this time asking the 597 who completed the first. They had about 575 responses. This response rate was higher at 96%. Two more weeks and the authors concluded their surveying with 562 people completing out of 575 responding.


Not too shabby, but the initial request was 760 people. They ended with 562 people; kind of a huge jump.


Article 2

Serving while being energized (strained), administered 5 assessments over a 1-month time frame. This blog will focus on 3 out of the 5 response rates which are: Team Leaders, Followers and Superior Leaders Response rates.

Team leaders had a 84.04% response rate. There were 474 people who responded out of 564.



Followers had the highest response rate, which as seen above, 94.07%. 3.719 respondents from the 3946 originally asked.



Unfortunately, the superior leader's response rate was slightly above half (do not have the hard numerical values to share).

From looking above, the higher the position the lower the total response rate, which could be due to time constraints of those in higher positions.


Article 3

The authors in Servant leadership to follower emotional exhaustion through impression management chose to conduct 2 three-wave surveys. One survey in China and the other in the US. Today, we’ll just focus on the US’s results

NOTE** The video attached says China, but the numbers below represent US**

The surveys were conducted over a 6-week period, with 2 weeks time in between each survey.




654 responded to the first survey. When we arrive at the second survey, out of the 654, only 429 responded. It’s important to note the authors mentioned from the original 654, 13 participants incorrectly entered information and couldn’t use their data. When we look at the response rate from survey 3, only 334 responded out of 429. Between the first and last survey, we see a 300+ gap The authors also noted this survey was voluntary and participants could choose whether to continue.


Notes on Response Rates

There could be numerous reasons why the response times from the second article were higher. A few options:


  • Incentives for participants from authors

  • Certain country’s/cultures may find value in such research

  • Survey questions were clear and concise

  • The time in-between each survey didn’t surpass a certain threshold that made it difficult for participants wanting to continue.

  • The company's leaders incentivized their participation


There are many reasons response rates could’ve fluctuated throughout the research journey. I believe it’s important to document the drop in numbers because it allows the audience to understand how selective our results can be. We can draw general assumptions about a topic but must understand how many were actually surveyed and can those numbers represent the larger population (in most cases it can, but let’s not assume).


It is hard to receive high response rates when surveying over a period of time and we should make note of that conducting quantitative research. I do believe it’s also important to remain transparent with the audience about the response rates.


Do response rates effect the overall conclusion of research? Ehhh, it depends on what you're researching. Authors are forming general assumptions about our population based on the responses they receive. This could mean, the more responses they receive, the less skewed their information could be. But this doesn't necessarily negate their overall research.


If you need more of a visual guide to this lovely data, please click the link below:


See you next time on Ed.D Leadership Chronicles.

Don’t forget to hit subscribe so you can be the first to check out the latest blog post!

-Amanda H.

References


Li, F., Chen, T., Bai, Y., Liden, R. C., Wong, M.-N., & Qiao, Y. (2023). Serving while being

energized (strained)? A dual-path model linking servant leadership to leader

psychological strain and job performance. Journal of Applied Psychology, 108(4), 660



Peng, A. C., Gao, R., & Wang, B. (2023). Linking servant leadership to follower emotional

exhaustion through impression management. Journal of Organizational Behavior, 44(4),


Zhang, M. J., Zhang, Y., & Law, K. S. (2022). Paradoxical Leadership and Innovation in Work

Teams: The Multilevel Mediating Role of Ambidexterity and Leader Vision as a

BoundaryCondition. Academy of Management Journal, 65(5), 1652–1679.

18 views0 comments

Recent Posts

See All

5 Takeaways!

After reading and analyzing Ruben and Gigilotti’s Leadership, Communication, and Social Influence : A Theory of Resonance, Activation, and Cultivation (I know, long name, right), there are 5 key takea

Comments


bottom of page