web analytics
How can we reach you?
Send along your info and we'll contact you.

HR Needs to Quit Cheating on Engagement Tests



paul hebert[HR Needs to Quit Cheating on Engagement Tests – from Paul Hebert-Vice President Solution Design. You can read more about him on our Leadership page.]

All too often we look over to the paper of the company sitting next to us to find the answer to our own employee engagement problems.  How else can you explain why zappos (everyone drink) created a profit center out of their company tours.  I checked for a conference – they charge for group tours now.

HR likes to see what other companies are doing.  HR likes to listen to speakers tell them about best practices.  HR (okay – everyone does this stuff but today I’m picking on HR) wants to do what has been proven to work in other places.  That’s the risk-adverse nature of HR. Don’t get mad at me – check a twitter chat with HR folks, they admit they’re afraid of risk.

So couple the hype about the new new thing from the consultants and the need to cover their a*$ses – HR is going to cheat a little and take a peek on the other person’s test.  I get it.

But you may want to answer your own questions instead of copying the “smart kid” in class.

Sometimes Random Means Random

Check this article on Bloomberg Businessweek entitled “For Better Schools, Start With Evidence.”  The gist of the article is that the US Department of Education runs something called a “What Works Clearinghouse.”  The clearinghouse is a collection of techniques that have been “proven” to work in a school setting using a method of proof called randomized trials.  That means the technique was tested using the same method the FDA uses to test and clear drugs for use on us humans.  Pretty stiff testing methodology right?.  This supposedly gives educators clear, usable data on what works in schools and what will help our little ones learn and be smart.

The process ensures evidence-based outcomes and techniques we can take to the bank!

Except we can’t.

Evidence Isn’t Always Evidence

The problem the article points out, is that while randomized trials for drug testing works, when we get into areas like education policy (or basically anything that relies on human beings) it is far from an effective process.

Using studies of classroom size on test scores as their point of comparison they found that in one Princeton study larger class sizes equaled lower test scores.  Yet, another study by MIT found a similar relationship but at 1/4th the impact of the Princeton study.  And the cherry on this best practice sundae?  Another study by MIT on students in India showed that larger class sizes actually showed better test performance.  Three studies, all using the same technique to ensure validity – all with different outcomes.  In addition, when they kept the location the same but changed the teacher they got different outcomes again!

In other words – all these evidence-based studies – designed help policy makers determine the best way to influence education (actually test scores which isn’t necessarily the best measure of education – but that’s for another day) – all of these studies had different results. All of them structured similarly. All of them using a rigorous testing method that supposedly gives better data.

And all of them with different outcomes.

WE Are the Variable

The upshot from the article was this:

It isn’t just where the policy change happens—it is who manages the change.

Sound familiar?

All the employee engagement activities at zappos or Netflix won’t necessarily give you the results they got. In fact, like in the class size experiments, you may actually get opposite and negative outcomes for your company by following their lead.

Your specific company, its history, its managers, its culture, its location – all have huge impact on the programs and engagement activities you run in your company.

You are dealing with variables beyond the control of us mere mortal managers.  You are dealing with – human beings.

The best way to engage with them – is to be human too.

Don’t be lazy and cheat on the other student’s paper.

Do your own work inside your organization.  Find positive deviants and study them.  Run small-scale experiments.  Rely on your OWN data and you’ll see much better results.


One Response to “HR Needs to Quit Cheating on Engagement Tests”

  1. Chris Fields, MLHR says:

    A wise man once told me “If you ain’t cheatin’ you ain’t tryin’.”

Leave a Reply