Why Most Feedback Sucks

Getting better at anything requires feedback, that’s obvious.  But what is often more important is selectively ignoring most of the feedback you receive.  If you’re basing your decisions entirely on a comment by one person, you’re probably going to make a mistake.

Although it may seem like arrogance, I think following your own intuition and reasoning before the suggestions of other people is a much safer strategy.  Feedback can be helpful, but only after it’s been through a heavy washing of skepticism, so what you’re told matches what you experience.

People Don’t Know What They Want

I’ve done a number of projects which required getting feedback from the end user or customer.  Gathering opinions at this stage is critical, because it helps you benchmark your ideas of what people want, with what they tell you they want.  I’ve had a number of times (many of them through this website) where I got a sample of people to tell me whether they liked a book, program or offering.

Unfortunately, my experience has been that people simply don’t know what they want.  If you ask someone what they want, the best answers you get are usually in a narrow range.  When I would test a program prototype, people would make comments about whether a particular graphic could be improved, instead of focusing on the core features.

Most people understand what they would prefer, given two options.  But people usually can’t tell you what they want more between what you have now, and all the hypothetical improvements you could make.

Although this warning applies to someone developing a product, it’s really true anywhere.  Whenever you have to help another person, and their happiness is important to you, asking them what they want often isn’t the best remedy.  Because it misses out on all the things they might want more, but haven’t even considered.

Feedback Never Comes from an Unbiased Sample

In statistics, getting an unbiased sample from the population is crucial to getting accurate data.  If you interviewed 25 people about the importance of jazz music, you won’t get accurate results if everyone you interview keeps a saxophone in their closet.  Ideally, samples should be a random selection from the entire group.

Unfortunately, feedback never works this way in real life.  Unless you’re running a formal study with rigorous techniques, you’re going to have biases in your sample.  When I ask for feedback on a website feature, I know that the comments I receive will overwhelmingly be weighted towards people who typically write in comments.  The silent, but still valuable readers, won’t usually offer an opinion.

If you’re getting advice on whether you should stay in the same degree program, the feedback you get from others will be weighted towards the people who speak up the most.  Your quiet, but otherwise intelligent, friends will probably be under represented in your quest for honest advice.

Other People Can’t Speak for Your Motives

Feedback represents the perspective of the advice giver, not the advice receiver.  Even when people don’t have ulterior motives, they can rarely see through their own position to give advice best suited for you.

Although you may not have as much experience as the advice giver, you do have one advantage, you’re you.  You will always have access to your own perspective when trying to solve your problems, something often missing in feedback.

An entrepreneurial example of this is pricing.  Don’t ask your customers how much they would be willing to pay for something.  Only look at what they actually are willing to pay in practice.  Customers may highball the price to make you feel good.  Other customers will lowball the price because they want to pay less, even though the service is worth more to them.

Going out and asking customers whether they would buy something won’t produce the same results as actually selling it.  Some customers will say “yes” to avoid the social discomfort and rudeness with refusing your request.  However, if money is entered into the picture, people change their minds.

People Have Split Personalities

I’m not the same person five minutes after waking up as I am in the middle of the day.  Just after waking up, I’m groggy, semi-conscious and my decision making is very different from when I am alert at noon.  Drinking, lust, anger, fear, enthusiasm all make different people out of us, and those people won’t give the same feedback.

If you’re asking for feedback, make sure your asking from a person who is in the same state as when their opinion will matter.  In Dan Ariely’s book, Predictably Irrational, he mentions the outcome of a study where students were asked to imagine themselves in an aroused state, and what decisions they would make.  He then did a follow-up test to see what peoples decisions would be when they actually were in an aroused state.

The results were overwhelming.  Even though a simple emotion like arousal hits us all the time, most people can’t accurately predict their decision making process in that state.  People who claimed they would never have unprotected sex or take advantage of someone changed their opinions when their state changed.

What Feedback Doesn’t Suck?

Feedback is necessary, but you need to sanitize it with a healthy dose of skepticism before it is useful.  This means filtering out all the irrelevant feedback, feedback that represents motives different from yours, feedback that focuses on too narrow a spectrum and feedback from a person who won’t be the same when making a decision.

A few rule of thumbs I feel are useful when getting advice:

• Watch what people do, not what they say.  Behavior is a much better indicator than feedback forms.  Do a test run on an idea, and watch the reaction, instead of asking people what they would do hypothetically.
• Avoid incredibly skewed samples.  Some people like to complain.  Others like to give compliments, even when everything is horrible.  Recognize when the people you talk to doesn’t represent the whole and ignore advice from a vocal minority.
• Vision should plan, feedback should correct.  Your decisions, goals and projects should be based on your own vision.  Feedback works best as a corrective measure, making small adjustments in your direction.  When you use feedback to form the backbone of your plans, you’ll often find their based on nothing at all.