We're glad you're here. We invite you to subscribe to our RSS feed.

All this talk about scientific research leads to the question, is there any value in non-scientific research? Am I down on research that is not scientific? Not at all!

Of course nonscientific research has great value at all levels of ministry! Not everyone needs to conduct a sample survey to understand the needs of a local target audience. Some of the most celebrated ministries in America were founded on non-probability research.

I am down on presenting unscientific research as though it is scientific. If you lead a ministry that strives to reach the masses, you owe it to your organization to get scientific research. If you have a national (or statewide) audience that pays attention to your research reports, ethics demand you put forward the most accurate data possible, especially if you make recommendations to ministry leaders based on your research. People who read your reports deserve the courtesy and respect of being informed about your methodology.

Nonscientific (Nonprobability) research has a very important place in ministry communication planning. Scientific research is quantitative research , research that uses scientific methodology. Another valid for of research is qualitative research  this form of research will draw from your practical understanding of people. People don’t make very good lab experiments, so you can’t always rely on science to understand people. Otherwise, we could all understand our spouses! :)

As a Ministry Marketing Coach you need to understand qualitative research methodology too. Together, the two forms of research (quantitative and qualitative), are two very powerful tools to help the churches you coach get in-touch with their congregation and their community.

What is Nonprobability (Qualitative) Research?

Nonprobability research is research that is conducted without using scientific sampling methodology. If you have conducted a survey and don’t know what sampling methodology is—you have conducted Nonprobability research.

Saying a survey (or poll) is not scientific is not an insult. Unscientific research can be very accurate and extremely important, and yet unscientific in methodology. If you are conducting Nonprobability research, you are not doing anything wrong. (Unless you are doing it and trying to pass as if you are doing scientific research). The point is to understand what are the limitations of the methodology are you using in your research.

The benefit of scientific research is the confidence you can have that the information you have is accurate. It is best used when you are working with very large numbers of people, such as in leading missionary work for a country, or working in denominational work (as I do). You use the scientific methodology when you need to know what groups of people think or feel with higher confidence, (because you are spending a lot of money). With scientific methods you can study people and extrapolate your research findings generalizing in detail across the board; i.e. 88% of the people think this, only 12% of the people do that, etc.

But for marketing in the smaller scale of the local church, you can take snap polls, conduct focus groups, do case studies, interview people and learn a lot about the people you want to reach. You just can’t talk to a focus group of 10 people in your church and know with any degree of accuracy how well the opinions expressed in the group represent your whole church. But how often do you get in a situation where you need such detailed assurance?

The Improbable Impact of Nonprobability Research

Major ministries in the USA have learned through non-scientific surveys many useful things. For example, before launching the Willowcreek Community Church, Bill Hybels and his church planting team asked the unchurched in their community a handful of questions that resulted in a ministry that reached the people of Barrington, Illinois and beyond.

They asked:

  • Do you actively attend a local church?
  • Why not?
  • What is it that makes it difficult for you to attend?
  • If such a church existed that was not like this, would you be interested in attending?
  • If we start such a church, may we call you?

Using the research findings, Hybels and the ministry team focused their efforts on meeting the specific needs of the people in their community. Because of their approach Willowcreek has continuously reached people who would not have otherwise come to church.

One of those unchurched people later became renowned Christian apologist, Lee Strobel. Strobel went on to write the unchurched ministry classic Inside the Mind of Unchurched Harry and Mary  
 

Later, Rick Warren was called by God to launch a new church plant in Orange County California. Taking what he learned from Bill Hybels, he adapted the questions to fit his research needs and the focus group for his church plant.

He asked unchurched Californians in Saddleback Valley a few questions:

  • Are you an active member of a local church?
  • What is the greatest need in Saddleback Valley?
  • Why do you think most people don’t attend church?
  • If you were looking for a church, what kinds of things would you be looking for?
  • What advice would you give to me, a Pastor of a new church?
  • What could I do for you?

God used the research to spark ministry ideas for Saddleback Valley Community Church that led to a 25,000 member church with a global impact. Nearly every church in America has benefited from the research Rick Warren conducted.
The results of what Warren learned led him to write the landmark book for reaching the unchurched, The Purpose-Driven Church. Thousands of churches consider themselves heirs to the Purpose Driven paradigm. Thank God, Rick Warren did his homework and shared the fruit of his research with us!

What will you church do with good research? The possibilities are limitless!

Non-Scientific Surveys and Polls

Even if you don’t use sampling methodology you can learn a lot from surveys. (Just don’t report the survey data as scientific.) I think of surveys of this type as like comment cards, like the ones you see used in businesses in your town. The companies can get a lot of good information about how to improve their products and services from the comments made in these surveys. But they should not use the data to drive major moves, unless they look deeper with scientific surveys.

The same goes for polls on websites, you can learn a lot about the preferences of the people who come to your website, but you can’t extrapolate your results and apply them to people who don’t. Also, you have to be careful because some people (like me) will vote more than once. Use good judgment when interpreting these surveys.

Focus Groups

One of my favorite tools for learning about people is the focus group. A focus group is a group of 6-10 people who meet together for an hour (or two, no more than that please) to discuss a focused topic. There is an objective moderator who guides the conversation along the topic, without taking the lead or lecturing on the subject.

The conversation is recorded (audio or video) for summarizing, reporting and showing the sponsors of the research what participant said. I have used focus groups to test programming, try new products and get an insider’s perspective from people. Imagine a group of people in your town who don’t go to church talking in a focus group about why they don’t go. You will be surprised what they say, I promise! Focus groups are a quick and fun way to get useful research.

A few tips for designing a focus group.

  • Get an objective person to lead the discussion. Someone who doesn’t have an agenda to get people to say what you want to hear.
  • Only bring people who fit the exact description of the people you want to reach into the group. Use a screening interview questionnaire to identify the people.
  • Outside the church membership, pay the participants for their time (Usually $50 each). Don’t worry this won’t jade them. They will be more committed to speak up in the group if they are getting paid.
  • Assure participants you will only use their comments in your research. Help them know they will not be contacted about their comments. Limit the people who see or hear the recordings. Don’t distribute the paricipants personal information.
  • Alert people they are being recorded and allow people who don’t want to participate an opportunity to leave if they don’t want to be recorded. Get people to sign a release giving permission to record and use their comments.
  • Develop 10-15 questions to guide the moderator in leading conversation along the areas where you want to focus discussion. Only the moderator needs to see the questionnaire. You don’t have to be too rigid with keeping them on the exact path. You want free flowing discussion. You just don’t want a focus group that lacks focus!
  • On major projects, use a professional focus group facility for your groups. They are designed with a one-way mirror on one wall with an observation area for people sponsoring the research. If the sponsors are in the observation room, they will learn a lot more than if they just read a report.
  • As a ministry marketing coach, learn to become a focus group moderator. It is fun and easy to do. If you are good at conversation, you’ll be a natural. You need to train yourself to include all the participants and stay on tract. Read, Focus Group Research Handbook by Holly Edmunds

 

Interviews

You can learn a lot by talking to people. Find people who fit the profile you want to study and ask them questions similar to the ones you would use in a focus group. You won’t need to pay them, but you could provide a meal or refreshments. Just don’t let what you provide become a distraction. (Like a noisy restaurant, etc).

Use the same approach you used in the focus group. Screen people, through conversation, not a questionnaire, to find out if they fit the group you want to study. Have 10-15 questions in mind that you want to ask. Let the conversation unfold naturally; don’t drive the conversation too much, unless you get too far off topic. Take notes; don’t record the conversation, unless they agree. Watch the time; keep the conversation under an hour.

I am surprised how few people in the ministry have had conversations with unchurched people where they are not trying to convert the person they talk to, or invite them to a church function. If we could learn to be a friend and learn to listen to people, our ministry would become so much more intuitive. I am all for leading people to Christ and I certainly want people to give church a try, but sometimes you need to set out to learn from people with no immediate evangelism agenda. People in the ministry are so used to leading and teaching people they become poor at learning from people and at not “owning” the agenda.

Case Studies

Some really good ideas come from studying in detail a particular instance of a behavior you want to learn about. How did your most active people become that way? What drives them to get so involved? Who were the key players in helping them grow? Conversely, who has recently left your church? What happened? What were the events leading up to them leaving? Also, new people have started coming to your church. How did they find out about you? What did they like most? Study a situation in detail and glean as much insight from it as you can.

Secret Seeker

A fun service you can provide as a Ministry Marketing Coach is to attend churches services and take notes as a new comer. I like to call it the “Secret Seeker”.

If you have never visited a church before, your trained eye will notice several immediate improvements the church you coach needs to make. You can also recruit and train people to visit churches for you. Train them how to record their impressions and get them into a report you can use for planning.

Businesses use the Secret Shopper concept to help marketing planning all the time. I think the concept has tremendous potential in the ministry realm. When selecting a “secret seeker,” make sure they are objective people with no agenda. Pay them as you would a focus group participant to assure their diligence in recording and reporting back to you.

Don’t use the secret seeker as an evangelism strategy. That is, don’t tell people you are recruiting them to report about a church, when in reality, you only want to evangelize them by getting them to hear the sermon. This is the same as sugging and is not ethical.

Everybody Does Research

All this talk of research may sound like a lot complicated stuff you think you will never try. But everyone does research. They may not call it research. They may call it programming. But every church will come up with ministry ideas and will try them to see if they fly or flop. Why not test your new ideas on a smaller scale and see if they connect well with people?

Research is good stewardship. It is so much easier to not forge ahead with a “bonehead idea” when you are only conducting a test, than it is to roll out a “lame brain” program and be stuck with it for years because you have invested too much money to turn back.
 

Posted on February 3, 2007

Categories: Develop the Message
Responses are currently closed.

One Response

  1. Read This Before You Jump on the Willowcreek “Reveal Study” Band Wagon! (Pro or Con!) | Ministry Marketing Coach Says:

    December 26th, 2007 at 1:12 pm

    [...] Ask Your Doctor is Nonprobability Qualitative Research is Right for You!   [...]