The First C: Choose your QUESTIONS

The quality of an answer depends on the quality of the question.

COMMON SAYING AMONG DATA ANALYSTS

The single most important step in any relevant analysis is to choose the questions that you are trying to answer. While this may sound obvious, it is an ongoing challenge for most analysts (including us) to stay on track by finding answers to the right questions needed to solve the problem at hand. Creating a list of questions clarifies your ultimate goals for your project, and is the best way to make sure that you are not "drifting off course" during the remaining 6 C's of your analysis.

In today's complex world, people in all departments and levels of a company are using data to make decisions. A great analyst needs to be much more than technically adept, because it's necessary to collaborate with others in order to determine what questions need to be answered. To start any project, you must assess what's going on within your company or department, so choosing questions is more of a "soft" skill when compared to the later steps of analysis, requiring a combination of art and science. Since it is a soft skill, it is often the most difficult one to master, so we will walk you through the typical thought processes behind starting your analysis, including ways to encourage input and gain support from your managers and colleagues.

Helpful Hint
If you are interested in learning the technical details of the process of analyzing data, don't worry.

You will learn plenty of them in the rest of the book, complete with illustrations.

Also, you may want to re-read this C after you read the rest of the book---you'll probably have new insights on how to choose the best questions.

In this chapter, we describe a general process of how you can determine the question(s) of relevance to your project. We mainly focus on the less complex situation in which you're answering the question for yourself, and then use this information to make a data-based decision for the business that you will present to others. In this case, it is possible that your managers, co-workers, or customers will give you a general idea of the problem area that needs to be analyzed.

We've divided this process into three steps:

1. Relax!

2. Gather Information

3. Select Your Questions

Then, we demonstrate the steps using a simple case study. Later in this C, we address the added complexity of starting an analysis project when someone else is the final decision-maker, ending with a short case study**. **

Take charge when you are making the decision

Step 1. Relax!

If I had eight hours to chop down a tree, I'd spend six sharpening my axe.

Abraham Lincoln President of the U.S. during the American Civil War

Take it Easy: Don't skip ahead---time spent on this overlooked "C" is the secret to success

There is a tendency for people to rush through this step so they can spend more time on the "real work" of collecting and analyzing data! However, choosing the right question(s) is not only the first C, it is also the most important C, because it guides the entire analysis (and is the reason you are doing it in the first place). So you want to make sure that you spend enough time assessing the problem to adequately identify what you need to find out. The upfront investment in forming useful, clear questions will save you a lot of time and frustration by avoiding unnecessary analysis, and even save you the embarrassment of not having a solution to the real problem at the end of your analysis.

Get Started:

Locate the general problem that requires analysis

Bosses, co-workers, or even customers of your business may need help solving an issue or problem. Although they can describe it in general, they may lack the time or ability to explain the details.

For instance, your colleague may notice that gift basket sales to women are down and asks you to find out why. She actually may be seeking a way to improve marketing to women, but doesn't state the problem this way. Or she may be able to tell you the specific goal, such as decreasing waste in product packaging or the number of calls to customer service. Often, she relies on you to do the legwork of specifying the exact question(s) of relevance before doing the analysis and coming up with a reasonable course of action.

At other times, you may be the decision-maker managing an area of business (based on your job description, or maybe it just needs to be done and you are stuck with it by default), and you notice that there is a problem that needs to be corrected. If you are not sure how to fix it, you must look further into the problem in order to form good questions.

Target Audience

Who will be acting on the decision that you make?

One way to consider the importance and nature of the problem at hand is to determine if the analysis will be used for strategic, operational, or analytical purposes in your business. At first, these may seem like technical terms, but you've probably had experience with these situations in your work.

If the results of the analysis will be used by executives, middle management, or business owners, the decisions made will probably affect long-term priorities for the business (often spanning months or even years). For example, they want to know if year-to-date sales in each of the company's three product lines have met the target amount, which is a 10% increase compared to the same time period last year. They are trying to decide if they should allocate more of the budget to marketing or hire more salespeople to increase sales in a particular product line versus another.

This is a strategic decision, which is usually based on less complex analysis and simple, clear presentation of results relative to stated company goals. Since executives typically review the results of strategic analyses on a monthly or quarterly basis, they do not want a lot of details for their review. Instead, they prefer some information about similar situations over time periods in the past (for instance, the first quarter of this year versus the first quarter of last year).

Heads Up

The best companies support a culture where strategic decisions are the result of great ideas combined with business knowledge and data.

Many companies are still in the process of developing an environment where data are incorporated into strategic decisions

Operational

Operational decisions usually involve monitoring live systems, such as overall data center performance or how well a call center is handling customer issues, to identify key problem areas. As a result, teams typically require a summary of performance in the past few days or hours relative to their expected performance to ensure that all is going well. When there are problems, operational people may need to know the "layered" details, or the details behind the details, to help them find and correct issues. Operational analysis results in more concrete results than strategic analysis.

Analytical

If the decision is oriented at an analytical audience, it will likely be used by analysts investigating key issues and possible courses of action. An example is an analyst investigating the reasons behind a drop in sales in the last quarter, as compared to the same quarter last year, to find possible solutions. These analyses typically include high-level summaries and detailed reviews "below the covers" to explore possible causes of problems. Analysts often spend significant time exploring these layers by several analyses that progressively reveal more information with each layer of data. This type of analysis may provide definite results for a problem or simply point you in a general direction.

Heads Up

You should also think about the personality, skills, and time constraints of the colleagues that you are helping.

For example, if your question and analysis involve complex technical details, and your colleague is always on the move and just looking for a general idea of what to do, you may be providing more information than he or she wants or needs. Also, you may waste a lot of your time creating a sophisticated analysis when a simple answer would be enough.

Step 2. Gather Information

A good decision is based on knowledge and not on numbers.

Plato Classical Greek philosopher, key to the establishment of Western philosophy and science

Learn From the Past

Collect information that is already known about the problem

It is amazing what you can learn by simply talking to, and more importantly, listening to other people with knowledge of the general issue. During this time you can learn what they know (this is a crucial point that we'll emphasize throughout the book) while gaining an ally for when you need help during the analysis. You also can incorporate their opinions into what you already know about the problem. Listening to others is especially helpful if you are new to the job or the company.

By speaking with others, you may be able to find out what relevant analyses have been done in the past, the results, if and how these results led to decisions that were actually implemented, and if the project ended up as a success. In general, other people often will help you develop great ideas of what to do. Perhaps one of the most important things these conversations will tell you, which often isn't documented, are the analyses that "failed". Analysis projects can fail for many reasons, including poorly designed questions, incomplete analyses, poor communications, or decisions that were never carried out due to company politics or other limitations.

These people may also be able to direct you to previous reports or presentations, so you don't waste time looking through irrelevant paperwork or re-doing old analyses. They also may be able to tell you where the relevant data sources are located, which will save time later in your analysis. Even if they tell you that nothing has been done on this issue in the past, this is useful information, because it tells you that any well-designed analysis and decision that you come up with is likely superior to what has been done before.

Heads Up

Incorporate all the information that you've discovered during the gathering process and use your own judgment before you determine what question will guide your analysis.

For instance, just because the company has always done things a certain way doesn't mean that it's the best way.

Play Well with Others

Bring colleagues on board

In addition to finding out the history of the problem, gathering input from others has another powerful benefit for you. It is usually a good idea to ask for people's opinions, particularly if they will be involved in gathering the data, performing the analysis, or implementing the decision resulting from the analysis. Never underestimate the importance of getting people on board with your work! They are more likely to do so if they feel their views are understood from the start and taken into account since they will then have a stake in the success of your analysis.

Heads Up

Be aware that there is a fine line between gathering people's input and pestering them.

The goal of this book is for your boss and colleagues to be happy to see you, not to run when they see you coming down the hall!

Expand the Possibilities

Avoid selecting questions based only on the usual data

The temptation at this stage in the process is to think about the problem at hand through the limited lens of the data you or your team "typically" uses for this area. This restriction can limit your success in making better decisions and constrains you from considering new and innovative analyses that may be possible.

In most cases, determine the questions to ask under the assumption that you could obtain all the data you need. You may be surprised to find that with a little searching you actually can incorporate new, invaluable data. However, if it turns out that optimal data collection isn't possible due to time or budget constraints, computer limitations, etc., then you can adjust which data are used. Even considering and addressing these limitations can be part of your final presentation to colleagues---and may inspire improvements in data collection in your business.

That being said, although you shouldn't base the decision to be made on the data you already have easily available, we realize that in the real world you often are in a rush to do analyses. If you are short on time, spend at least a few minutes considering possible enhancements or key data that would make your analysis more valuable. Discussing this possibility with your team may even create an atmosphere of innovation around your data collection efforts.

Step 3. Select your questions

The best way to have a good idea is to have a lot of ideas.

Linus Pauling Two-time Nobel Laureate, one of the greatest scientists of the 20th century

Put It All Out There

Create a list of the questions that might address the problem at hand

At this point everything is fair game---add every question that you think might possibly improve or solve the problem at hand. It usually helps to organize your thoughts by dividing the general issue into topics, and then listing possible decisions to be made under each topic.

By creating a simple list on paper, in a Word document, or in a spreadsheet, it will be easier to spot any major gaps and identify which questions may overlap with each other. Try to make the questions specific rather than vague. You may even want to share this simple list with colleagues to give them an opportunity to contribute or clarify questions. If it's on a whiteboard, you may even interest people walking by who will join in.

Reduce, Reuse, Recycle

Optimize your list of questions

Now it's time to refine the list so you have only the best ones remaining. Unless you are very new to your team, you can probably do this in 20 or 30 minutes.

1. Reduce by eliminating unrelated, inefficient or impractical questions

First, you need to remove the least useful questions. Does the question pass this point-by-point inspection for quality control?

Is the question...

Related to the problem at hand?

It may be a good question, but if doesn't help you solve the current issue, get rid of it.

Relevant to the department or person that is most affected by or interested in the final decision?

If you are solving a marketing problem, you may want to defer questions that require the involvement of another department, such as finance, in order to be successful. In an ideal world the best decision would be implemented, regardless of the department(s) involved---but this is the real world.

Focused on an option that is substantial enough to have a real impact on the problem?

Keep your overall goal in mind. For instance, your question may be how to cut down on packaging waste for your company. You know from a previous analysis that Product Line X contributes just 1% to waste. So, don't focus on that product line if there are only three product lines at your company.

Realistic because the solution would fit in with the current situation and not require a radical and quick change?

Remember that for most projects the company can only change so much based on one analysis. Therefore, drastic changes are unrealistic, at least in a short time frame, unless management specifically asked you to examine possible unorthodox approaches.

For example, if your question is "What would be the impact on sales if we double the number of advertising billboards in the next quarter?", but your company just cut the marketing budget in half, the question most likely is not worth your time.

A good fit for the corporate culture?

If your company is a web-based retailer, then examining the possibility of opening retail locations is likely a waste of your time, and may even irritate your decision-makers.

2. Reuse useful questions from the past

If you or someone else has done a similar analysis in the past, add those questions to the mix. Remember to evaluate the questions using the point-by-point inspection in the Reduce section.

Helpful Hint<br> Often the same issues arise repeatedly, so it may help you to start building a pool of standard questions in a spreadsheet or Word document (even if you don't use them in this analysis).

3. Recycle by revising improperly worded questions

The general point of a question may be valid, but it may be inaccurately stated.

Here's a simple example. The original question is: "How much has profit decreased our advertising budget?"

However, the question should be: "How much has our advertising budget decreased profit?"

Again, put the new question through the point-by-point inspection in the Reduce section.

Best Foot Forward, Please: Choose the best questions to analyze

There isn't enough time to answer all your questions, so you need to organize your list to figure out which ones to focus on. This is often the hardest step of the First C. We typically rank questions in order of importance, from most valuable to least valuable, while taking into account the following constraints:

1. How direct or easy is it to implement the likely solution if you answer the question?

2. Who will take action with the results of your analysis?
   Will they understand your line of attack?

3. Do you have enough time to find, organize, and analyze the data to answer the question?

4. Will there be enough time to implement the recommendation from the possible solutions?

5. Will this question require the assistance of other people?
   Is it expertise that you don't have?
   Is the question valuable enough that you will be able to acquire the needed time from the experts in your company?

6. Is there a budget for your analysis?
   If so, what are the budget limitations?
   For example, if your question requires purchasing third-party data or software you don't have, is this possible?

Some of the questions can be answered later, or you can do a more in-depth analysis of the most important questions in lieu of answering a long list of less-relevant questions. While it is tough to do, creating an outline with the most useful questions at the top of the list helps you to begin to see a path to completion.

Walk through the steps: The First C Case Study

This simple case study demonstrates the steps of choosing your questions.

Get Started

Jeff is a new analyst at an online pet supply company. The sales manager of pet food products approaches him with a problem. In the last month, pet food sales were up 30%, but dog treat sales were down 10%. Can he figure out possible causes?

Target Audience

The target audience for Jeff's analysis is the sales manager for pet food.

Learn From the Past

Jeff speaks with his colleagues to find out the last time dog treat sales were down. He discovers that this occurred in January. When he asks them why, they tell him that people give dog treats as holiday presents in December. He asks if there was any other time sales were down, and no one remembers this happening.

Play Well with Others

Jeff updates his boss about what's going on. Also, before he starts the analysis, he brings the issue up at his group's weekly review meeting to get input. He is checking to see if someone else has either investigated the problem or plans to, so he doesn't duplicate effort.

Expand the Possibilities

Jeff wants to know more about the problem before he works on the data. He looks at the pet food web pages. He wants to know what the shopping experience is like for the customer. Maybe dog treats were on the home page last month and now they're not, while regular dog food is? He was hoping to solve the problem at this point, but after researching relevant issues, he is still not sure of the cause.

Put It All Out There

Jeff compiles a list of all the questions he can think of that are relevant to the issue:

1. Was there a shortage in dog treats in the inventory for the last few months?

2. Were there any special offers for pet food or dog treats in the last few months that might have affected sales?

3. Treats aren't a necessity, so have other unnecessary dog accessories shown any changes in buying behavior?

4. Are there new pet food products that might have increased pet food sales?

5. Did our competitors have the same issue?

6. Did the dog treat manufacturer have a coupon that is now expired?

7. Examining sales of dog treats over the past few years by month, is there a seasonal pattern by month or time of year (beyond the commonly known "January effect")?

Reduce, Reuse, Recycle

Jeff then eliminates or rewords some of the questions, and adds a relevant question (#8) that a colleague investigated in the past:

1. Was there a shortage in dog treats in the inventory for the last few months?

2. Were there any special offers for pet food or dog treats in the last few months that might have affected sales?

3. Treats aren't a necessity, so have other unnecessary dog accessories shown any changes in buying behavior?

4. Are there new pet food products that might have increased pet food sales?

5. ~~Did our competitors have the same issue?~~ *Can't find out this detailed information, but can reword as: Did our competitors have a big sale?

6. ~~Did the dog treat manufacturer have a coupon that is now expired?~~ *Hard to find this out.

7. Examining sales of dog treats over the past few years by month, is there a seasonal pattern by month or time of year (beyond the commonly known "January effect")?

8. Was there a manufacturer's recall of poor-quality dog treats in the last few months?

Best Foot Forward, Please

Jeff prioritizes the questions with the ones that he thinks are most useful first:

1. Were there any special offers for pet food or dog treats in the last few months that might have affected sales?

2. Did our competitors have a big sale?

3. Was there a shortage in dog treats in the inventory for the last few months?

4. Was there a manufacturer's recall of poor-quality dog treats in the last few months?

5. Treats aren't a necessity, so have other unnecessary dog accessories shown any changes in buying behavior?

6. Examining sales of dog treats over the past few years by month, is there a seasonal pattern by month or time of year (beyond the commonly known "January effect")?

7. Are there new pet food products that might have increased pet food sales?

Now Jeff has a list of questions that he can start to analyze, beginning at the top!

Plan of attack when someone else is the decision-maker

True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.

Winston Churchill<br> Prime Minister of England during WWII

What's their point? Translate a request into a question

If you are very lucky, decision-makers can explain to you the decisions that they need to make. Presumably, your analysis can help them have more confidence in their "gut" feelings or possibly steer them in another direction. Ideally, they will clearly articulate the problem and explain it sufficiently so that it makes sense to you. If this happens, then you can celebrate and move on to The Second C: Collect Your Data.

However, another scenario is typical: you are presented with general issues and need to translate what the decision-maker wants into questions that she can take action on once you have answers at the end of your analysis. Often, she can't quite explain the issue to you.

Even if she thinks she knows, you may be unsure if she is asking the right questions, or even know that she isn't. There are variations on this scenario---for instance, we've had clients whose colleagues show them a graph and request an analysis that results in a similar graph, although it may not be applicable to the actual problem at hand. Or colleagues may stay in their "comfort zone" by asking the same questions about a particular problem area, but the questions need to be updated based upon new developments.

Although this may be difficult, the first step to "translating" what your colleague is asking for is to look at the issue from his or her point of view.

To translate, there are three crucial questions that you should ask yourself:

1. What does she care about? For example, is she in Sales, Finance, or Human Resources? Is she in charge of customer acquisition or customer retention? Does she have a year-end closing soon, or is she just checking to see how receivables are doing?

2. How do you make her more successful? What will make her more effective in her job, helping her to deliver results to her managers or colleagues?

3. What are the goals of the team and company? Is the company trying to open new stores or close old ones? Become environmentally sustainable?

Even if you don't understand exactly what she is asking for when you speak to her, write down everything you can during the meeting, and as soon as possible afterwards. Listen for cues, especially "points of reference" that she typically refers to in her explanations.

Here are some examples of what we mean by points of reference, including what the decision-maker might say and the possible translation that you can act upon.

Point of Reference (cues in bold)	Translation
Sales not on target	versus target
New products not doing well	versus target or previous products
Hard to find the best sales leads	Needs sales leads ranked in order of importance
Expenses are out of line	versus target or versus historic norms
Too many underutilized servers in our data center	capacity versus expected performance

Helpful Hint

Even if you collaborated with the decision-maker to choose the questions, sometimes she will tell you later that they aren't the right ones.

This may be frustrating for both of you, especially if you're far along in the analytic process. Remain calm and try to assess how far off the mark you are by asking her what information she needed to know, but couldn't get from your analysis.

**** Role-play: Find out what the decision-maker cares about

Since translating what people want into questions that you can analyze is such a difficult skill, here we'll walk you through a few examples. Jeff, our analyst, is meeting with different members of the sales department.

• Vice President (VP) of Sales

The VP asks Jeff:

"Are we finally meeting our monthly sales numbers?"

Jeff translates this as:

"Are sales doing better this month when compared to our monthly sales target? Show me this month and the last few months for context."

• Regional Sales Director of the West

The Sales Director of the West asks Jeff:

"Is the marketing campaign in California worth the money?"

Jeff thinks this means:

"Are sales in California picking up since the marketing campaign was launched?

"How much, relative to the cost of the campaign?"

• Salesperson for Southern California

The salesperson for Southern California asks:

"Am I effectively following up on my recent sales leads?"

Jeff translates this as a reference to how timely the salesperson has been in following up:

"How many of my open sales opportunities from last month have I spoken with this month? How many from the past quarter?"

As you can see, people at different levels of the same team will often request more details as you move further down the team, starting with upper management. Often, all their questions can be answered using the same data sources, but they need the data analyzed in different ways.

By asking questions of direct relevance to the audience at hand, your work becomes much clearer and more relevant to the people you seek to help. Additionally, it becomes much easier to identify a set of "core" analyses for your job that you quickly can repeat over time.

Heads Up

It is important to remember that at any point in your analysis, you are able to go back to earlier steps if necessary, because analysis is a cyclical process.

For instance, you may discover that none of your questions have good answers that help you solve your problem, so you need to go back to the drawing board and come up with a new list. When you do this, be careful not to skip steps such as running the new list of questions through the point-by-point inspection for quality control.

For current Accidental Analyst courses, resources, and updates, visit https://the.accidentalanalyst.com/. more!

Recap of the First C: Choose Your Questions

• Choosing your questions is the crucial first step in any analysis, and returning to those questions throughout the course of your work is the best way to keep things on track.

• Either you are doing the analysis for yourself, so you are the decision-maker that will determine how to correct the problem, or you are doing the analysis for someone else who is the decision-maker (the first situation is usually much easier).

• The three steps in choosing your questions are:

  • Relax: figure out the problem and who will act on the solution

  • Gather information: find whatever's known about the problem

  • Select your questions: create and prioritize a list of questions

• Review the simple case study, where Jeff the analyst comes up with questions.

• Translating colleagues' requests into questions that you can act upon is quite difficult, so look for points of reference and what their goals are.

• Review the role-playing case study, where Jeff the analyst determines what different colleagues need from him.

  © 2011-2018 YakData, LLC