Barb Krasner chairs our Special Interest Group (SIG) on Metrics for Measuring the Effectiveness of a Reference Program. Its goal is to develop best practices in this critical area, and it's been up and running now for six months with a talented and diverse group of reference pros. I touched base with Barb recently to find out how the SIG is doing and get answers to such questions as what are the most critical areas of reference management to measure and why.
Btw, Barb's day job is Director of Customer Advocacy Marketing at Lucent Technologies, where she started that firm's (ITSMA) award-winning Client Reference Program in 2004.
Question 1: The big rap against marketing is that marketers can’t establish the business value of their programs. How close do you think reference professionals are to this holy grail? How much closer do you think they can get?
Answer: I think we're probably closer than most marketing programs, because the goal of reference fulfillment is revenue. Some companies track the relationship between the two systematically, through Siebel, for instance, and others track it manually by opportunity, using whatever means they have available to them. Data collection remains a challenge, particularly systematic data collection, and the SIG has two members who will be developing a best practice approach to that later this year.
Question 2: Who’s been participating in the Metrics SIG? How do you define your mission?
Answer: Participating companies include: Oracle, SAP, PTC, Epicor, Qwest, EMC, Intel, TomorrowNow, and ProjectLine Services. The SIG's goals are to: (1) develop a scorecard with a finite set of reference program metrics for executives, sales, marketing, and operations efficiency that represents best class practices; and longer term, (2) develop an industry standard through benchmarking for each of these metrics (for example, what constitutes a "good" result?).
Question 3: In what areas should reference programs measure results and what are the key metrics in each? And why are they key?
Answer: The SIG has identified a single most important metric in each of these four categories: executive, sales, marketing and operational efficiency. We call these the reference landscape. For executives, the most important metric is the same as the most important metric overall for a reference program: impact on sales. Same goes for sales. For marketing, it's customer recruitment to fulfill marketing requests and/or sufficient collateral by industry, region, product/service/solution, segment, etc. Finally, for operations efficiency, the most important metric is conversion of leads to references.
Question 4: What are some other valuable metrics in these four areas?
Answer: The audit we presented at the Customer Reference Forum in November in Dallas covers the gamut of possibilities. Some hard-hitting metrics in each area include:
(1) Executives focus on financial and business objectives--in particular, revenue impact.
(2) Sales focus on incremental impact--revenue impact; reference consideration (percentage of bid responses that include references); sales cycle impact (e.g., decreasing the sales interval); and, key customer coverage.
(3) Marketing focuses on inputs to deliverables-- such as reference coverage by key account/geo/product/ solution; conversion of new wins and "go lives" to references; and collateral usage.
(4) Operations Efficiency focuses on program expenses-- including reference request fulfillment and self-help fulfillment.
Question 5: In your opinion, what are the critical success factors for creating a successful reference program?
Answer: I love this question, because certain conditions must be present for success:
(1) Know your stakeholders and their needs--what results they're looking for (in other words, how they define success) so you can set appropriate measurable targets that are meaningful to them and gaining alignment around targets and metrics.
(2) Senior executive sponsorship--we cannot emphasize this enough--it's critical to have advocacy for your program at the CMO level or some other key exec level (even at the Board level in some companies). This can make or break the program.
(3) Clear focus on results and results reporting.
(4) Communicating the value proposition for the program and putting in place a feedback loop from the "voice of the customer" back to those internal stakeholders who build the company (product/service/solution) value prop.
(5) Strong communications plan--keep the program in the forefront by announcing big reference "wins", for instance, posts to your internal web site of success stories, etc.
(6) Recalibrating the program at least annually-- keep your program relevant by ensuring it is in lock-step with your company's business priorities and any shifts in those priorities.
(7) Tools!--make sure you have processes, workflows, etc. to drive recruitment and productivity.
Question 6: In measuring the most important metric (impact on sales), do you basically just ask the sales person “how important was the reference(s) to closing the deal”? And is that not fraught with the risk of, um, bias:)
Answer: It all comes down to tracking sales opportunities and deals. Companies with systematic metric methodologies can retrieve the information from Siebel, for instance, without relying on word of mouth. If you know references were used in the sales process, you can track that opportunity through to the close of the deal either manually or through a system like Siebel. Asking about a reference's importance in the process is a different question and would most likely be anecdotal. The best scenario is asking the newly won customer how important references were in making the purchase decision.
Question 7: The measures you list seem pretty quantitative. But can quantitative measures -- as important as they are -- fully capture a CRP’s importance? What about qualitative measures such as improved relationships, improved understanding by customers of your firm, or improved understanding by your firm of customers?
Answer: Frankly, executives want to see hard impact on the bottom line. Even improved relationships could be quantifiably presented through customer sat results or incremental revenue by account. The SIG's goal is to focus on a simple but effective scorecard that clearly demonstrates the value of a reference program.
Question 8: Do you plan to look at the effectiveness of customer communities, or is that beyond your scope or too early in their development to address?
Answer: That is indeed beyond our scope at the moment and yes, it's a bit too early to measure.
Question 9: Does the SIG plan to address Net Promoters and the potential connections between CRPs and NPS? SAP could definitely help with that.
Answer: No, we have no plans to address NPS. It hasn't come up in any conversation.