And the winners of the 2015 CRM Watchlist are...

Media Inquiries: Contact Karen Budell, CMO

The season is upon us.

This year, 142 companies (out of 153 registrations) submitted their questionnaires. I have to say that there were some surprises, both good and bad. Winners that lost, winners that became honorable mentions, honorable mentions that won, new winners, elites that won but dropped from the elite category -- the shift was significant, more so than in years past. There were fewer winners and elites than ever, and more honorable mentions than ever, too.

The interesting issue is why, and I'll address that a bit later.

But first, a few things that I need to either reiterate or newly make clear about the contest.

IMPACT AWARD

The CRM Watchlist is, as I've said many times before, an impact award. I am looking to see what kind of impact a company has in the market, both in 2014 and what I anticipate from roughly 2015 through 2017. But this year, unlike in the past, I'm going to tell you what I mean by impact, and why it matters.

If a customer thinks of your company when they are thinking of doing something that is related to you, that's impact. If your competitors see you as someone worthy of competition, that's impact. If you are seen by pundits, press, and practitioners as a leading solution or provider in your field, that's impact. If a prospect thinks of your company as the one that they need to investigate because of interest in a categorical solution, that's impact. I could go on ad infinitum, but you get the idea.

Impact doesn't have to be global to win the CRM Watchlist. It can be geographical, for example EMEA, Latin America, or strictly North America. It can be the domination of a category of software or services if the category is more than a small niche. But it has to be so prevalent and deemed by me to continue to be so prevalent over the next several years that there is no doubt at all that your company is making a major impression on a market and actually changing or strengthening that market by its presence.

To do that, it takes a company that is well rounded -- it has the stability, management, products, services, culture, and partner ecosystem necessary to create and sustain the impact over time -- and that means providing what customers are looking for in a way that the individual customers find meaningful without damaging the stability and finances of the company. That's why my criteria are company driven, not product driven.

But it also means that the outreach, the engagement with the market and those in it, the prospects, the customers, the analysts, and the journalists -- those who can influence the marketplace -- have to be there, too. That takes a complete (and complex) set of tools and activities which could include marketing, outreach to influencers (analysts, journalists), the subject matter expertise via the content produced and distributed for consumption, and the activities necessary to stay top of mind as well as capture share of wallet.

That is not an easy task.

THE 2015 CRITERIA

There are two sets of criteria: One embodied in the questionnaire with the 12 questions that have to be answered, and the second with over 20 intangible characteristics I'm looking for that are not publicly apparent. Before I list the 12 criteria, I have to confess to an error I made that impacted a small number of earlier registrants.

In an early iteration of the questionnaire, I hadn't included two things: A basics list, which asked for name of company, date of incorporation, number of employees, location, etc; and secondly, a question on customers (question number six below). For those companies that submitted this questionnaire, because I failed to replace your older one with the new one, don't worry. I did the research independently and got what I needed for each of you to be able to score the question appropriately.

Here are the 12 criteria (though not the full set of questions I asked about each):

Corporate overview: Who are you and why should you be on the Watchlist?
Financial overview (confidential): Tell me as much financial information as you are comfortable revealing, but make a case for your financial stability and growth.
Management: Who runs the show there when it comes to all customer-facing things, both generally and specifically?
Vision and mission: Not only what is your vision and what is your mission, but also how is it manifested in your business and interactions with the rest of the world?
Technology: If you're a tech company, what are your products, services, and pricing? As a consulting/SI firm, what practices do you have and what is your expertise in those?
Customers: Who are your customers, what is your deal size, give me some marquee names if you can?
Partnerships: How do you think about partnering? Who are your alliance partners and why? What kind of programs do you have?
Marketing strategy: What is the approach you will take to marketing in the upcoming time frame? What kind of actions does that lead to?
Outreach (analyst relations; public relations): What is your outreach game plan? Who are you talking to and what kinds of interactions do you have with the analysts and journalists?
Market impact: How top of mind are you in the marketplace? What awards have you won? How is your presence sought after and found?
Thought leadership: What kinds of things are you doing to capture mindshare, not market share?
Culture: How great are you to work for? Work with?

These are merely paraphrased, and by no means complete descriptions of the requirements of submission. Each of these is weighted differentially each year based on real-life market forces. There are also more than 20 components that drive either positive or negative intangible scores that can have a serious impact on the final result, which aren't necessarily a direct answer to a specific question.

Please bear something in mind when it comes to all of this: Just because you didn't win doesn't mean you are a bad company. For example, going through a major transition in some way can have a serious effect on your score because of the uncertainty of that transition. Remember this is an impact award, and if I can't see both the impact you had in the prior year (for example, 2015 award, 2014 impact) or easily foresee the likely impact you will have in the next three years, your chances of winning are diminished. Transitions make that clear line of sight foggier. There are an enormous number of factors at play in winning the CRM Watchlist.

Another thing, just to be entirely candid and perfectly clear, this is geared to established companies, so the larger ones have a better chance (though no guarantee) of winning the award. They have the resources that a lot of smaller companies don't have to execute on the things they need to do to have an effect on a market. They have a longer-standing history typically, and they have a lot more money. They have a much greater reach and market recognition. They are mature and can invest not only the money, but also the manpower in the specialised activities that can be required to impact a market. In smaller companies, the resources just often aren't there, but that by no means negates smaller companies from having an impact. There are a few smaller companies that won this year, as in every year that I've run this.

To win this, you have to be a formidable presence in the market. You are a company that has a great value, not just to the customers you serve, but also to the market you reside in. You are a company that has a sustainable impact on the actual direction that your market takes, and thus, for a company considering the purchase of products or services, a more than viable candidate for being the top choice in their arsenal (with all the caveats attendant on that statement).

THE CRM WATCHLIST 2015: THE PROCESS, THE CHANGES

I had 142 submissions out of 153 registrations this year. I read what was roughly 2,000 pages in the month of December and the early part of January. I thought with a submission last year of 61 pages that I had reached a record size. This year, I had six submissions that beat that number. Two were 66 pages, two were 71, one was 72, and one was 90 pages. Once again, no holiday, no life. Poor little me. Of the 153 registrations, 12 chose not to submit and six of them told me they weren't going to be submitting in advance, as I had asked the registrants to do if they decided to not submit. To those that did submit, thank you for honoring me. For those that didn't submit but told me, thank you for telling me. For those that didn't submit and didn't bother to let me know even though I had been researching them since they registered and explicitly said not to register if you aren't submitting, please don't register next year.

My process is straightforward. I put out the call for registration starting at the end of February. I leave registration open until about a month before the submission is due. I had a submission due date this year of December 5 to allow for the interruption of the Thanksgiving holiday.

As soon as a company registers, I start to track them. The reason for that is there is a lot of information I need to gather (it varies by company and how much I know them) prior to the submission that will help me corroborate their statements and supplement their submission when the time comes. Over the year, I spend hours with each registrant's company, gathering info. But to be honest, that's not hard, because I need to track some of them anyway (they are my clients, or part of my coverage as an analyst) and I like getting to know new companies this way. I started using a tool this year late in the process called RivalIQ, which tracks the digital footprint of companies in a comprehensive and terrific way and led to insights for me. For example, Oracle's social presence is by far greater than any other company I tracked -- far greater -- which was a bit of a surprise, to be honest, and this impacted its score.

Once I get the submission, I go over every single word and, key to this, the submission is 85 percent of the material I use to derive the score of the particular company. The other 15 percent is the external material I need. This is a metaphorical number for the ratio. It could be 80 percent to 20 percent. The point is that the submission had better be convincing if you want to win.

As I said above, the scoring is done with weighted criteria. Each year, depending on the market, the scoring weights change. For example, in years of financial uncertainty (think 2008), the financial stability of a company is weighted higher than normal, though, of course, it is always important. Some years, they are remarkably flat. This year, there was some flatness but heaviness at the edges.

Once the scoring is done, the result is "automated", meaning I have a spreadsheet that sees the score and color codes the number. The resulting color code shows me whether the company was an elite, a winner, an honorable mention, or didn't place this year, based on the rules.

Following the scores, there is the announcement of the winners and honorable mentions and various other awards -- and then, of course, the winners are reviewed as I clump them together in categories that I create for organizing purposes. To reiterate something that I seem to have to reiterate every year, there are no categorical winners. The categories are conveniences that allow me to group winners who do similar things. That is all. They will be published in groups over the next few months.

Because of the reviews, I often say that this has got to be the only award that vendors fear winning. It's basically, "Hi, you win and here's what's wrong with you!" The reviews explain why an individual company won the award and then suggest a few areas for change that might improve the company -- advice that they can, of course, take or leave.

There were surprises, with some companies sinking in the standings, some newcomers, and some jumping up in the standings.

One thing I dropped was the guaranteed place in the Watchlist for the previous year's CRM Idol winner. That was sparked by my determination that while the company is clearly having an impact (the winning of the CRM Idol alone indicates that), it also needs to be on the same playing field as the other companies, which is impossible without filling out the questionnaire. So this year, the CRM Idol winner isn't on the board.

I also revived a category with some modifications: The company to watch. This is kind of a rookie of the year award. It's a company that qualified for the CRM Watchlist but was too early in its life to win, but, oh, man, the potential is sitting there right in front of our faces and will be on my radar ever after. This year, there were two companies that earned the award.

One very big change this year, which made this a much harder year to win the Watchlist than ever before, is that you could get a negative intangible score. In the past, you received positive points to your unweighted score if you met some of the intangible criteria. This year, that still held, but you could lose points if you did some of the things that were "no-nos" related to the intangibles. This will be the formulation going forward.

I'm also toying with the idea of publishing the names of the companies that scored at the elite level (which is quite different than the overall elite score) in the different categories. So, for example, I would list the elite level scorers in partnerships, outreach, marketing strategy, market impact, thought leadership, and culture. That would come as the last post as we close out the reviews this year.

Finally, if you are a company that submitted your questionnaire and didn't win, you have the right to ask me why you didn't do as well as you thought you would. I will take some time and explain it to you, because I know that you worked hard to submit what was likely a comprehensive answer. However, the big caveats on this are that it will only happen after all the reviews are done and you'll only get the answers I can give you without exposing some of the criteria I keep to myself. Sorry for the rules, but I'm one person and I want to accommodate as many as I can who request it. So there have to be limits.

2015 CRM WATCHLIST NOTABLE 'TRENDS'

There were some notable trends this year, which I will interweave with the reviews. Please keep in mind that when I call them "trends", it may be stretching it. The reality is that they are common themes noticeable in the submissions, so not really trends. They are specific to these submissions, but they are interesting, sometimes generally, sometimes specifically interesting in understanding the mindset of companies. Among the general themes for your viewing pleasure:

CRM suites going away: Oracle CX (though it has an underemphasized CRM solution inside that portfolio), SAP CEC solution portfolio and platform, Salesforce Customer Success Platforms. Only Microsoft still sells a complete suite as a CRM suite as its flagship.

Data Science on the rise: A very significant number of entries classified themselves as masters of data science. Several of them proved their case. Big data and analytics were a huge part of this year's Watchlist submissions, ranging from what the company does for a living to a service they offer to a component of their platform or application suite.

Real lack of top of mind re: corporate giving: On the downside, in the discussion on culture, there was a surprisingly large number of submissions that never even mentioned giving back to the community or corporate giving. But of course many, to their credit, did. To be blunt, how can that not be a top of mind part of your culture -- whether a big or small company?

Sales and marketing alignment not a theme this year: Last year, sales and marketing alignment was a significant part of the messaging of many companies; this year, far less.

Customer engagement more of a theme than in the past, though not overwhelmingly so:Customer engagement (meaning an explicit mention of it) was a noticeable theme, but not as dominant as I would have expected. That said, other permutations of it, like CX or specific technologies like customer engagement analytics, were everywhere. It was a reflection of how nascent this booming market is.

Lots of rebranding, new markets, and management movement: This year, there have been more major management changes, moves into new markets, and rebranding efforts than ever before, creating a lot of "impact uncertainty" in otherwise very good companies.

Real lack of outcome-based messaging: There is far too much product-dominant messaging at this stage of the game. Customers are looking for outcomes -- how what you provide provides the outcomes that they need to achieve their goals.

Platform as a service (PaaS) gaining ground: Verint customer engagement platform; KANA end-to-end omni-channel service platform, Salesforce Customer Success platform; Totango customer engagement platform, etc. This barely touches the surface of a shift among the submitters to platform-as-a-service offerings.

RightX3 overuse - This is just me bitching about cliché driven marketing. I don't know about you but I'm sick of RightX3. It goes something like this: "We're giving customers the right blah for the right ugh in the right yuck." This seems to have superseded the need to make sure that the value of your offering is expressed in words that all start with the same letter as the most obnoxious messaging trend. Too many submissions had right, right right. Far too many. That would be one or more.

FINALLY, MY DISCLAIMER

As always, a disclaimer.

Some of the winners are my clients; some are not
Some of the honorable mentions are my clients; some are not
Some of those that didn't win are my clients; some are not
Some of the companies that didn't enter the Watchlist at all are my clients; some are not.