In recent weeks there’s been a lot of talk about the role of analyst firms. Let’s be honest; the way the business works, pretty much nobody likes these organizations, but many (including me) think they’re necessary, and since Gartner is closing in on a billion dollars a year of revenue, they’re obviously selling something that people will buy. Herewith a survey of the discussion, some personal anecdotes about the relationships between vendors and analysts, and some thoughts on the future.
What happened was, earlier in February there was a fairly hard-edged story in Information Week, highlighting “the appearance of a conflict of interest”. Then in mid-month, The Register gave a pretty severe thrashing to an IDC Itanium market study. That day, in a footnote to his Ning post (search down for “register”), Jonathan Schwartz pointed to both those pieces, remarking “it'd probably help their credibility if they, like the financial analyst community, started disclosing revenues they receive from the vendors they cover”. Strong stuff. Finally, James Governor wrote a big, intense piece entitled Why Open Source Analysis Will Grow: Learning from The Sumerians saying, among other things, “Gartner is like a mainframe in 1979. Ripe for deconstruction.” (Oh, and relatedly, I recently ran across the quite decent Gartner Watch blog.)
But it occurs to me that none of these pieces come right out and point to the elephant in the room. Here it is:
Are Analysts Corrupt? · It is believed by many in the industry that if you’re a vendor and become the customer of one of these companies, then they’ll say nice things about you. The more money you give them, the story goes, the nicer the things you’ll say.
How do I know people believe this? Because when I was a bright young entrepreneur doing my first start-up, that’s what the grizzled industry veterans told me; and they weren’t subtle: “You gotta sign up with Jupiter or Meta or someone and pay them big bucks to advise you, more than you think you can afford, then they’ll write reports saying you’re great, which nobody actually reads, but that’s OK, you can hand them out at sales calls and prospects will be impressed”.
Well, I have some personal experience. On three different occasions I got to make the call as to whether a startup signed up with an analyst firm. Twice, I decided it was too expensive. Once, I did it because my VCs told me I had to.
The two times I refused to sign up, I managed to get us covered and mentioned favorably in the analyst white-papers anyhow. In the third case, we paid a whole lot of money, didn’t get much useful advice, and the analysis they wrote was lame.
So my personal experience would suggest that corruption at least isn’t universal, and furthermore, to the extent that it exists, it doesn’t work very well.
I will say that I have seen analyst white-papers with billows of glowing prose about companies that I knew pretty well and was convinced were obvious no-hopers. I don’t know if that coverage was bought and paid for, but I wouldn’t be that shocked if that was the story. On the other hand, I’ve been wrong, have written companies off as obvious bozos, then they’ve gone on to fame and fortune.
The perception of corruption, whether it’s true or not, hasn’t gone away. Shortly after I joined Sun, I was trying to figure out why large parts of the industry seemed hell-bent on re-creating CORBA, only more complex and less efficient, under the WS-* banner. One senior technology strategist, not from Sun, told me “Obvious! It’s because IBM and Microsoft paid the analysts megabucks, megabucks I tell you, to go out and tell everyone that this was where the future was, and anyone who wasn’t going that way was dog meat.” Mind you, he’d had a few beers. But that’s not the only time I’ve heard that particular suggestion.
What Do I Think? · I really don’t know. The one time I did sign up with an analyst firm, it was made clear that the package did include writing about us. They didn’t say, and I didn’t ask, how much we could influence what they wrote. I do believe that there is the potential for corruption, and that every group of ten people or more contains someone corruptible.
On the other hand, I know analysts, and the ones that I know are by and large pretty smart and pretty nice and hard-working and in my direct personal experience, they’ve given me good coverage when I deserved it, and ignored me when my story was weak.
Also: Never ascribe to malice that which can be explained by stupidity, and I mean collective stupidity. Our profession has a history of galloping en masse off in really bad directions (Ada, AI, OODBMS, there are more examples). So you don’t have to believe in mass corruption even if you think, as I do, that the analysts are mostly wrong about WS-*.
It’s a Problem · It’s a problem for the analysts, and their customers, and for the industry, that there’s this elephant in the room. Because I totally believe that we need analysts. I know for a fact that there are those who read people like me and Don Box and Bob Sutor and use what we say about Java or messaging stacks or ODF as serious business input. But dammit, we’re vendors, our paychecks depend on selling you expensive stuff! At least with us, our hearts are on our sleeves and the conflict of interest is screamingly obvious.
It seems that in a rational world, there’d be a place for professional intermediaries; someone who has a non-tech business to run doesn’t really have time to drill down on whether crazies like me who are dissing WS-* are right or wrong. They should be able to outsource that research. (By the way, no analyst from a mainstream firm has ever raised the WS-* issue with me, which seems a little weird).
If the analysts were paid entirely by technology buyers, the elephant would just go away; but that doesn’t seem likely. No, I don’t know the solution, but now at least you know about the large invisible object in the room with peanuts on its breath, the one that the people writing about the analyst industry are dancing around.
