Alex,
Thank you for a thought provoking post that will stay relevant for a long time... Like you, I will also admit to being "guilty as charged" more often than I would like. However, I also want to point something out that I believe is the corollary to the Rule of Equal Incompetence, and closely related to "Paralysis by Analysis". A few years ago I "felt" or "believed" that Click-to-Play display advertising that I built for the dealership I worked at should HAVE TO BE a better performer than the plain old fashioned display ads we were running at the time. I built numerous CTP Video ads, which were fun, entertaining and made use of some great clips that our OEM supplied us with, along with custom video we had shot for making TV commercials... The campaigns kept falling short of expectations, and I put tons of work into making them produce results because I BELIEVED THAT THEY SHOULD WORK. Nothing... Nada.... None of the consumers the ads were targeted at would click on the "Triangle Lense" to play the videos within the ads that they appeared inside of. Then, about 2 years goes by and I am supervising a team of Digital Advertising analysts at ADP Dealer Services. I am trying to encourage them to innovate and be creative... One of the DA Analysts comes to me one day and suggests that we start doing Click-To-Play Video advertising on websites. Ironically enough, he wanted to start with a big Chevy dealership that competes with my old dealership that I had worked at when I tried making CTP video. At first I was going to whip out all my old data... The exhaustive reports, charts and A-B Analysis comparisons which all PROVED THAT CTP VIDEO ADVERTISING DID NOT WORK FOR CAR DEALERS, or so I "Believed" based on the data I had. Data I knew was valid and which I had tried so hard to make the outcome different...
Anyways, I elected not to do that I noticed how excited this analyst was and I let my desire to encourage innovation and creativity amongst the team outweigh my commitment to data empowered, logic based decision making. I encouraged his enthusiasm, gave him several parameters to stay within and let him set up the call with the dealer to introduce his idea. During the call, it was everything I could do to restrain myself from not speaking out and citing the data that was sitting there on my hard drive like a skeleton in the closet.
The dealer approved the campaign, which was for GM Certified Used Vehicles and featured a video clip showing independent reviews from industry experts as to the value of the GM CUV Warranty and financing. Even though I knew CTP Video Ads for Chevy Dealers would not work based on the months of data, numerous campaigns and A-B testing results I had collected trying to prove otherwise!
Imagine the shock I experienced when the CTP Video advertising campaign that my employee constructed worked better than any other display advertising we had done up to that point for any dealer, of any brand! I kept checking the reports and going over the numbers to verify accuracy, but could not find where the measurements were wrong. I now had to sets of data from Google Adwords showing the opposite results for the same brand of dealership, in the same market.
I was very pleased at the excitement and the increased advertising budget the dealer invested showed he appreciated the results as well... But to this day I am perplexed by the difference in results.
What I learned from this experience is that one of the variables in measuring and managing decision making, especially in marketing and advertising, is timing... No matter how much data we collect during a specific time period, when we go back to duplicate those results during a different time period (6 months, a year, 2 years in the future) it is NOT UNUSUAL to get different results.
The other thing I learned... When you are a responsible professional and are measuring and comparing data, but in your gut you know something exists, or you "Believe it should work", and you have a lot of experience to back up that instinct... Do not ignore those "feelings" or "beliefs". Sometimes you do "gotta believe" in something that the data does not clearly reflect and take another approach, a different message, or different placements... Or, maybe a different dealership at a different time.
Of course, we need to measure, compare and analyze... But, we should also use sound business judgment and our management instincts to provide enhancements to the decisions based on data. So, I do not disagree with you, Alex, but I do know that many of us who are committed to digital marketing and a scientific approach to decision making have discovered that not all data is duplicated when the same set of variable are applied at a different time, in the future. And, that is where experience and wisdom should be used to enhance the decision matrix.
Let me introduce you to the rule of equal incompetence. It applies to many of us. When a market of competitors all hire the same people and adopt the same practices you end up with equivalency. When business decisions are not based in fact, you create incompetence. Thus, the rule of equal incompetence.
