What are the best supply chains in the world?

The reality is there is really almost no way to determine that, absent an incredibly detailed study of leading candidates that would even then lead to potentially dubious results and certainly be obsolete by the time the research was finished. Or, we could look at the Gartner top 25 supply chain list.

I spent two days at the Gartner Supply Chain Executive Conference last week in Phoenix (See Trip Report: Gartner 2015 Exec Conference)  but left before the big dinner Tuesday night where the top 25 list has now become unveiled each year.

The former AMR Research brilliantly came up with the top 25 idea in 2004. Gartner then acquired AMR in 2010. Over the last few years, the concept has been extended, so that we now have a "Next 25," a top 25 healthcare supply chains, top 25 industrial supply chains, and maybe some others I have missed.

So this year, I asked around a bit, and found - not surprisingly - that very few supply chain managers have any real idea how the list is created. They only know if they are in or they are out, and that's about all that matters.

This year for the first time Amazon came out on top, after Apple had grabbed the top spot for a few years running.

Somewhat oddly, to my view, Apple and Procter & Gamble were left off the top 25 this year because they have been consistently near the top of the list. So both those companies have now been placed in a separate new category, called "supply chain masters," a sort of "hall of fame" we might say.

Not sure the reasoning behind this. It frankly may have to do with in effect getting more companies in the top 25 plus new master's category - Gartner clients like that recognition, of course. Was the move in reaction to some criticism last year from an ex-AMR/Gartner executive, who said the top 25 was becoming boring because it changed so little?

Regardless, this new construct seems to me something that will be tough to manage over the years. If P&G stumbles, will it be moved out of the masters list but maybe still be in the top 25? Will the masters list grow and grow over time, so that we have these competing masters and top 25 lists each year?

With Apple and P&G withdrawn from the competition, the rest of the top 10 was number 2 McDonald's, followed by Unilever, Intel, Inditex (Zara), Cisco, H&M, Samsung, Colgate-Palmolive and Nike. L'Oreal, Toyota and Home Depot rejoined the top 25 this year. In addition to Apple and P&G, Caterpillar dropped off.

Here is a quick visual of the top 5 for 2015, with a link to the full chart of all 25 companies:

View Full Top 25 List

So, how on earth is the top 25 determined?

Gartner starts with the Fortune 500 list of top US companies by revenue and the Forbes global 2000 list that basically does the same thing on a worldwide basis. It then eliminates a lot of those companies because they do not much operate what most of us would think of as a real physical supply chain - companies in banking, insurance, software, and many more.

From that culled list, Gartner analyzes publicly available financial data, specifically looking at three metrics:

Return on assets (ROA): Net income / total assets
Inventory turns: Cost of goods sold / inventory levels
Revenue growth: Change in revenue from prior year

ROA and revenue growth use a three-year weighted average, meaning the most recent year gets the most weight and the two prior years somewhat less. Inventory turns, smartly, uses the prior year's quarterly average (reducing impact of end of year games). These three metrics together are given a full 50% of the total score weight (25% to ROA, 15% to turns, and 10% to revenue growth).

Now keep in mind that this formula gives a tremendous advantage to some companies, such as Amazon given its huge revenue growth or McDonald's and its 157 inventory turns per year. It also penalizes companies like a Home Depot or a Lowes, for example, which are only going to have turns in the mid-single digits, because of their need to stock every item under the sun to meet customer service targets, many of which are very slow movers. In general, this approach penalizes a company within a given sector that strategically decides on a higher service, lower turns strategy (even though we can all agree that inventory efficiency is a very important attribute of supply chain excellence).

Companies that have heavily outsourced production and distribution also have an inherent advantage. Why? Because they have chosen to shed assets, and that drives their ROA metric higher. While outsourcing can be a very smart thing for many reasons, it is not inherently a better supply chain move. This metric also inherently discriminates against asset-intensive businesses, such as chemicals and automotive. That no doubt why we see only three such companies (3M, Cummins, and Toyota) in the top 25, towards the end of the list, and probably a big factor in having the separate list of the top 25 industrial supply chains.

So, at this point, you must be a decently large and public company to be considered in the analysis. The smallest company on the Fortune 500 list for 2014 had about $5 billion in sales. Private companies do not have the public financial data needed for this part of formula and cannot make the list.

Another 25% of the final rankings come from so-called "peer opinion." For 2015, this consisted of about 200 apparently very influential respondents who first select a group of companies from the master list that they believe are doing the best job of being a "demand-driven value network orchestrator." Sure, we all have that list in our heads.

From those selections, respondents are then asked to rank those companies from first to last, from which points are assigned to the companies selected based on how they are scored across respondents.

So, the reference point, in theory at least, is not "the best supply chain," but rather leadership in "DDVN orchestration." Are these the same things? I would say certainly not. But I suspect the panelists in the end are probably really voting based on their perceptions of which companies have the best supply chains, DDVN orchestrators or not. (And yes, being "demand-driven" is again certainly an attribute of supply chain excellence, all things being equal.)

The final 25% comes from votes from Gartner's own supply chain analysts. They use the same tool  and criteria that the peer group does in ranking company supply chains.

Take the financial rankings and the votes from peer group and Gartner analyst group (again, 50%, 25%, and 25%, respectively), and voila, out spits the top 25 in something like a mathemetical fashion.

Is the process perfect? Certainly not. The unstated assumption is, for example, that stellar financial results equals supply chain excellence.

And it is a bit mysterious - while Amazon has strong revenue growth, it had 0% ROA and a middling 8.7 inventory turns, yet it somehow winds up on top even though those two measures account for 40% of the total score. It did have by far the top score in the peer rankings, I will note, so I guess that's what did it. Amazon is an innovator, that's for sure, and spends a lot of money on facilities and technology,  but DDVN orchestration is not something I associate with Amazon.

So, with all that, here in general is the advice I give to companies hoping to crack the top 25: (1) understand the methodology, especially with regard to the financial data. Not much you can really do about that, but you can at least understand how it works and do some comparisons to key competitors; (2) encourage others outside your organization to participate in the peer review process and rate you highly; and (3) most important, if you are really serious about this, arrange "briefings" with Gartner analysts touting what you are doing in supply chain in the same way that software vendors do it. Ladle on significant helpings of demand-driven orchestration-ness. Is there a Dummies book? (Shoot me an email if you would like to discuss any of this.)

The Gartner top 25 supply chains - it has many faults, but it is the best we've got. I look forward to it every year. It certainly stirs the pot.

What are your thoughts on the Gartner top 25 for 2015? Do you see any ways it could be improved? Let us know your thoughts at the Feedback button (email) or section (web form) below.