Thursday, July 22, 2010

Canada's Top Performers in CSR: Use of Measures

Following up from my previous two posts on the Corporate Knights rankings, this final blog discusses in finer detail the methodology Corporate Knights uses to measure CSR and sustainability of firms. One could argue that the very methodology and ideology surrounding the measurement system fosters a reductionist approach to sustainability whereby companies receive accolades by focusing on isolated systems or even isolated parts of systems rather than understanding the interconnectivity of social, ecological, and governance systems.

First, let’s consider the measurement criteria they are using. Corporate Knights uses three broad measures for CSR: governance, social, and environmental indicators. Two posts ago, I discussed dangers of limiting a firm’s environmental performance based solely on the energy, carbon, water, and waste of company operations. As expressed in my blog about Loblaw, these measures ignore performance of the broader supply chain that these companies influence. The governance and social indicators Corporate Knights uses are inappropriate proxies for measuring the company’s contribution to the integrity of systems. ‘Sustainability leadership’ is measured based on whether the company has a “sustainability” board committee while ‘sustainability remuneration’ is based on whether one senior executive has a portion of his/her pay linked to sustainability issues. Now, hypothetically speaking, if I wanted to perform well on this exercise, I would arbitrarily create a board committee knowing that there is no due diligence on Corporate Knights’ part to evaluate the integrity of what this committee does and whether it is in the best interests of social, ecological, and governance systems. I would also allocate say 3% of the compensation of one of my senior executives to sustainability issues which would grant me a score of 100% against a company who received a score of 0 for allocating 0% of the compensation of an executive to sustainability issues. Governance is much more than measuring whether a board committee exists or whether there is diversity on the board. Governance is about measuring whether companies have organizational systems and structures that assure social and ecological systems are considered in their decision-making processes.

Social measures used by Corporate Knights are based on employee safety, percentage of tax paid, pension plans and pension funding which unfortunately overlook 99% of the social systems considered relevant to sustainability such as supply chain labour conditions, social equity, poverty, obesity, disease, malnutrition, and health systems. On the transparency measure, half the score is based on how many voluntary metrics are reported (only those Corporate Knights can measure) which on the one hand is the tip of the sustainability iceberg and on the other hand, more importantly, says nothing about the integrity of these measures. The other half of the transparency measure gives points for whether the company uses international standards such as GRI or uses a third party auditor. This doesn’t help much because internationally accepted reporting standards are heavily criticized because they only provide measures companies can use but leave any commitments up to the company.

What this is telling us is that we have a fundamental gap in our ability to measure company contribution to sustainability. On the one hand, I sympathize with Corporate Knights because they are doing what is possible given the tools available. But on the other hand, I criticize Corporate Knights because there is a massive disconnect between what they are claiming to measure (i.e. firm CSR) and what they are in fact measuring. This is misleading to readers who don’t have the time to verify the methods and associated measures and in fact encourages companies to engage in shoddy CSR efforts.

When evaluating the integrity of measurement systems like Corporate Knights, it’s important to consider the ramifications associated with company behaviour when responding to these evaluations. Due to the lack of a comprehensive measurement of systems, their instrument creates tunnel vision because companies are so deeply focused on say the reduction of water use that they fail to consider the fact that reducing water use requires heavy industrial chemicals that negatively impact other systems. This fosters a reductionist approach to sustainability where commitment is meant to be piecemeal rather than part of a company’s culture and way of thinking. This measurement tool also promotes incremental improvements or eco-efficiency which means that companies are encouraged to improve water, waste, carbon and energy within the existing mode of operations rather than reexamining the actual mode of operations itself.

In conclusion, I reiterate that the work that Corporate Knights is doing provides an important service to the business community and to society. But they should refrain from claiming that their work measures a company’s commitment to sustainability or corporate social responsibility. Based on the measures Corporate Knights is using, we should either change the definition of CSR or they should avoid the use of CSR as their dependent variable. To me, this initiative is not measuring CSR, it is measuring waste, carbon, energy, and water intensity, employee pay, safety, and pension, taxes paid, and whether the notion of sustainability is even thought about in an organization. These are either required by law or represent a very trivial voluntary effort. In other words, Corporate Knights needs to revisit what they claim to be measuring. Failure to do so will give an inaccurate depiction of what CSR and sustainability is and more importantly fuel the criticism and skepticism associated with the private sector contributing to a sustainable society.


  1. Mike , fascinating post, highlighting the inadequacies of almost all ranking and rating systems. What is absolutely critical about these is the transparency of the methodology, which many other ranking systems fail to provide.
    Warm regards, elaine,

  2. Please see my piece for Sustainable Life Media. Reflects some alignment in thinking...Let's dialogue!

    The Next Step: Meeting the Need for Universal Measurement, Transparency and Accountability

  3. Great post Mike. As I read this, it is easy for me to sympathize with Corporate Knights as the problems they are grappling with are probably similar to issues I had to deal with when I worked for Zumer. I must admit that when faced with the daunting task of measuring sustainability our instinct was to break it down into more understandable and manageable pieces; however, like you say, this contradicts the very nature of sustainability, which at its roots is the interconnection of a number of related systems. Although I understand the issue better than before, I still do not have a definitive answer to this dilemma. I only hope that Corporate Knights will be able to modify their system's methodology to account for more of the complexities of CSR in future.

  4. Hi Mike,
    Thank you for taking the time to put down this thoughtful analysis. We share your concerns on the potential for gaming criteria and the risk of missing the forest for the trees.

    At this point, we recognize that a company's supply chain is crucial to its impact, and we are committed to finding a way to objectively measure this impact going forward, as data availability allows.

    We do believe that the 12 indicators used for the TSX 60 assessment represent the best objective and obtainable metrics for measuring a company's "responsible" business practices--a term we prefer to "corporate social responsibility."

    Two of the key aspects that have held back the "CSR" movement from becoming a more core part of financial markets and CEO's priority lists are its scatter-gun approach (what John Elkington has referred to as carpet-bombing with metrics), and subjectivity.

    From a ranking point of view, and recognizing the imperfections of our methods, we believe monitoring and transparently publishing performance on a focused set of objective and obtainable responsible business metrics offers the best chance of overcoming these key barriers and moving the ball forward with regard to corporate practices.

    Please feel free to contact me with any specific ideas you may have for how we can make the ranking better within these constraints.

    Toby Heaps
    toby at

  5. Hi Toby,

    Thanks so much for jumping in here. I completely understand that you're likely very constrained by the availability of data given that your role was never meant to be the collection of new data; clearly a very arduous task.

    I guess it partly comes down to what Corporate Knights wants to be or considers it to be. Is the organization acting as a mechanism to consolidate and compare existing data provided by the firm or does it want to take perhaps a more ambitious role associated with determining these measures and then discreetly pressuring companies to use them. I realize that this is a radical change that likely requires substantial resources that you don't have.

    Although i haven't seen much on this, I believe there is an opportunity to consider the firm's CSR performance using a backdrop of carrying capacity on the ecological side and stakeholder perspectives on the social and governance side.

    For the former, could we not come up with an allowable ecological footprint based on the company's socio-economic contribution? For instance, Loblaw provides food to the masses, which should afford them a certain ecological footprint as a percentage of Canada's allowable footprint relative to other countries, the product of which matches the limits of the Earth's carrying capacity. Could we devise a formula based on the social and economic contribution of the company that spits out their maximum ecological footprint?

    I wonder if it might be worthwhile to team up with Ivey where some of my students could work under my and your supervision to work ont this (very low cost).

    On the social side, we may need to simply start with evaluating companies based on whether they are in fact considering the social implications of their business and how complex this consideration is. Do they recognize that their actions have implications for the health system and for social equity in developing regions? Are they looking at this?

    Just some thoughts for now. But please contact me if you think we might be able to partner with an eager group of University of Western Ontario students come the fall.