Introducing a Metric System:
Challenge and Opportunity for Cultural Changes in Organizations[i]
Benjamin Mordecai Ben-Baruch, Senior Market Intelligence Consultant and Applied Sociologist, CPRi
bbenbaruch@earthlink.net
The first conference at which I presented a paper was
on “Conference on Critical Perspectives in Organizational Analysis”.[ii]
The three of us, graduate students at the University of Michigan, discovered that a core group of the other
participants considered themselves “sociologists of accounting". We
didn’t understand this field of the “sociology of accounting” until Mayer Zald
explained it to us: “If you want to understand the historical sociology of
capitalism and capitalist economic firms, you need to understand what the
capitalists thought was important to count.”
Weber understood this. Rationality and predictability
were crucial components of bureaucratic organizations in capitalist societies.
Weber and his students understood that these were ideal types and that in fact
the formal rationality of organizations could indeed by quite irrational.
Throughout the nineteenth century and even well into the twentieth, small
businesses could operate on the basis of cash flow. Business records were
primarily ledgers of cash flows -- usually with a separate ledger for wages.
Large organizations needed accounting methods – and these methods included
inventing concepts of what was worth counting.
I contend that for many organizations a new idea of
what is worth counting is always necessary – and sometimes sufficient – for
changing organizational cultures. For many organizations – especially large
bureaucracies and firms – a new way of counting is always necessary, and
sometimes sufficient, for enabling cultural changes.
Productivity is an obvious example. The way a firm
views its labor force is very much affected by how it measures productivity.
Is productivity the sum of the market price for good and services produced
divided by total wages? Or is it the quantity of goods (measured in weight or
volume) produced divided by the person-hours of work needed to produce them?
Do American auto manufacturers want to compare themselves to foreign
manufacturers by calculating their wage bills relative to sales revenues or by
comparing the number of person hours required to produce an automobile? The
current standard in the auto industry is the annual Harbour report that measures
labor-hours-per-vehicle.[iii] Auto manufacturers assume a
direct relation between hours worked and production costs, but comparing actual
labor costs across nations is both theoretically and practically problematic.
The problem with this proper approach, however, is that as American
manufacturers become increasingly productive, a decreasing labor force is
supporting an increasing number of retirees contractually guaranteed pensions.
If auto manufacturers only calculate productivity in a way that allows them to
compare their manufacturing costs to the competition, a very large part of the
structural costs of American companies will be overlooked.
And so there are separate accounting procedures for
looking at structural costs.
One aspect about the rationality of bureaucracies is
that it is dependent upon what organizations decide to count and calculate.
Some of this is conceptual – and some of it is practical. If I want to compare
the productivity of American and Japanese auto workers, I need to know how many
hours are worked in each company. This is possible in the auto industry
because it is mature enough to have developed the demand by auto companies for
this information and the willingness of auto companies to allow third parties
to collect this data to share across the industry. If I want to compare the
productivity of American farmers to the farmers in the developing world, I need
to know how many hours the farmers work in countries without the capacity to
reliably collect or calculate this. If I don’t have that data, I can’t use
that as a measure. If I want to know how effective my internet advertising is,
I need to know how many people see my advertising and how many people act upon
that. Without that information, I am speculating and setting myself up for a
bubble that will burst. Silicone infusions might lead to sagging rather than
perky profits. But if measuring web page hits has become the routine basis
upon which an organization – or a group within a larger bureaucracy – has
organized itself, then clearly any more rational business process or strategy
will be dependent upon a cultural change. Some will change. Some won’t.
MY EXPERIENCES I: Evaluation Research in a nonprofit setting
A group of researchers in 15 nations had developed a set of measures to assess the impact of early childhood settings upon cognitive development and were used to seeing results in terms of early childhood education settings and programs having a positive or a negative impact upon cognitive development. I analyzed data in which there was no positive or negative impact. Evaluation research does not take kindly to such “no findings”. I concluded, however, that “no finding” was a finding and that a conclusion of early childhood settings having a neutral impact upon cognitive development in a country with very little money and resources was rather positive. I concluded that the evidence showed no harm was being done and that fact should be encouraging to granting agencies considering funding programs that could have lasting positive impacts.[iv]
I don’t think that this incident itself had any real impact upon organizational culture, but it did challenge some tacit cultural assumptions. Furthermore, it indicated to me how changing the routine ways in which organizations look at data and interpret evaluation measures is important to the cultures of the organizations – and affects their clients. On the other hand, the fact that we interpreted the data for the World Bank and for Trinidad-Tobago in this way was, I believe, important. Without this understanding and interpretation, our data would have been treated by these bureaucracies as performance metrics rather than research findings – that is, as pure measures rather than useful knowledge.
The interpretation of the data in a non-bureaucratic way was a reminder to the educational research foundation of the importance of our critical and analytical thinking and a warning that non-profits, too, often commodify research products. More importantly, we also prevented the bureaucracies that were using our findings from commodifying the key metrics that we presented to them. We gave the findings a meaning they would not have had without the “spin” we provided, i.e. without the context and meaning.
MY EXPERIENCES II: Loyalty and “Customer
Relationship Management”.
CRM (Customer Relationship Marketing) is a new “buzz
word” in marketing. Essentially it means using customer databases efficiently
to customize and personalize communications and services. When you click on
Amazon.com and it suggests other books in which you might be interested based
upon your previous purchases and/or searches and upon purchase patterns of
others in the Amazon.com customer database, this is CRM. When a company sends
you a brochure that is mailed out in bulk but which addresses you by name and
has content customized to information about you which you either provided or
which the company bought, this is CRM. The goal is to be able to use
technology to recognize each customer in a database, their preferences and past
communications and transactions, in the same way Ms. Greenberg or Mr. Park
recognize their customers when they walk into the neighborhood green-grocer.
The hope is to be able to apply artificial neural net technologies that work
almost as well as Ms. Greenberg’s and Mr. Park’s real neural networks. In
fact, however, most CRM is done using SQL queries, cross tabs, and when
marketers get really sophisticated, logistic regression algorithms (which
business people call “data mining”).
Most thoughtful treatments of CRM call out one of the
revolutions in marketing today: power has shifted from the business to the
consumer. Car customers, for example, now typically walk into dealer showrooms
armed with Consumer Reports ratings, information about what the trade-in
value is of their current vehicle, and the cost to the dealer of the vehicles
they are interested in. They also know what vehicles they have in their
household and what they are willing to do – short term and long-term – to
juggle vehicles so that they get what they want. What does the dealer know?
The dealer knows what the new vehicles cost them (according to the accounting
rules and procedures the dealer has in place), what the expected profit is from
financing and future service work (again according to the application of
accepted accounting procedures), what the sell the trade-in value of the
current vehicle, and possibly some information about the household and its
vehicle history to the extent that this is a repeat customer and that the
customer can be recognized as a repeat customer.
Finally, what does the automotive manufacturer know?
The OEM knows its expected profit if the dealer sells a vehicle. After the
sale, the OEM might know if this customer previously purchased any of the
vehicles it manufactures and might be able to recognize this customer as a
member of a household. But the OEM remains ignorant about this customer’s
behavior until after the sale, after it is too late to affect the purchase
decision – unless the customer chose to contact the OEM and to identify
his/herself in a way the company could link them to its customer database.
Businesses are uncomfortable trying to manage customer
relationships when customers have power. Bureaucracies want to control everything
they manage and manage only those things they can control. And all MBAs learn
that “you can’t manage what can’t measure.” Decisions therefore have to be
made about what is going to be measured:
1.
What are the units of analysis?
What is a customer? Who are our customers?
2.
What are the units of
measurement? What is going to be measured and how?
3.
What is going to be reported?
What information is going to be available to drive organizational decisions and
behavior?
Good business analysts are only going to provide
information that can be used and acted upon. Providing too much information or
information that cannot be used by existing bureaucratic procedures leads to
information overload and renders the information provided essentially
meaningless. On the other hand, good managers want to be able to act upon
information that suggests they should change the ways in which they do things
but they need this information provided when it can actually be used to change
procedures. When they are under time and resource constraints to just “get the
job done” information about how the job should be done differently is useless.
One colleague maintains – based upon empirical
research – that workers can only handle 3-5 changes over a 1-2 year period in
the units of analysis and measurement they routinely use in their jobs. More
precisely, bureaus or units within bureaucratically organized modern businesses
can only assimilate 3-5 changes in the units of analysis and measurement that
are routinely used in the business processes they implement.
Let us assume that I am an automobile manufacturer
that has recently merged with another automotive manufacturer – Mercedes-Benz
and Chrysler perhaps. And let us also assume that now I am interested in
promoting customer loyalty that includes purchases of products from each of
these units that are now part of the same company.
As much as I may want to make the merger succeed and
facilitate the necessary cultural changes, if my job involves the counting of
either customers or products and the two merged companies counted customers and
products differently, I can’t do my job. Somebody – an applied sociologist
perhaps but more typically an accountant or a market researcher with an MBA –
needs to figure out a new way of accounting for customers and products. Then
somebody has to “teach” the organization how to use the new method of counting
– i.e. has to manage the processes of changing the way things are counted,
measured, conceptualized and how this information is used.
If I’m Daimler-Chrysler, I may not care whether my
Chrysler customer’s next purchase is a Chrysler, Jeep, Dodge, or Mercedes. (On
the other hand, if I am a dealer with Dodge, Toyota and Volkswagen dealerships
“loyalty” and repeat business is going to look a lot different! And if I’m a
dealer with Mercedes, Fiat and Saab franchises I will have yet another
perspective!)
With the frequency of multi-vehicle households today
and the mergers of automotive companies, all of the larger manufacturers face
similar needs as they try to adapt to market power shift to consumers.
Cultural changes in the organizations are required and these cultural changes
are contingent upon being able to act upon the following units of analysis and
measurement:
1.
Customers are both households with
multiple vehicles and multiple vehicle needs as well as the individuals within
those households;
2.
Sales include vehicles from all
divisions/units of the corporation as well as ancillary products (leases/loans,
parts and accessories, etc.)
3.
Loyalty can be measured and
reported in a number of ways – all of which will affect how a company treats
its customers. For example, the units of analysis for loyalty might be:
a.
repeat acquisition of a unit’s
product in terms of all prior acquisitions by a customer
b.
repeat acquisition of a
corporation’s product in terms of vehicles currently owned and in the household
c.
repeat purchase of the same make
and model in terms of the vehicle being replaced.
4.
Furthermore, the units of measure
might be
a.
Customers, or
b.
Transactions, or
c.
Dollars spent by customers.
A loyalty rate might be percentage of customers who
are loyal or it might be the percentage of either transactions or revenues
which comes from loyal customers.
To do any of this, managers have to be able to
actually recognize and keep track of customers and their households over time
and space – and as household compositions change. They also have to do record
linkage across merged corporate divisions so they can match product purchases
with customers. And this must be accomplished with dynamic databases in real time
(unlike academic researchers who typically have the luxury of doing record
linkages across static databases without time considerations that limit
accuracy and reliability checks).
And then, after figuring out what is practicable, analysts
have to figure out a strategy for getting the new units of analysis and
measurement used. The bureaucracy has to have the “will” to change as well as
the capacity to change.
In short, the job of the applied researcher is to
demonstrate why and how the new units of analysis and measurement will benefit
both the organization as a whole as well as the people who have to actually
change their routines. Simultaneously, the issue of how the new information
will be used needs to be addressed. The demand for cultural changes needs to
be created and then the changes themselves need to be encouraged, facilitated
and enabled by introducing new “metrics”.
CONCLUSION
Introducing new ways of looking at data can help drive
organizational change by creating a perceived need to do things differently.
Introducing new
- units
of analysis, and/or
- units
of measurement, and/or
- ways
of reporting data
are
necessary to supporting many cultural changes in organizations. Teaching an
organization to understand and use these new data helps create a perceived need
and therefore willingness to change as well as enables the changes in
organizational behaviors to occur.
Similarly, bureaucracies can only manage what they can
measure. Organizational change is dependent upon successfully implementing
changes in “metrics”, i.e. in the ways in which data are collected, analyzed
and reported.
Therefore an inability to operationalize metrics can
prevent desired changes from occurring. Organizations can only manage what
they can measure. If I can’t recognize you as the person who moved from Walla Walla
Washington to Kalamazoo and who buys a Dodge every 5 years and is married to a
person who buys a Mercedes every two years then I can’t treat you as a loyal
customer and I can’t communicate with you as if I know you. If I measure
loyalty only in terms of what you purchased relative to the vehicle you traded
in and you are a person who buys a new vehicle every year but alternates
between a Ford and a Chevy then I can only treat you as a disloyal customer.
MY EXPERIENCES II: Effectiveness at Change
In my own work, I have been responsible for both
trying to create the will to change as well as operationalizing new metrics to
support changes. The latter is a technical task and is partially dependent
upon resources available and partially dependent upon the skill and creativity
of the researcher/analyst. I like to think of myself as both competent and
creative and I have helped companies develop new metrics and reports. But I
want to spend a few moments discussing some of my failures.
I was once hired to help a company make better use of
data it was collecting on automotive interiors. The company had an excellent
reputation for running clinics but needed to distinguish itself by also
becoming excellent at using the data it collected to help its clients make
strategic decisions. My company was hired to help in this process and I was
sent to help my company analyze data which was compiled and sent back to the
client by yet another company. Most of the data were in the form of a
three-point ordinal scale. The problem was that half of the data put into
ordinal scales were nominal data. Ratings such as “uncomfortable, neither
particularly comfortable nor uncomfortable, comfortable” or “unsatisfactory,
neutral, satisfactory” that applied to the comfort of seats or other components
of an automobile interior are quite different from “too far to the left, just
right, too far the right” or “too hard, just right, too soft”. Yet all the
data were provided to me in the form of aggregated mean rank scores. I looked
at the data, told the company that half the data were gibberish and that the
raw data needed to be reanalyzed. So I was put on the phone with the company
that provided the aggregated reports and explained to them that regardless of
what they were asked to do, as the data experts they were at fault for not
realizing that nominal data cannot be given mean rank scores and that they had
to at least share the responsibility – and cost – for reanalyzing the data.
There being no more work for me to do until all of this was straightened out, I
was sent home before noon and paid off for the week. Because I saved them
money by telling the data company that they had to eat part of the cost of
reanalyzing the data, the company was glad to give me a week’s pay for a single
morning. But I suspect that I did not help this company make their badly
needed cultural change from a company that collected data to a company that
provided strategic insights from the data. I may have even temporarily halted
this cultural change process by not allowing them to go forward at the schedule
they were planning for themselves.
Because cultural changes in large organizations are
dependent upon the metrics that drive these changes and measure success,
dissemination and alignment of new metrics is a critical challenge. The
following comments are based solely upon my observations rather than upon any
institutional analysis. First, there is the problem of uneven willingness and
capacity for change across large bureaucracies. The fact that some parts of
the organization see a need for a change and have even been given a mandate
from senior management to make changes does not, of course, guarantee that change
can or will occur. Successful change is dependent upon successfully
integrating the change processes across multiple units or “bureaus” of the
bureaucracy. My sister is fond of repeating something she was once taught:
“Most organizations are perfectly organized to get the results they are
getting.” If they really wanted to change, they’d have already changed. In
other words, resistance to cultural change is natural and structural. And one
of the chief structural obstacles to changes is often found in the ways in
which employees are compensated. If compensation is based upon previous ways
of accounting, on previous metrics, then change will simply not occur. If you
merge companies but continue to pay people for selling the products of the company
from which they came, employees will continue to compete against each other and
will not act as if they are part of new company. If you pay manufacturing
field representatives to make dealers happy, they will pay some attention to
customer satisfaction surveys pertaining to the dealer experience but will not be
concerned about customer satisfaction with the vehicle itself. If you pay
people to sell SUVs, they won’t help customers buy one of your other cars when
that is what the customer wants. If you pay IT people to push data at a
certain rate, you might not get the data quality checks you need to provide
quality analytics -- and if you pay people to get the data quality perfect
you’ll never have the operational efficiencies you need to operate in real
time.
When organizational changes are dependent upon
creating new metrics to support the desired changes, those new metrics have to
be aligned with accounting practices in the financial department and with the
metrics used for establishing employee compensation – or else you will be
paying your employees to undermine the desired changes. Businesses often find
themselves in the “prisoners’ dilemma” whereby the rational decisions of the
employees lead to irrational organizational behaviors and undesirable outcomes.
It is not enough for organizational change agents to
understand how a large bureaucracy should operate. It is also necessary for
organizational change agents to understand what changes are possible within
given time frames. Developing new metrics based on reconceptualized units of
analysis and units of measurement is a necessary part of many organizational
changes. Therefore institutionalizing these new metrics throughout the
organization and integrating the adoption of new standards and new ways of
counting is one of the challenges to successful organizational change.