Posts tagged ‘Performance Measures’

Performance Measures and Incentives, Part 2: What They Teach Us About Government Behavior

In the first part of this series, I wrote

Many readers will know that I have spent 25 years working with government agencies in a company that privately operates public recreation facilities.  Not infrequently I have had my managers, in frustration over something our agency partners have done or not done, complain that the folks they are working with in the government are “bad” people.  More generally this is a common refrain of government critics, that state agencies are full of “bad” people.  I always disagree with them.

The people that the government hires are no different on average than the people hired in private industry.  Sure, there is some self-selection as people may migrate to institutions they trust more than others or to work cultures they find more appealing, but this is true as well among private entities (e.g. choosing to work at a startup vs Exxon).  My strong belief, from theory and long experience, is that when government employees appear to act “badly” or irrationally, it is not because they are (or began as) bad people but because they are working in an organization with terrible incentives and a counter-productive performance management system.  This is not unique to government organizations – many or most of the great failures of once-proud private companies have come about due to issues with incentives.  The difference is that when private organizations go bad, there is a culling by customers and competitors that has no equivalent in the perfect monopoly of government agencies.

In this post, I want to go further into incentives and metrics helping to explain the behavior of government agencies. In particular, I want to look at two interrelated complaints often voiced about government agencies: "why are they so inefficient and/or slow?" and "why do they say 'no' to every new idea?" The answer to both comes down to performance metrics.

But in looking at the performance metrics of government agencies, we immediately run into a problem: many, perhaps most, government agencies don't have any written performance metrics or productivity standards for the majority of their employees. I have worked with public recreation agencies for decades (think NPS, USFS, BLM, state parks, etc) and I have never once seen anyone with a set of written performance metrics relevant to their job (some agencies will claim hey have metrics and then point to their 200-page strategic plan that was crafted a decade ago).

I understand that creating a good set of job-related performance metrics might be hard for some staff functions, but even for people, say, running a park I have never seen any goals for expense reduction, visitation increases, or visitor satisfaction (eg customer review scores) -- all metrics every one of our managers who privately operate public parks have. And even when they do have performance metrics, they are often for the wrong things (** see bonus story in the postscript).

So is Coyote full of sh*t? How can their behaviors be driven by performance metrics when they don't seem to have any?

Here is Coyote's first law of incentives: There are always incentives. If they are not embodied in written performance metrics, then there are unwritten ones that rule behaviors. And these unwritten incentives are generally a) very powerful and b) almost never aligned with the greater organization's goals.

If there is a very good manager, the incentive might be that manager's praise and recognition. Small teams can sometimes be energized by a shared mission they all believe in. To some extent the agencies I worked with were better than most because public lands agencies (eg NPS, state parks) attracted people with a sense of mission which could motivate people even when the organization did not.

But in general, government employees operate in a vacuum without any positive metrics -- they can't prove themselves by meeting or exceeding this or that goal because the goals have not been assigned and are not measured. So the default metric becomes this: to avoid screwing up.

Government employees operate in a web of hundreds, even thousands of procedural rules. The most obvious of these are very strict budgeting rules where (unlike in the private world) money is not fungible but is divided up into scores of buckets and has to be spent within both the dollar limit of its bucket and the narrow expense categories the bucket is assigned for. Everyone lives in fear of violating these spending and budget rules. I will give one example -- I know of $4 million from a private insurance settlement in favor of a public agency that has languished in a private bank account FOR FIVE YEARS because the government agency can't figure out how to receive the money and decide what account it should go in. Fearful of making a mistake, the money sits dormant.

But beyond money-related rules there are scores of others -- rules for environmental reviews and approvals and rules for archeological reviews of any digging. We had to replace a road culvert a year ago in California on public lands and among the 12 reviews that had to be completed was an historical review to make sure the culvert was not some sort of historical artifact. I remember we inherited a shack on the end of a dock that we wanted to replace (because it was falling apart) and the agency who owned the dock said "wait a minute, we think that is a historic building." I have to admit I laughed pretty hard when my COO lost it and said "then I will drop the f*cker on the Washington Mall in front of the Smithsonian for all the tourists to see but it is unsafe where it is and needs to be replaced."

And the list of rules keeps getting longer. When the organization has new top-down initiatives, they are layered on top of all the other initiatives and mandates to add more layers of review. In several California agencies, for example, simple contract changes have to be reviewed automatically by BIPOC and gender groups. There are climate and sustainability reviews of everything.

And all of the above does not even get to the subject of decision-making authority. In many organizations people are unclear who has the authority to make a decision, so "to be safe" they will kick it upstairs to their boss. Who might kick it upstairs to their boss. The hardest problem we often had in working with agencies was, even when we were pretty sure person X could approve our request, person X often felt safer kicking the decision to a higher level. Now the decision is on the desk of someone who isn't directly involved with the issue, and thus who is reluctant to do make a decision for something they don't really understand.

I want to remind folks that this is intended to be sympathetic -- the behaviors that can be so irritating are NOT the result of bad people being jerks. They are the rational responses of absolutely normal people operating without positive incentives and walking through a minefield of ways to screw up.

All of this leads logically to several behaviors:

  • If you go to an agency for a decision, there is absolutely nothing positive they can gain from the situation. They don't get rewarded for satisfied citizens or number of requests approved. What looks like a simple decision to you is fraught for them. Every decision is a chance to screw up on any one of scores of rules without any possible upside for them.
  • Even the simplest approvals and decisions take forever. Given that they are now saddled with your request, they are going to make sure they respond to the only incentive they have, which is to make sure they can't get criticized later. They are going to run an environmental review even if it is not necessarily required. They are going to insist a historian does an evaluation. They are going to check with their DEI folks to make sure you are being inclusive. They are going to kick the decision upstairs one or more levels higher than they actually need to. Meeting after meeting will be held with everyone who could possibly be involved, sometimes over a dozen people.
  • They will not care about the time, because they are not rewarded for responding quickly. They have no upside and can only focus on making sure that they don't skip a step some later Monday morning quarterback thinks they should have followed. This could, by the way, be by an external group -- say the Audubon Society or a tribal group -- that sues the agency for its decision. The US Forest Service, which has to balance diverse activities on public land from mining and off-road vehicles to conservation, is sued over everything.
  • The default answer is "no." Every "yes" is a risk, and government employees don't get any reward or recognition for a "yes." But they frequently get called on the carpet or trigger a lawsuit when they say "yes."

I will add that every agency has brave people who get positive things done for the public and their agency despite the incentives above. I met one such person a few weeks ago, Superintendent Trimble at Mammoth Cave National Park. He and his team are doing a fabulous job creating new ways for the public to enjoy the park. But folks like this are recognized and stand out as exceptions. It takes an unusual person to rise above this quagmire of bad incentives. Many folks that see these problems but don't have the nerve or energy or skill or will to persevere eventually leave government service for other things.

There is a lot more I want to get into on this, but before I do I want to step back and review another old business-economics chestnut, the agency problem. That will be the next post.

**Postscript -- Bonus Story. It used to be that California State Parks had a rule that only front-line employees badged as law enforcement could be promoted to higher ranks of the Agency. This was incredibly limiting and distorting and my favorite parks director Ruth Coleman got the rule changed 10-15 years ago (she later took the fall for a financial problem in the state, in part I think because the union was still angry about this change).

Anyway, before this change (and as a good illustration of why the change needed to be made) I was going to visit one of their parks and looked at the online reviews. They were terrible! Apparently the state park rangers handed out a zillion parking and related citations. When I visited their office, I saw on the wall a performance metric! Apparently there was a tracking sheet showing who had made their minimum citation goals and issued the most tickets. Apparently since this was the only metric they had, this is the one they focused on. And according to the reviews, they did a bang-up job finding any excuse to drop a citation on their visitors. But these were the folks responsible for the parks management -- it was like a McDonald's manager spending all their time citing customer cars in the parking lot. It was crazy.

Performance Measures and Incentives, Part 1: Lessons From A Famous Corporate Implosion

Part of a Series:  Organizational Design, Behavior, and Change

To understand why organizations grow senescent, get fat, and fail – whether they be public or private – there is probably no topic more important than performance measures and incentives.  Many readers will know that I have spent 25 years working with government agencies in a company that privately operates public recreation facilities.  Not infrequently I have had my managers, in frustration over something our agency partners have done or not done, complain that the folks they are working with in the government are “bad” people.  More generally this is a common refrain of government critics, that state agencies are full of “bad” people.  I always disagree with them.

The people that the government hires are no different on average than the people hired in private industry.  Sure, there is some self-selection as people may migrate to institutions they trust more than others or to work cultures they find more appealing, but this is true as well among private entities (e.g. choosing to work at a startup vs Exxon).  My strong belief, from theory and long experience, is that when government employees appear to act “badly” or irrationally, it is not because they are (or began as) bad people but because they are working in an organization with terrible incentives and a counter-productive performance management system.  This is not unique to government organizations – many or most of the great failures of once-proud private companies have come about due to issues with incentives.  The difference is that when private organizations go bad, there is a culling by customers and competitors that has no equivalent in the perfect monopoly of government agencies.

I will return to the issue of government incentives in the next post, but I want to reinforce the importance of incentives to organizational failure by discussing one famous private example, a company everyone thinks of as failing due to fraud and malfeasance – ie bad people – but whose downfall was at its core due to bad incentives:  Enron.  In what follows I don’t want to take away from the criminality of various executives.  But I believe the failure at Enron started WAY before the criminality and was rooted in bad incentives.

The story starts with Jeff Skilling coming to Enron to implement the gas bank model he came up with at McKinsey.  Having personally been on this study for a brief time (as a very junior associate), I don’t think there was anything particularly wrong with the strategy.  The problem was perhaps in Jeff himself.   As has been reported many times, he was certainly brilliant, but my guess is that he was likely manic-depressive, and he certainly did not have a very good read on people.  I won’t say he was on the spectrum but had a sort of quasi-autistic inability to understand how people really tick.  He could fall in love with his own intellectual creations, which might be marvelous in theory, but fraught in actual implementation with real people.

Here is my theory – the downfall of Enron can be traced directly to Jeff Skilling’s implementation of what he called “mark-to-market accounting” for new Enron contracts, an academically intriguing idea that met with disaster in the face of real people.  To understand it, we first need to know what he meant by “mark-to-market accounting."

Since the dawn of capitalism, enterprises have struggled with how to provide the proper incentives to their sales force and dealmakers.  The simplest example to reward salespeople with a percentage-of-revenue commission.  It certainly seems logical to reward salespeople for making sales.  But even something so simple quickly becomes fraught when it encounters real human behavior.  The problem is that sales people will chase the easiest sales with the highest discounts, building a book of business that maximizes their commissions but may miss the most profitable sales. The same is true for deal makers, who (if allowed) will pursue the riskiest deals with the highest potential returns, leaving the company on the hook for future potential losses.

And this problem was an order of magnitude worse at Enron, where Skilling implemented a strategy in which traders are tasked with executing complicated long-term deals.  Traders were executing deals as long as 20 years in which natural gas might be bought here and then sold there and then converted to electricity which is sold somewhere else all tied to a capital investment at a certain plant and sale of tax credits to other companies.    Most, meaning pretty much all, companies would account for this by simply reporting the net profits from this deal in each year as they occur.  But what if most of the profit in the deal was 10+ years out?  Should the company, or the individual dealmaker, really have to wait 10 years to be rewarded?  Would the organization even bother pursuing these contracts if that were the case?

Skilling hated this traditional accounting.  Companies are often criticized for not thinking about the long term – how can one reward the organization for selling business 20 years out? He believed strongly that once the deal was signed these gains were all earned and locked in, such that the present value of the contract was essentially earned at signing and should be booked at signing.  Rather than waiting 20 years for the earnings to flow through, Skilling wanted to book the whole profit immediately, crediting both the company and the individual deal-maker with the entire value immediately.  This is what he called mark-to-market.  All the present value of profits from a multi-year contract would be booked when signing the contract.  And the deal maker would be rewarded for a percentage of those profits on signing.  Like much of Skilling’s thinking, this was a theoretically compelling approach.

Mark-to-market was a term well known at the time in the banking and securities world.  It refers to the process of adjusting the book value of investments to their market price.  All well and good – this was something the government was trying to get banks and financial institutions to adopt – so the government was open to approving Enron’s proposal for what was very unusual accounting in the energy world.

The problem for Enron was that they were, by their own admission, doing deals that were unique, that no one else was doing.  So how does one establish a market price?  Not everything was locked in by the contract – profits might depend on commodity prices or cost overruns or interest rates or even the bankruptcy of a counter-party.  Skilling may have called it mark-to-market, but that was effectively impossible. What Enron was really doing could more reasonably be called mark to forecast.  The forecasted revenue and profits for the contract would be recognized immediately on signing.

And now we arrive at the disaster.  Enron had the dealmakers themselves creating the pro formas or forecasts from their deals from which the present value determination was made.  So, what is this forecast going to look like?  The forecasts were going to be hugely optimistic -- the deal is going to make a freaking fortune with little risk, the forecast is going to be a hockey stick upwards.   This does not necessarily even require fraud – every sales person and deal maker who has ever existed is optimistic about their own deals.  And given that these are 20-year forecasts, the valuation of the deal might turn on the price of natural gas 20 years out.  Whose to say that $5 is more or less reasonable than $3?  The net effect is that all the mark-to-market valuations skewed high.

Skilling and Lay seemed mostly blind to this problem.  They did (nominally) create internal agencies that were meant to neutrally review deals and these mark to market valuations.  But these groups got little support from executives, particularly when deal makers seemed to be making so much money for the company (at least on this mark-to-market basis).  Deal makers had become the elite of the organization, virtually unchallengeable even by staff originally tasked with challenging them.  And remember that Lay & Skilling’s prestige and compensation – not to mention the compensation of every manager at Enron given their compensation system -- was largely stock-based, and the stock price was being driven up by the stacking up of these skewed mark-to-market valuations of projects, causing the company to effectively pull forward years of future (potential) earnings into the current year.  Anyone who started challenging project valuations was effectively attacking the entire organization’s compensation.

Enron eventually applied mark-to-market accounting to everything.  Change in tax law?  Take a one-time gain for expected net present value of decades of tax breaks.   Sign a video streaming deal with Blockbuster?  Immediately book profits for 20 years of hypothesized streaming revenue.  Of course, if the assumptions behind a deal were to change for the worse – say a reduction in natural gas prices from those forecast – then there should have been a mark-to-market loss taken on the contract, but that almost never happened. 

This was Skilling’s underlying ethical failure – mark-to-market was his conception, and he was responsible for making sure both halves of the process were implemented:  both taking credit for future gains when contracts were signed but also taking losses for impairments on those future results as they became obvious. I am not sure the mark-to-forecast approach could ever have worked in the real world, but its only chance was to have Skilling create a strong ethics and value structure in the company around rigorous and honest evaluation of current and past marks. And that sure as hell did not happen. If anything, Skilling behaved in the opposite manner, rewarding creativity in evading any loss and eking out new gains from re-marking past projects with rosier assumptions.

Enron’s reported results quickly began diverging from reality.  Reported results looked fabulous, as they were based on stacked project valuations that were in turn based on optimistic forecasts of dealmakers who had every incentive to be optimistic.  But at the same time many of the deals were crap – some just projects where core assumptions such as natural gas prices had not played out as expected but increasingly including dumb domestic merchant investments and even worse international projects.  Crap projects were being approved based on insanely optimistic forecasts that no one had an incentive to challenge. Eventually, in bankruptcy, outsiders would be staggered by how much of Enron’s investments were absolute money-losing garbage.  Enron’s strategy went awry almost from the beginning, but that fact was hidden by the bad measurement system and so Enron kept doubling down on worse and worse investments, with the inflated mark-to-forecast numbers convincing everyone, especially themselves, that they were brilliant.

At some point the divergence between mark-to-market project values and actual results could not be ignored, and with the refusal to mark project values down, some other alternative was needed.   The solution they found was via corporate weasel Andy Fastow, their CFO who created off-books entities to hide losses from bad merchant investments rather than mark the losses to market.  Because the bad deals created little cash, new borrowing was constantly required which again was dumped into off-books entities (which were made worse by Fastow’s gluttonous self-dealing). 

It is these deals that get most of the historic attention, but in my mind, the off-books fraud merely delayed (and magnified) the reckoning of an organization already set up for disaster by the incentives and measurement plan of mark-to-market accounting that Skilling put in place.

New Series:  Organizational Design, Behavior, and Change

Today I am starting a new series on organizational design and behavior and the very difficult process of creating organizational change.  To some extent this is a return to my roots on this blog, and to some extent it is an admission that I have no desire to hover around my computer for enough hours in the day to keep up with the incredibly rapid news cycle the Trump Administration is driving.

But that does not mean I intend this to be irrelevant to current politics.  At the core of a lot of what is hitting the news – DOGE investigations, government headcount reductions, government employee accountability – are issues of organizational design and behavior and of how organizational change can be managed.  Most of the media discussion of current actions by the Trump Administration is not at all informed by any reasonable assumptions about how organizations work or how they can be improved. 

To some extent I started this series out of order, beginning with a post on downsizing before I ever thought of this as a series.  I realized after I wrote that post that I wanted to take a step back and cover more background first.  In my next post I shall embark on the series, starting first with incentives and performance measures.