Like what you've read?

On Line Opinion is the only Australian site where you get all sides of the story. We don't
charge, but we need your support. Here�s how you can help.

  • Advertise

    We have a monthly audience of 70,000 and advertising packages from $200 a month.

  • Volunteer

    We always need commissioning editors and sub-editors.

  • Contribute

    Got something to say? Submit an essay.

 The National Forum   Donate   Your Account   On Line Opinion   Forum   Blogs   Polling   About   
On Line Opinion logo ON LINE OPINION - Australia's e-journal of social and political debate


On Line Opinion is a not-for-profit publication and relies on the generosity of its sponsors, editors and contributors. If you would like to help, contact us.


RSS 2.0

Why smart firms should experiment

By Andrew Leigh - posted Tuesday, 12 January 2010

In a British fruit farm, the seeds of a quiet revolution in management are being sown. While the neighbouring farms go about business as usual, the owners of this establishment see each season as a chance to run experiments aimed at raising productivity and profits.

Halfway through the 2002 season, the pay scheme for all fruit pickers was suddenly switched (PDF 905KB) from one that paid workers relative to their co-workers, to a scheme that paid based on absolute performance. Midway through the 2004 season, a managerial bonus scheme (PDF 479KB) was introduced, allowing the best managers to increase their earnings by around 25 per cent. In 2005, fruit-picking teams found their performance charted (PDF 554KB) at the end of each day. Later that season, the best teams began winning bonus payments.

In each case, the changes were engineered by a trio of young economists: Oriana Bandiera, Iwan Barankay and Imran Rasul. Tracking the changes, they found that rewarding workers for absolute performance raises productivity by a whopping 50 per cent, and that managerial bonus schemes boost output by 21 per cent. Yet when it comes to motivating teams, it turns out that charting performance is a bad idea (the underperformers become discouraged), while rewarding performance is effective (the best try even harder).


To see how radical this approach is, imagine how the typical firm would have approached the same problem. Step one: hire a team of well-dressed management consultants. Step two: pay six-figure bill. Step three: hold nose and implement recommendations.

Business experiments turn this logic on its head. Rather than paying for expert wisdom, the experimental approach says “try it and see”. Where the traditional literature is passionate and confident (with titles like “Straight from the Gut” and “Re-engineering the Corporation”), the experimental approach is modest and open-minded (with titles like “Team Incentives: Evidence from a Field Experiment”).

While the traditional approach listens to the consultant or the highest paid person in the room, the experimental approach listens to the data. Just as new drugs are evaluated using randomised trials, business experiments aim to bring both simplicity and scientific rigour to evaluating the impact of new strategies on the bottom line.

Perhaps the aspect of business that has made the best use of experiments is advertising. In formulating advertising campaigns, firms are increasingly recognising the limits to what can be learned from experts and focus groups. Rather than pick between two possible campaigns, the smarter approach may be to find test markets, and try both. This is most easily done in direct mail or online campaigns, where it is possible to get customer-level variation (indeed, Google’s AdWords software even allows small-time users to run such marketing experiments). But by exploiting variation across chain stores, or in regional markets, firms have found ways to test other marketing strategies too.

Business experiments have also helped to quantify the value of a strong track record. In a series of experiments on eBay, Paul Resnick and co-authors compare the prices of batches of vintage postcards sold by an established seller with a strong reputation versus a first-time seller. They find that first-time sellers receive prices that are 8 per cent lower, illustrating that reputation on eBay carries considerable financial value.

Experiments can even help firms decide how to set prices. Economist Steven Levitt gives the example of his work with a direct-marketing travel service, which experimented with randomly changing prices on its mailings, and found that demand barely fell when it raised prices. Not surprisingly, the company now charges considerably more than it did in the past.


Randomisation, Levitt argues, “cuts through the complexity”. He points out that “All you have to do is flip a coin and put people into treatment and control groups. In the end, you just have to be able to add it up and say, ‘Am I selling more or am I making more profit in the group that got the treatment versus the group that got the control?’”.

From fruit farms to travel agencies, experiments represent a powerful challenge to business as usual, who are increasingly embracing the opportunity to get better feedback on everything from their human resources policies to their pricing strategies. Why settle for grizzled wisdom when you can try it and see?

  1. Pages:
  2. Page 1
  3. All

First published in the Australian Financial Review on January 5, 2010.

Discuss in our Forums

See what other readers are saying about this article!

Click here to read & post comments.

3 posts so far.

Share this:
reddit this reddit thisbookmark with Del.icio.usdigg thisseed newsvineSeed NewsvineStumbleUpon StumbleUponsubmit to propellerkwoff it

About the Author

Andrew Leigh is the member for Fraser (ACT). Prior to his election in 2010, he was a professor in the Research School of Economics at the Australian National University, and has previously worked as associate to Justice Michael Kirby of the High Court of Australia, a lawyer for Clifford Chance (London), and a researcher for the Progressive Policy Institute (Washington DC). He holds a PhD from Harvard University and has published three books and over 50 journal articles. His books include Disconnected (2010), Battlers and Billionaires (2013) and The Economics of Just About Everything (2014).

Other articles by this Author

All articles by Andrew Leigh

Creative Commons LicenseThis work is licensed under a Creative Commons License.

Photo of Andrew Leigh
Article Tools
Comment 3 comments
Print Printable version
Subscribe Subscribe
Email Email a friend

About Us Search Discuss Feedback Legals Privacy