Military history

Chapter 34

The Sociological Challenge

I learned a great deal about military history and Confucian metaphors.

But the only practical advice that we were given was that every company should send teams of people from different disciplines to country hotels every year to think about the future.

—Participant in a fifteen-part course on business strategy given by a leading name in the field, quoted by John Micklethwait and Adrian Wooldridge.

We now need to follow the second strand in management scholarship, drawn more from sociology than economics, which was inclined from the start to consider human beings as social actors and organizations as bundles of social relationships. Although this strand had a separate course, there were overlaps with the economic strand in the challenge to managerialism and in the propensity to follow fashion. It was influenced by the counterculture of the 1960s in two respects. The first was distaste for bureaucratic rigidity and hierarchy. This challenged the processes of rationalization and bureaucratization, arguing that a new and more enriching form of organization needed to be devised. The second was the influence of postmodernism, not only in the critique of the modernist forms of rationalist bureaucracy but also in offering a completely new way of considering human affairs.

The critical anti-managerialist literature of the 1950s presented a monolithic, homogenized dystopian vision, only one step short of George Orwell’s 1984. The elites of large corporations were described as presiding over armies of white-collar workers, formed in their own bland—and obedient—image. During the 1960s and 1970s, however, demographic trends and lifestyle choices worked against conformity. The new businesses based on information and communication technologies often seemed to celebrate relaxed workplace practices and freethinking rather than crude hierarchy. Moreover there was a better anthropological understanding of organizations, the complex social formations that developed within and between individual units, and the incentives for individuals to develop practices that satisfied their needs as much as those of the organization for which they were supposed to be working.

The human relations school provided the foundations for this work, but it moved on after the war and turned into a rich field of organizational studies. Once organizations began to be viewed as social systems in their own right rather than as means to some management goal, questions arose not only about how this insight could lead to greater efficiency—which had been the concern of Elton Mayo and Chester Barnard—but how organizations could be arranged to make for a more fulfilling life for the workforce. This also fit in with a trend for individual pathologies to be explained by reference to their social settings. Structures that encouraged harmony, solidarity, and support should also therefore promote general well-being. An example of this was the book by the influential British social psychologist, James Brown, who after his experiences in the army and industry had concluded that mental illness was more of a social than a biological problem. He argued that organizations should be judged by their social as much as their technical and economic efficiency.1

Douglas McGregor’s The Human Side of Enterprise opened with the question, “What are your assumptions (implicit as well explicit) about the most effective way to manage people?”2 He offered two alternative theories. Under Theory X, which had developed with the factory shop floor, the presumption was that people disliked work and preferred direction rather than initiative, and so they must be controlled by means of threats and rewards. Under Theory Y, individuals wished for fulfillment and responsibility, and if offered the chance, they would commit themselves more thoroughly to the organization. He developed these ideas while on the staff at MIT and then had a chance to put them into practice as president of Antioch College. While he found support for his theory, the experience of coping with fractious students and faculty convinced him of the need for active leadership. He had believed, he later recalled, “that a leader could operate successfully as a kind of adviser to his organization. I thought I could avoid being a ‘boss’ . . . I hoped to duck the unpleasant necessity of making difficult decisions . . . I finally began to realize that a leader cannot avoid the exercise of authority any more than he can avoid responsibility for what happens to his organization.”3 He did not, however, reject his more humanistic approach to management or embrace authoritarianism. While critics might have worried that the dichotomy between Theory X and Theory Y was too sharp, and that actual practice would be contingent on circumstances, McGregor appeared as a champion of consent against coercion, the democratic against the autocratic, the active against the passive.

Herbert Simon’s ideas of bounded rationality encouraged a realistic assessment of how managers actually went about their business.4 Another organizational psychologist, Karl Weick, challenged standard models in his book The Social Psychology of Organizing by demonstrating how uncoordinated and apparently chaotic systems could nonetheless prove adaptable when faced with the unexpected—more so than systems geared to assumptions of linearity. Weick drew on a range of disciplines, and introduced into the lexicon concepts such as “loose-coupling” (a distance and lack of responsiveness between individual parts of an organization created a form of adaptability), “enactment” (how structures and events are brought into existence by individual actions), and “sensemaking” (the processes by which people give meaning to experiences). Sensemaking was necessary because individuals must operate in inherently uncertain and unpredictable environments (“equivocality”). There were a variety of ways individuals could make sense of things, and his work focused on the different forms communication could take within an organization, notably in the face of external shocks. Weick’s theories were, however, complex and did not offer the easiest read. His definition of an organization, for example, was “the resolving of equivocality in an enacted environment by means of interlocked behaviors in conditionally related processes.”5

Business Revolutionaries

The idea that management should focus on the softer side of organizational life came to be developed and promoted by two McKinsey analysts, Tom Peters and Robert Waterman. The starting point was the pressure felt by McKinsey’s in the late 1970s to come up with a credible response to Henderson’s Boston Consulting Group. Peters, who had recently returned from completing a Ph.D. at Stanford in organization theory, was asked to work on a project out of the San Francisco office that addressed “organization effectiveness” and “implementation issues.” At the time McKinsey’s was still working largely with Chandler’s concept of structure following strategy. At Stanford, Peters had been influenced by the work of Simon and Weick, both of whom challenged simple models of rational strategy formation and decision-making. He was joined by Waterman, who was also heavily influenced by Weick (“mesmerized,” according to Peters), and wanted to reshape the way McKinsey’s thought about organizations. One weekend with Tony Athos of the Harvard Business School and another McKinsey’s consultant, Richard Pascale, who had been working on the success ofJapanese firms, they developed what came to be known as the “7-S framework.” Athos insisted— correctly, as it turned out—that any model had to be alliterative. A memorable shape was also required, in this case demonstrating, in contrast to the idea that strategy drives structure, that no a priori assumption could be made about which of the seven would make the difference at a particular time. The seven S’s were structure, strategy, systems, style, skills, staff, and the somewhat awkward “superordinate goals.”

The model was launched in a 1980 article. “At its most powerful and complex,” the authors suggested, “the framework forces us to concentrate on interactions and fit. The real energy required to re-direct an institution comes when all the variables in the model are aligned.”6

Athos and Pascale used the model specifically in a Japanese context. They argued that the Japanese scored on the softer side of management, by developing a sense of common purpose and culture in ways that American management had forgotten, if it had ever known.7 A translated book, originally published in 1975, by Kenichi Ohmae, who had been head of McKinsey’s Tokyo office, explained how strategy in Japan would not come from a large analytical department, fully formed in terms of rational, structured steps, but as something more ambiguous and intuitive, relying on a key figure with a grasp of the market whose ideas could be grasped in terms of the organization’s culture.8

The most important book to emerge using the model was Peters and Waterman’s In Search of Excellence? Their book was presented as an answer to a straightforward question: what makes an excellent company? Possible candidates were identified by what appeared to be a sophisticated methodology. Sixty-two companies that appeared fairly successful were evaluated according to six performance criteria. The forty-three truly successful companies were those that were above the fiftieth percentile in four of the six performance metrics for twenty consecutive years. These were then studied in more detail, with key executives being interviewed. Out of this they distilled eight shared keys to excellence: a bias for action, customer focus, entrepreneurship, productivity through people, value-oriented CEOs, sticking to the knitting (that is, do what you know well), keeping things simple and lean, and simultaneously centralized and decentralized (that is, tight centralized control combined with maximum individual autonomy).10

Twenty years after publication, Peters acknowledged that the research that had gone into the book had been unsystematic though he remained convinced by the message.11 The book was, he claimed, “an inflection point—a punctuation mark—that signaled the end of one era and the beginning of another.” The target was not so much the Japanese as the American management model. Peters described his motivation at the time and since as being “genuinely, deeply, sincerely, and passionately pissed off!” His targets included Peter Drucker, because he encouraged “hierarchy and command- and-control, top-down business operation” and organizations in which everyone knew their place, and Robert McNamara, besotted by systems at the Pentagon which had led to people being “driven out of the equation.” A third target was Xerox Corporation, where he had worked as a consultant, which to Peters demonstrated all that was wrong with the modern corporation: “the bureaucracy, the great strategy that never got implemented, the slavish attention to numbers rather than to people, the reverence for MBAs.” He therefore saw the book as challenging “Management 101” based on Taylorism, reinforced by Drucker, and implemented by McNamara. He objected in particular to the bean-counting mentality focused entirely on numbers and finance. “The numerative, rationalist approach to management is right enough to be dangerously wrong, and it has arguably already led us astray.12

Waterman provided a slightly different, although not contradictory, account. In an article he coauthored, published in 1999, claims were made about the role of the book in translating the key themes in organizational studies, to the point of describing it as an accessible version of Weick.13 They addressed the issue of whether it was possible to simplify without being simplistic. Even if the situation demanded complex theories, managers would not find them interesting and so good theory would not affect practice. The article claimed immodestly that In Search of Excellence succeeded by saying “pretty much everything there was to say about behavior in organizations and got it right, by virtue of the experts cited.” Ideas of learning organizations, bounded rationality, narratives, and agenda-setting could all be found, with key theorists getting mentioned. Yet a description of the key messages suggested a set of values as much as scholarly findings, for example that “it’s OK for guys to have feelings”; “don’t take yourself too seriously”; it’s “not your fault” if the world does not look neat and tidy; and “people who espouse rational models of decision-making want you to feel responsible for the disorder in the world, but don’t for a moment let them get away with that silliness.”

Whether or not this was truly an act of translating academic theory for practitioner consumption, the account of the book’s gestation did reveal the effort that went into ensuring its appeal. There were some two hundred briefings to managerial audiences before publication. “During this process it became apparent that if the examples were retold in the form of a story then they compelled attention and promoted retention.” Their audiences were averse to “numbers, charts and graphs,” and also to “mid-level abstraction.” Feedback also suggested that the original twenty-two attributes seemed too many, so they were whittled down to eight. The original number was seen as “too confusing not to mention also antithetical to the basic premise that it isn’t as complex as you think if you pay attention to people!”

The book’s positive message (America did have excellent companies) and uplifting prescription for success (work closely with your staff and customers and do not get bogged down with committees and reports) was a runaway success. It was the first business book to become a national bestseller, and eventually sold well over six million copies. Neither author stayed long at McKinsey’s. Peters, resenting the patronizing attitude of the New York headquarters toward the marginal endeavors at the San Francisco office, had left before the book was published and was soon in demand as an inspirational, though expensive, speaker. His style, in speaking and writing, was dramatic and extravagant. The message and its ebullient communication were more important than the method. Whatever the original sources, In Search of Excellence relied on anecdote and secondary material rather than hard research.14 It had failed to identify a reliable basis for sustainable growth or even survival. The excellent companies often struggled: soon after the book was published, a third were reported to be in financial difficulties.15

Instead of numbers, bureaucracy, control, and hard metrics, Peters and Waterman argued for people, customers, and relationships, which were much softer but could explain how things actually got done and what was accomplished. Business should be about heart, beauty, and art—not some “disembodied bloodless enterprise” but “the selfless pursuit of an ideal.” As with most revolutionaries, the creative and destructive were never too far apart. In Liberation Management, an explicitly countercultural title, Peters wrote: “R-I-P. Rip, shred, tear, mutilate, destroy that hierarchy.”16 In 2003 he asserted that “a cool idea is by definition a Direct Frontal Attack on the Holy Authority of Today’s Bosses.” 17 Peters was undoubtedly a Theory Y man. A constant theme in his many books was to emphasize the positive side of work and argue that companies that cherished and encouraged this side would do better than those who suppressed their employee’s creativity by trapping them in doleful hierarchies and assessing them against soulless metrics. Beyond that there was not a lot of consistency. He made the point himself when opening his 1987 book Thriving on Chaos by observing that there were “no excellent companies.”

He was hardly alone in pointing to the need for flatter structures; units with more autonomy; and attention to quality, service, and innovation—not just to cost. Nor did he even claim much influence for himself. He opened a 2003 book proclaiming himself “madder than hell.” He had “been screaming and yelling and shouting about bankrupt business practices for 25 or 30 years . . . mostly to no avail.” Notably this book began with the army (about to go into Iraq but not yet experiencing real difficulties) as an innovative organization. He had already shown an interest in John Boyd and now he embraced the revolution in military affairs with its combination of “greater battlefield flexibility and greater information intensity,” the decentralization and networking, the pursuit of indirection in strategy. He did not note the additional need for an operational environment that would allow the army to play to its strengths, rather than have the irritating “asymmetry” of an opponent playing to different rules.

Peters could express the frustrations of the functionary stuck in a cubicle, as he had been the neglected bright spark in a secondary regional office, too far down the management food chain to be able to exercise influence and put right all those things self-evidently going wrong. Much of his success was in expounding on the need for more humane and “cool” enterprises in countless speeches and seminars “with the exuberance and evangelistic zeal,” according to the Economist, “of a 19th-century cough-syrup salesman.”18 Others spoke with both awe and alarm at how he turned management theory into something “so personal, so spiritual, so impractical.”19 This quasi-religious theme was the reason why Peters, and other leading management thinkers, came to be known as “gurus” (from the Sanskrit word for a teacher who could introduce light where there was darkness). Drucker, who came to be retrospectively described as the first of this class, disliked the term, observing sniffily that “guru” was used “because ‘charlatan’ is too long to fit into a headline.”20

Gary Hamel had similar targets to Peters and a similar commanding presence at high-priced seminars. He worked in business schools and as a strategic consultant, and was regularly named as one of the top—if not the top—gurus. His focus, at least initially, was much more explicitly on strategy. His starting point was the transformation of the business environment as a result of deregulation, the decline of protectionist pressures, and the impact of information technologies. These opened up markets and introduced a new fluidity, requiring companies to be very clear about what they were good at but also agile enough to see opportunities for new types of markets and different sorts of business relationships. Those who stuck to the old models were doomed to fail; those who embraced the new had a chance.

Hamel originally gained attention with a series of articles with C. K. Prahalad, a professor at the University of Michigan, where Hamel had been a doctoral student. Together they attacked past strategic constructs, mocking the various qualities adumbrated by the consultancies and the business schools and suggesting that companies were trying to cope with the Japanese challenge by looking at surface features rather than the underlying concepts from which their competitors derived their “resolution, stamina, or inventiveness.” They cited Sun Tzu: “All men can see the tactics whereby I conquer, but what none can see is the strategy out of which great victory is evolved.” From strategic intent, once identified, could be derived a sense of direction, discovery, and destiny.21 Their notion of “core competence,” which suggested something more straightforward than turned out to be the case, was described as the “collective learning” in the organization. This was not so much about doing one thing well but about coordinating diverse skills and integrating streams of technology.22 In a 1994 article, they claimed that the discontinuity in business practice was now so great that the various strategic concepts developed during the previous couple of decades—by Porter, for example—were no longer valid. They had assumed stable industrial structures, focused on business units, relied on economic analysis, and separated strategic analysis from its execution, which was presented as an organizational matter. Instead, Hamel and Prahalad argued for an approach that recognized the major transitions in industrial structure then underway, acknowledged the interplay of economics with politics and public policy, and involved those charged with executing strategies in their original design.23

Hamel’s explicitly revolutionary turn came two years later. Although the medium was the Harvard Business Review, Hamel invoked Martin Luther King, Nelson Mandela, Gandhi, and even Saul Alinsky. Corporations, he argued, were reaching the limits of incrementalism. Everything now was at the margins, so there might only be a bit extra market share and a bit less cost, a bit faster response to customers and a bit more quality.24 Hamel assumed his audience would not be satisfied with just getting by. They were unlikely to be the rule makers, the big companies who were the creators and protectors of industrial orthodoxy, but they would not be satisfied with being mere rule takers, those following behind for whom life was bound to be hard. Better to be among the rule breakers, the “malcontents, the radicals, the industry revolutionaries.” They could overturn the industrial order because they were shackled “neither by convention nor by respect for precedent.” The various trends that had opened up the international economy, coming under the heading of “globalization,” meant that this was the time for the revolutionary. To those managers clinging to the status quo he raised the specter of being left behind in the revolutionary tide. In this vision, the only role for strategy was to create the revolution. “Strategy is revolution. Everything else is tactics.”

To be revolutionary, it was necessary to rethink the business. In this respect he echoed Mintzberg’s castigation of strategic planning, which took the boundaries for granted and failed to look for the opportunities in new, uncontested space. With an elitism that hampered any capacity for discovery, the planners harnessed “only a small proportion of an organization’s creative potential.” By not engaging the lower reaches of the organization, senior management encouraged reaction, as change became a “synonym for something nasty,” something to be feared, imposed from above. So strategy-making had to be democratic. This was where Hamel quoted Alinsky, who had decried elitist planning as anti-democratic, as “a monumental testament to lack of faith in the ability and intelligence of the masses of people to think their way through to the successful solution of their problems.”25

Hamel did not deviate from his core theme, that the old strategic model was as outdated as the business model it sought to help. His 2000 book, Leading the Revolution,26 developed what had become familiar themes but with the motivational, inspirational style expected of a business guru, suggesting that the only limits to what they could accomplish lay in their imaginations. This was not a book, he insisted, about “doing better” or for “people who want to tinker at the margins.” Instead, it was an “impassioned plea to reinvent management as we know it—to rethink the fundamental assumptions we have about capitalism, organizational life, and the meaning of work.”

Unfortunately, he adopted Enron as his company of choice. Enron had transformed itself over the 1990s from a pipeline company to an energy trader, using its expertise and muscle to buy and sell contracts. Hamel celebrated Enron as a company that had “institutionalized a capacity for perpetual innovation” and as “an organization where thousands of people see themselves as potential revolutionaries.” He became chair of the Enron Advisor Council. Enron’s management had a suitably populist rhetoric (“power to the people”) and claimed to have empowered its employees, describing them all as fellow revolutionaries.27 It adopted Hamel-type themes, including likening its quest for free markets with the civil rights campaign of the 1960s, and challenged all conventional assumptions about how businesses should operate. Enron was celebrated as having found a way to extraordinary profits through forms of integration and agility that had eluded others. But the company collapsed at the end of 2001, taking auditor Arthur Anderson down with it. The source of its major profits was exposed as fraud, helped by deals of such complexity that nobody quite understood what was going on. It had made a political push for deregulation of energy markets, ready to accuse any external analyst who expressed doubts about its claims as being ideologically antagonistic. Hamel expunged Enron from the second edition of his book and could argue that he was by no means alone in being caught out by the elaborate efforts undertaken by Enron’s senior management to hide its debt and its vulnerability to a deteriorating trading position.28

In a 2003 book, Hamel complained that companies were being driven by “the theorists and practitioners” who had invented the rules of “modern” management a century earlier. Contemporary managers were still beholden to the ideas of Frederick Taylor and Max Weber (whose ambivalent attitude toward bureaucracy Hamel was apparently unaware of). The old management model had become dysfunctional in a world where the need was for flexibility and creativity. Instead of the “stultifying” focus on the bureaucratic values of “control, precision, stability, discipline and reliability,”29 he sought innovation, adaptability, passion, and ideology. Reflecting the traditional romantic reaction against rationalism he urged organizations to be more like communities, dependent “on norms, values, and the gentle prodding of one’s peers,” offering emotional rather than financial rewards.30 Martin Luther King’s most famous speech was invoked, as Hamel described his own dream in which “the drama of change is not accompanied by the wrenching trauma of a turnaround . . . An electric current of innovation pulses through every activity . . . where the renegades always trump the reactionaries.” What he was careful not to do was predict the future of management. His aim, he insisted, was “to help you invent it.” A later book, which addressed directly questions of norms and values in business, captured the underlying complaint: “There’s nothing wrong with utilitarian values like profit, advantage and efficiency, but they lack nobility.” Organizations needed an uplifting sense of purpose and individuals an allegiance to the “sublime and the majestic” and a cause greater than oneself.31 Although Hamel began writing about strategy, he had veered into broad social theory. The analysis had become almost a parody of Theory X and Theory Y, pushing dichotomies to their limits, community versus bureaucracy, renegades versus reactionaries, innovation and change versus stability and order, emotional rewards versus financial rewards.

The underlying propositions could be rephrased in terms of classic radical thought, demanding the upending of obsolescent hierarchies so that shackles could be removed, productive energy and imagination could be released, and all could realize their potential. But this was always a strange revolution, certainly more bourgeois than proletarian. As it was never a real movement, it lacked institutional expression. It reflected the counterculture’s revolt against rationalism and bureaucracy, a yearning for passion and the play of imagination, and the urge to trust in feelings and experience, assuming that the best things happened spontaneously. But, as with the counterculture, this was a false prospectus. It exaggerated the democratic possibilities of a business organization. There was also the same presumption that participatory democracy would not lead to reactionary and myopic policies but instead to the most progressive, in this case the sort that a sagacious strategic consultant might advise.

Work could be fun and exciting; full of challenge and innovation; with congenial, stimulating, and supportive colleagues; it could also include essential but boring tasks, pressing deadlines and tight budgets, angry customers and slipshod suppliers, irritating co-workers and myopic bosses. It was one thing to recognize the value of a workforce and regret that too much of it was left untapped; it was quite another to suggest that the inspired subordinate, with drive and imagination, could subvert power structures, recast cultures, and reshape institutional systems. Common sense argued for engaging employees earlier in company decisions and drawing upon the expertise of those actually running the key processes before overhauling them. Only at the top, however, was it possible to take an overview of all aspects of a company’s activities, make authoritative decisions, allocate resources, and accept responsibility.

This is why corporate claims of higher purpose were often treated cynically. Occasional transformational change might be exciting, but too much could also be exhausting. Some calm and stability might be welcomed. Structure, discipline, and accountability were necessary for the innovative to make changes and then sustain them. Many employees would assume their senior management should work out the strategy and would prefer not to be badgered for new ideas that were then ignored. The need for an antidote to the soaring rhetoric of the gurus and the exaggerated claims of the consultants was reflected in the popularity of Scott Adams’s subversive cartoon strip “Dilbert” with its world of persecuted engineers, fantasizing marketeers, stupid bosses, and greedy consultants. Consultants, Adams observed, “will ultimately recommend that you do whatever you’re not doing now. Centralize whatever is decentralized. Flatten whatever is vertical. Diversify whatever is concentrated and divest everything that is not ‘core’ to the business.” In Dilbert’s world, companies needed strategies “so the employees will know what they don’t do.” Dilbert explained how he put together a strategy: “I collected optimistic data, put it in the context of bad analogies, seasoned it with saliency bias . . . added herd instinct, a pinch of confirmation bias.” When his company announced that it would abandon a strategy of making good products in favor of a “desperate strategy of mergers, business spin-offs, fruitless partnerships, and random reorganizations” and an accelerated “program of paying the good employees to leave,” the stock price went up by three points.32

If you find an error or have any questions, please email us at Thank you!