BPM State of the Nation 2009

Filed under: by: Art Style and Design

Written by Terry Schurter
From it’s beginnings in the early 90’s to the boom in the early 00’s – where is BPM at in 2009?

While BPM stands for “Business Process Management” the definitions of Business Process Management in use in the marketplace have dramatic differences in their meanings. So the term itself does not define a specific problem, approach or set of activities. That definition comes from the person or organization using the term as they place “BPM” within the context that they care about.

The biggest disparity exists between the commercial software market and the business pundits who define BPM in completely different ways. On a historical note, BPM did indeed start as a management practice, coming directly out of the more experienced practitioners of Business Process Reengineering. From these people, Business Process Management was originally conceived of as a way to better manage the business operations of large commercial entities.

Early entrants into the BPM software market created new “work flow” tools that often required little code development to express “business processes” heavy with human interaction. The belief within the early BPM software community was that by providing business users a simple set of tools with which they could model these human-centric processes, the processes could then be “orchestrated,” dramatically improving the process efficiency and quality for these business operations.

Early adopters of these products often applied them to a unique subset of business processes indeed heavy with human interaction, where little technology was in place, and where consistency in the process was poor at best. The case studies that arose from using BPM software on these processes were very, very compelling.

As the market began to gain widespread attention and mainstream media coverage the larger software vendors began to eye the market with interest. Between acquisitions, product development and repositioning of existing products a fundamental shift started to take place.

That shift is almost complete now. What was the shift? The BPM market shift is the move from the original definition of business processes to one that focuses almost exclusively on IT systems and IT operations. The shift occurred for several reasons.

The most important reason is that the BPM tools in the market, at least those with work flow engines, were found to be ill suited for most human-centric processes. They weren’t the right tools and they still are not the right tools. If we have learned anything, we have learned that BPMS products are actually the wrong tool to use for improving business processes heavy with human interaction.

Now if you’re a software vendor what are you going to do? Focus on where you can be successful, where you can sell your products, and just make sure your “story” is properly prepared for your audience. This is exactly what has happened and the messaging from this market segment has purposely redefined the term “Business Process Management” on a broad basis to fit what is effectively nothing more (or less) than the next iteration of Application Integration software and intimately linked to another IT-centric concept – Service Oriented Architecture (SOA). This is where the vast majority of vendors and market share in the BPMS market exists today, and is the basis for revenue, market growth and market predictions cited by industry analysts.

The one hold-out in the software market is Business Process Analysis software. Many of these products still provide value-added capabilities for improving human-centric business processes although the technology is used by people to identify opportunities for improvement - not to implement the improvements or perform work flow functions.

Those “business” business processes have been left to the Business Process Analysis software market and even more so to the business consulting market. This is reflected in the commercial market break-down as follows:

BPM Commercial Market Segments

1) Executable BPM software:
Software that is executable, meaning it moves data (in one of a number of forms) from interaction to interaction (between any combination of systems and people).

2) Non-Executable BPM software:
Software that is used to plan, manage, architect, analyze and visualize “processes.”

3) Technical Consulting services:
Services that design, manage, implement and maintain executable BPM software and/or other software to achieve a similar result.

4) Process Consulting services:
Services that help organizations address problems from a “process” perspective; deal with the change requirements associated with “business” changes, and lead improvement initiatives.

BPM Focus Areas

The focus areas of each of these segments in the commercial BPM market sheds much light on how the different problems that people face are fulfilled by different segments of the BPM commercial market. In fact, it is the problem area commonly dealt with by each of the market segments that tells us what we should expect to “get” from expenditures in each one. Knowing what problem we are attempting to solve, what value we will receive from that, and what problems we aren’t going to solve is a critical milestone in the evaluation process.

Organizations that fail to perform this analysis have a far greater likelihood of suffering disappointment, failure to reach goals, disillusionment and an ongoing sense of frustration. To date there has been no significant increase in proper leading analysis of BPM even though the market is more mature and process initiatives are more likely to successfully complete.

While the commercial market overall continues to grow at a steady pace, there is a general frustration from “insiders” – at least within the Executable BPM software and Technical Consulting services segments – that the market just won’t seem to “pop.” The majority of third-party analytical and research markets continue to predict a sharp increase in BPM market growth. Again, it is the focus areas that help us bring this into perspective and highlights the factors inhibiting market “explosion.”

Systems-based Cost Reduction (Integration)

Commercial Market Segments: Executable BPM software, Non-Executable BPM software, Technical Consulting services

The systems-based approach to cost reduction is based on the perspective of the technology in use behind the processes in our organizations. In this approach to BPM cost reduction, the processes in the organization are looked at from the perspective of the technology that supports them and the role the process serves. Then the process is “moved” to a more organized, streamlined and updated process model to reduce the operating cost of the process.

Cost reduction benefits from systems-based BPM initiatives come from a mixture of automation, process streamlining, error reduction, elimination of duplicity, and better control over processes at both the work level (people doing work in the process) and management level.

Return on Investment (ROI) in systems-based BPM cost reduction is more often than not a workload reduction. It is extremely important that we understand this result, as in and of itself workload reduction does not translate directly into tangible ROI.

Workload reduction potentially increases capacity and it frees up resources for other work to be done but in most cases is does not reduce the actual operating costs of the business in any significant way. That gain is usually only achieved when the reduction in workload (i.e. resources now available for doing other things without increasing costs) can be used to accomplish something else of financial benefit to the organization. In most cases organizations taking the systems-based BPM approach are making this linkage at the departmental level – or not at all – limiting the real benefit the organization realizes from the BPM initiative.

Software companies that provide configurable applications with embedded processes in them (like ERP, CRM, etc) and BPMS software products are also focused on this form of cost reduction.

System-based cost reduction approaches the opportunity of cost reduction very similar to the approach taken in manufacturing. In manufacturing we make things. Those things are made using machines, and the act of making them is a process. Anything we can do that reduces our cost in making our “things” is an improvement – as long as it doesn’t reduce quality and it doesn’t create new costs.

The Executable BPM market is now predominately focused on this problem-solution. This is why Executable BPM software is more often used in back-end systems, IT, HR and customer service areas. Back-end systems and IT are really the same problem. If you think about the plumbing, electricity, networking, communication, etc. for homes as the home’s “back-end” these services have to be there and they need to work right - often times better than they do now. In our businesses, this also includes many of the “appliances “we have in our organizations including computers and the programs we use on those computers every day.

This makes a lot of sense. Back-end systems and IT “processes” are already within the ownership domain of the people who approve such software purchases.

HR processes are usually either not well understood to start with or generally not very well organized. HR and other indirect process areas that do not handle the finances of the company are generally perceived as “necessary overhead” without being of “critical dependence” in the successful operation of the organization. The low risk and perceived benefit arising from improving these areas is very hard for many managers to resist. However, benefits from these initiatives have rarely stood the test of measurable ROI as a financial impact and often times the initiatives run into change issues that compromise the original goal.

Because Customer Service is looked at by most companies as a Cost Center, there is a very strong desire to use less expensive labor, reduce inefficiencies wherever possible, and automate pretty much anything that can be automated – even if it shouldn’t be. This is the reason why customer service is often targeted in BPM initiatives although it is not a “back-end” systems issue. The vast majority of implementations in customer service where costs are reduced have the knock-on effect of reduced Customer Satisfaction – due to the emphasis on internal measures.

Process-Based Cost Reduction (Change)

Commercial Market Segments: Non-Executable BPM software, Process Consulting services

When the focus moves to changing how work gets done in the organization the commercial market immediately moves away from the IT professionals directly into the arena of Business Analysts and Business Consultants. To help place this into perspective it should be noted that the process-based cost reduction focus is where we find the various improvement methodologies such as Six Sigma, Lean, TQM, CEM, Change Management, and so on.

While these process initiatives were originally at the center of the “bulls-eye” for the Executable BPM software market segment such is no longer the case for the reasons already discussed. There are also indications of a new BPM market beginning to form that attempts to directly address the problems associated with managing and changing human-centric processes. That market will be covered at the end of this report.

Process-based cost reduction seeks to improve existing “processes” by refining how work gets done in the organization. This is why Six Sigma and Lean have quickly been “crossed over” into the process market as these improvement methods or philosophies are specifically intended to help organizations optimize existing operations heavy with human interaction.

However, this market has also fallen dramatically short of its perceived value creation. The problem again comes from the fact that like BPMS products; Six Sigma, Lean, TQM, etc. are based on taking the “AS IS” process and refining it to an improved design of the “TO BE” process definition. The reason why this approach is so limited is that in general the philosophical approach for these types of techniques accepts the basic process shape (the AS IS) as valid, seeking to refine the process shape within the current constrained model of the process.

Change Management and CEM are the two areas where this is not the case. In these approaches the incoming perspective is based on the idea that there are fundamental changes that must be made to bring business processes into alignment with the current environment. The assumption within these approaches is that how work gets done now is based on assumptions that may have be relevant at some time in the past but are no longer relevant to how we need to do business today. Change management tends to look at this from an isolated perspective that assumes succeeding in bringing processes into relevance is a complicated and lengthy endeavor that is performed periodically to “update” the organization to the modern paradigm. Approaches like CEM look at the need to change on a regular basis in very short time frames with the idea that the “approach” becomes an ongoing behavior of change in step with the changes in context as they occur.

The other impact of significance is the issue of AS IS process complexity, where understanding the “model” requires a high level of attention and focus, leaving little to the challenging of the process as a holistic entity (like the adage that sometimes we can’t see the forest for the trees). Even though Change Management does seek to challenge the “status quo” it traditional falls prey to the complexity issue. CEM is the only practice in the market that deals directly with the issue of complexity.

No Bridge for the “GAP”

Much has been said about the need to bridge the gap between business interests and the application of technology. Business Process Management was originally believed to be a means to do just that, along with the cost reduction and quality improvements expected from it.

That has not happened – not at all. While in the early days of the BPM software market there was significant effort placed on helping to address the “gap” issue, the movement by the majority of the BPM market to primarily Application Integration has dissolved any gains made by early market leaders. Now, rather than being a bridge across the business-technology gap, for most of the Business Process Management software market BPM tools have become part of the gap - and a contributor to the dysfunctional affects that having the “gap” imposes on the organization.

One of the most obvious movements in this regard is the now broad adherence to industry standards and best practices that have arisen purely out of the logical, rational, and structured approach required in the use and application of technology. What has now become for many the very “fundamentals” of Business Process Management are completely ignored by the business people on the other side of the gap due to the lack of relevance to their perspective, work activities, interests and concerns.

Another part of the change in the BPM market is the number of people involved in propagating certain beliefs. Because the software market and the adjunct markets it has spawned off (media, research, conferences, etc.) are extremely well financed there has been a tremendous amount of money spent in developing a large consensus of opinion in regards to a “BPM Market Message.” This consensus echoes the technology viewpoint and has caused many business people to back away from any technology associated with Business Process Management.

The simple fact is that the gap between business and technology is now wider than it was 10 years ago and growing. More and more business people are turning a deaf ear to any discussion around BPM that includes technology. The commercial market has solidified on the split, with one side serving IT interests and the other side serving business interests. BPM as a commercial offering is no longer part of the solution to “bridging the gap.” Depending on the type of product or service consumed from within the BPM market, Business Process Management is either addressing Business Issues or Technology issues – not the disparity between the two.

Lessons Learned

There is, however, what could be the beginnings of a silver lining in the BPM clouds beginning to emerge. From lessons learned in attempting to address business issues and bridging the business-technology gap certain things have been learned, and some steps to address these issues have begun to emerge in the software market.

1) Lessons Learned – The Need for Individual Process Orchestration

What have we learned? We have learned that business activities do not conform to the traditional work flow technology paradigm. What really happens is that the “processes” business people are engaged have a high degree of individual variance and adjustment to contextual influence that affects how work gets done on a daily basis. We now know that in general business processes (as the concept was originally defined) are fraught with the high degree of variability that occurs anytime people are the primary performers in the actions of a process.

Therefore any technology used to support these processes must also have the ability to flex and change on an as needed basis. This includes the ability of supporting technology to flex and change per individual as they are doing their work – enabling people to “orchestrate” their own behavior in the process.

We also know that for this kind of individual process orchestration to work within supporting technology the individual orchestration cannot impose additional “work” on the individual. This means that supporting software for these business processes must grant individual users the ability to “do” their individual process orchestration by simply doing their work their own way. Several niche players in the BPM software market now offer this ability in at least some form within their products, primarily from vendors that are focused on solving specific business issues faced by their target market.

2) Lessons Learned – Work Flow Queues are Extremely Limited

One of the common approaches in BPMS products is to build work flow models of processes and then present the work each person is supposed to “do” in a work queue interface. While this works well when the general concept of a task list is already part of the existing work metaphor it does not work well at all when the task list metaphor is not already in place.

What we have learned is that presenting new software interfaces to people in general creates a change issue that must be dealt with. When changing an existing interface from a design that is difficult to use and unwieldy for the people it serves - a simple, well designed and more useful interface will present a change impact that is very low.

When changing from an existing interface to a new one that is not obviously simpler and easier to use (to the people using it) the change impact increases dramatically. Basically, when we change these interfaces in a way that the people using them immediately can see the benefit, the change impact is very low. As soon as we expect people to learn something “new” before the “benefit” becomes noticeable the change impact becomes significant.

Where no task list metaphor exists, imposing a new interface on the people doing the work represents a very high change impact. Again, if the benefit to the people who must use the new interface is obvious to them then the change impact becomes low but if that benefit is not obvious the challenge in getting people to make the change will be very high, very costly, and is highly likely to significantly reduce the benefit expected from the implementation of the underlying technology.

What we now know is that somehow Business Process Management technology for business users must move away from the general concept of “work queues” and “work queue interfaces.” What business users need is something that works within their current technology comfort zone to help them stay “in sync” with their work in a way that does not impose new activities on them. Because this can take many different forms and be supported through many existing technologies already in use by the business the means by which this is accomplished is highly variable. Numerous BPM vendors have approached this as an extension of their basic concepts of queues and work flow with interfaces into email, email clients, and mobile devices but there is still a long way to go in really solving this problem.

3) Lessons Learned – The Need to “Capture” Reality

As discussed earlier, the processes business users engage in are highly personalized and variant. When we “solve” the issue of how to enable them to engage with underlying BPM technology in a way that allows them to self-orchestrate their behavior in a process (without thinking about it that way) we will have the opportunity to start capturing behaviors of our people that we can learn from to improve our overall business operations.

This is actually brushing against something entirely new in the world of business software. Rather than embedding “domain knowledge” into a process so that we can impose “best practice” on everyone in our organization there is the potential to capture significant behavioral information that can be related to both context and results.

This behavioral information related to context and results represents a wealth of learning we can tap to improve our organizations. Why did one sales person achieve such high results in a given market during a time when others did not? Why does one department, shift or group get numerous complements from our customers when others don’t? Why are certain people always on time with their work when others are not? These types of questions can start to be answered when we reach the place where our underlying BPM software is capturing individual behavioral nuance within context and in connection with results.

Again, in a few cases there are BPM software products that have touched on these kinds of benefits but only at an extremely simplified level. The problem is complex, because to actually capture this kind of information the underlying BPM software must effectively be “unnoticed” by the user. This represents a fundamental shift in how we think about BPM software although it is the future of the product genre - though it may well be called something entirely different to help distinguish it from the Application Integration message dominant in the commercial market today.

4) Lessons Learned – Unstructured or Free-forming Processes

We have also learned that a number of our highest value processes are really ad hoc processes that form within a known framework of relationships. These processes do not have a predetermined start and end point, they form where they are needed, when they are needed and exist until they complete.

For example, a new product offering may start in a number of places such as product development, marketing, research, strategy or executive planning sessions. Depending on what the new product is and how it relates to the other products of the company the “process” may involve any number of functions, departments and activities many of which can be performed in parallel.

From a business perspective what we want is for these processes to operate as smoothly as possible, with timely communication, information sharing and event awareness. Because the relationships behind these ad hoc processes can be identified there can also be structure placed behind these kinds of processes that simplifies both the creation and execution of these ad hoc processes. The challenge though is that these types of processes do not fit the classic BPM work flow perspective of process and are ill suited to BPMS products. While there are a number of interesting technology twists that can help address this issue already in the market they are seldom even given the label of “BPM” and generally do not address all of the challenging aspects these kinds of processes create for the organization.

5) Lessons Learned – The Customer is Nowhere in Sight

The final lesson learned is that the BPM market (except for CEM) has abandoned the customer. What is meant by this is that those processes which represent what the customer experiences (the processes that are the customer experience) are not being dealt with by any of the commercial BPM market segments as end-to-end customer processes.

What we have learned is that these customer processes – the end-to-end processes that are what the customer experiences – are the relationship with the customer. It’s hard to believe that anyone would not consider these processes as the most important processes of the organization. Yet very few organizations have actually documented these end-to-end customer experience processes and even fewer have taken the steps to own them by shaping these processes to produce a known and desired experience for the customer that is specifically optimized to make the customer’s life simpler, easier and more successful.

Considering that these experiences are the business – from the customer’s point of view – and that these processes directly impact our ability to grow customer loyalty and expand the customer relationship it seems obvious that these processes would be first and foremost on our agenda for Business Process Management. That is not the case in the BPM commercial market with the exception of CEM and specific one-off consultancy practices.

BPM Practice Guidance

While this report exposes a number of misconceptions about the BPM market including a significant amount of industry hype that is being used to promote sales, it also recognizes the value of each segment of the BPM commercial market.

The use of Executable BPM products is appropriate for Application Integration and existing practices that include the concept of “job-tickets” and “to do lists.” It can also create value in some cases where an existing process is very costly due to poor management that has failed to successfully address a known and costly issue in the organization.

Non-executable BPM software can certainly be used to document existing processes and to help improve them just as professional services can using any of the techniques available in that market.

Some Executable BPM products are focused on improving repeatable processes that truly are business processes. While these products are likely to create real value when applied in this way they are still limited to a pre-defined way of thinking about the process and their value creation potential is tightly constrained.

Executable BPM products can play a value-add role in agility (this is the BPM-SOA connection technologists’ care about) if the right decisions are made as to how that systems-based agility will break down within the organization. Tying this systems-agility initiative to strategic plans of the business through explicit milestones is a very compelling way to move this technology opportunity forward successfully.

Considering the economic challenges faced on a world-wide basis entering into 2009 it is imperative that decisions on BPM at every interest level in the business be considered clearly and thoughtfully. Knowing what we are using a given commercial offering for, why it will create value and what it isn’t going to do for the organization are the necessary ingredients of successful BPM strategy. Employing a BPM strategy that does not include these ingredients is foolhardy in the extreme. And in an economic slowdown like we are experiencing now that kind of mistake can literally take a business “out of the game.”

About the Author:

Terry Schurter is a Director of the nonprofit International Process and Performance Institute, former CEO of Bennu Group, Research Director for Process with Bloor Research and CIO of the BPM Group. An internationally recognized thought leader in Business Process Management, Performance Management, and Customer Expectation Management he received the Global Thought Leadership Award in 2007 from the 20,000+ member BPM Group.

Terry has founded, co-founded and served as an executive officer for multiple technology startups, won international awards for Thought Leadership and Engineering Excellence (George Westinghouse Signature of Excellence Awards – 1990; 1991) and is an author – contributing author of several books including "Customer Expectation Management - Success without Exception".

Terry Schurter
Global Thought Leader
Director International Process and Performance Institute