Most companies can explain how an important process is supposed to work.
An order enters the ERP, an approval follows a defined route, inventory is updated, finance receives the transaction, and management eventually sees the result in a report. There may even be an SOP describing every step.
Then you sit beside the people who actually perform the work.
Someone exports a list to Excel before approving it. Another employee keeps a separate customer file because the system does not show something they need quickly enough. A manager receives an email before a transaction moves forward, even though that approval does not exist in the official workflow. Finance adjusts a figure in a spreadsheet before using it in the monthly meeting because the formal report does not reflect an operational exception correctly.
None of these activities may appear on the process diagram, yet without them the real process may not work as expected.
This is the world of shadow processes.
What Is a Shadow Process?
A shadow process is simply a part of the way work actually gets done that sits outside, beside, or sometimes around the official process.
The official process is what the company designed, documented and usually built into systems such as ERP, CRM or workflow applications. It represents the expected path: enter the information here, approve it there, follow these rules, then produce this result.
A shadow process appears when employees create another step that the official design does not contain. It might be an Excel tracker, a WhatsApp message, an email approval, a handwritten note, a local calculation, a second report, or simply the knowledge that “when this happens, ask Ahmad before continuing.”
The difference is therefore not necessarily between a correct process and an incorrect one. It is often the difference between the process as designed and the process as lived.
This distinction matters because management tends to have much better visibility into the first one.
Systems record transactions. Policies describe responsibilities. Reports show what reached the database. But part of the judgment, coordination and exception handling that makes the process work may be happening somewhere else entirely.
The result is that a company can have a beautifully documented process while still having an incomplete understanding of its own operations.
Why Do Shadow Processes Appear?
It is tempting to blame the user.
The company invested in a system, provided training and established a procedure, yet employees still created another spreadsheet. The natural conclusion is that people are resistant to change or unwilling to follow the process.
Sometimes that is true, but treating it as the default explanation can hide something much more useful.
People usually create workarounds because they are trying to accomplish something.
Imagine a credit controller who exports customer balances into Excel every morning. From an IT perspective, this may look unnecessary because the information already exists inside the ERP. But perhaps the ERP report takes too long to prepare, does not combine the balance with recent payment behavior, or cannot highlight the exceptions the employee needs to investigate first.
The spreadsheet is not the real problem.
It is evidence of a problem.
This is similar to the paths people create across grass when the official pavement takes an unnecessarily long route. Urban planners sometimes call them desire paths. The worn grass tells you something about where people actually need to go.
A shadow process can tell us the same thing about a business.
Perhaps the official process is too slow for a particular situation, perhaps an important exception was never designed into the system, perhaps information is divided between departments, or perhaps employees need a view of the business that the current application does not provide.
Of course, not every workaround is justified. Some exist because of poor habits, weak controls, personal interests or a desire to bypass rules. Understanding a shadow process does not mean approving it.
It means finding out why it exists before deciding what to do with it.
The Dangerous Response Is to Simply Eliminate It
When a new ERP or transformation project begins, there can be a strong desire to clean everything up.
Remove the spreadsheets. Stop the manual work. Force everyone into the standard workflow.
That sounds reasonable until the new system goes live and employees quietly recreate the same spreadsheets somewhere else.
The organization may then conclude that users resisted the new system, when the real issue is that the project removed the workaround without removing the reason people needed it.
If employees used a side calculation because the official system did not handle a certain commercial rule, replacing the system without understanding that calculation does not eliminate the business requirement. It simply makes the requirement invisible again.
The same applies to an informal approval. If a manager was being consulted because a certain transaction carries unusual risk, deleting that step from the new workflow does not remove the risk.
This is why shadow processes should be investigated before they are judged.
Some should absolutely disappear because they duplicate work or bypass necessary controls. Others should become part of the official process because they represent a legitimate requirement the original design missed. Some may remain as controlled exceptions, while others reveal a deeper issue in the way information, responsibilities or business rules are organized.
The objective is not to preserve every workaround.
The objective is to understand the business reason underneath it.
This Is Also Why “User Resistance” Can Be Misunderstood
People do resist change. Familiar tools are comfortable, new systems require learning, and some users naturally prefer the way they have always worked.
But resistance is not always irrational.
If a new system asks an employee to abandon a spreadsheet that solves a real problem without providing another way to solve that problem, resistance may actually be useful feedback.
Consider a sales manager who keeps a private customer-performance file even after the company launches a new dashboard. It would be easy to describe that behavior as poor adoption.
But before doing so, someone should compare the two.
Perhaps the spreadsheet includes returned goods that the dashboard ignores. Perhaps the manager groups customers differently from the official hierarchy because that reflects how accounts are actually managed. Perhaps it contains notes about customers whose ownership changed during the year. Or perhaps the dashboard is completely correct and the spreadsheet really is an outdated habit.
Those possibilities require very different responses.
Adoption should therefore not be measured only by whether people stopped using their old tools. A better question is whether the new environment actually supports the work those tools were helping them perform.
That distinction becomes especially important in BI.
BI Projects Are Particularly Vulnerable to Shadow Processes
An ERP project can at least see when users leave the system to perform part of a transaction elsewhere.
A BI project has a more subtle problem.
It can produce a dashboard that is technically perfect and still fail to represent the way the business understands itself.
Imagine that a company asks for a profitability dashboard. The BI team connects to the ERP, builds the data model, agrees on the accounting fields, calculates margin and publishes a clean dashboard.
Then the commercial manager says the numbers are wrong.
The BI team checks the database again and confirms that the calculations match the ERP exactly.
Both sides may be right.
The commercial team may have been using a separate calculation for years to account for rebates, special customer agreements, freight adjustments or another business condition that never became part of the official system. That logic may live in Excel, in a monthly manual adjustment, or simply in the experience of the person who prepares the management report.
If the BI project only asks, “Where is the data?” it may never discover this.
The more important question is, “How does the business currently arrive at the number it trusts?”
That does not mean reproducing every spreadsheet formula inside the Data Warehouse. Quite the opposite. It means understanding why that formula exists, deciding whether the logic is valid, and then creating one governed definition instead of allowing several versions of the same KPI to continue circulating.
This is one of the most important connections between shadow processes and BI.
Shadow processes often produce shadow data.
A local Excel workbook creates its own customer classification. Finance keeps another version of overdue debt. Sales calculates revenue one way while management reporting calculates it another. Departments create their own Power BI workspaces or reports because the central environment does not answer a particular question.
Eventually the company does not merely have several reports.
It has several versions of reality.
That is when a BI problem stops being a visualization problem and becomes a trust problem.
A Dashboard Can Be Correct and Still Be Incomplete
One of the most dangerous outcomes in BI is the technically correct dashboard that nobody trusts.
The data loaded successfully, the calculations are valid, the refresh runs every morning and the charts display exactly what was requested, yet during the management meeting someone still opens another spreadsheet to explain what is “really happening.”
At that point, forcing users to close Excel will not repair the dashboard.
The BI team needs to understand what information or business logic exists in that second view that the official analytical model does not capture.
Sometimes the answer will expose bad practice that should be removed. Sometimes it will expose a valid business rule. Frequently it will reveal context that transactional systems were never designed to store.
This is why a strong BI project should not begin and end with databases and requirements documents. It needs enough contact with the actual work to understand where the numbers are interpreted, adjusted, questioned and combined before someone is willing to make a decision from them.
The purpose is not to document every mouse click an employee makes.
It is to discover the pieces of business meaning that would otherwise remain outside the analytical model.
AI Makes This Gap More Important, Not Less
The same problem becomes more serious when AI is introduced.
AI can search, summarize, compare and reason over information remarkably well, but it cannot reliably use business knowledge that the organization itself has never made visible to it.
If the official process says one thing while experienced employees quietly follow another rule, AI sees only the version it has been given.
If the collections policy exists in a document but the actual exception logic lives in the head of a senior employee, the model does not automatically inherit that experience.
If three departments calculate the same KPI differently, adding AI does not magically decide which definition represents the business.
AI does not create operational clarity. It consumes it.
That is why exposing important shadow processes before an AI project matters. Otherwise the company risks teaching AI the formal version of the business while employees continue operating according to another one.
The result can look intelligent while being disconnected from reality.
Do Not Hunt Shadow Processes. Learn From Them.
The phrase “shadow process” can make these activities sound like something an organization should eliminate on sight.
I think that is the wrong instinct.
A shadow process should first be treated as a clue.
Why did people create it? What problem does it solve? What information does it contain that the official environment does not? What happens if it disappears tomorrow? Does it protect the business from a real exception, or does it create unnecessary risk? And if ten people independently invented similar workarounds, what does that tell us about the process they were given?
Only after answering those questions should the company decide whether to remove, redesign, formalize or automate it.
That approach is especially valuable in BI because every shadow report, side calculation or unofficial KPI potentially tells us something about the gap between the data the company stores and the information people actually need to run the business.
A BI project does not fail because somebody still has an Excel file.
It fails when nobody bothers to ask why that Excel file still matters.
The goal should not be a company with no shadow processes at all. Businesses change too quickly, exceptions will always exist, and employees will continue adapting when systems do not match reality perfectly.
The goal is to avoid having important parts of the business remain invisible.
Because once those invisible processes influence the numbers, the decisions, or the way people interpret performance, they are no longer a side issue.
They are part of the business, whether the official process map admits it or not.
