Data Cleansing is a must for decision accuracy & savings
Today, there is an increasing noise within the logistical data we collect, and the overall data process itself. This leads someone to wonder about the accuracy of the supply chain decisions made when it comes to optimizing supply chains, selecting new DC and warehouse locations, tendering out warehouse or transport, and optimizing their transport and fleet operations.
The starting point for doing effective supply chain risk management (SCRM) or just supply chain management (SCM) begins with the data quality of your supply chain operations. To quickly test the confidence level of how accurate or not decisions you make, you can address the following six points:
I know exactly the source of the data feeding these KPIs.
I know how data is being processed, analyted, manipulated, cleansed etc
The data being analyzed via an automated system (no manual interventions).
And I understand the analysis and calculations behind it (ideally there is a process map documented, explaining the processing steps).
The data source is a system (WMS, TMS, ERP, etc.) and not a spreadsheet, paper form, etc.
Data are captured automatically i.e. no manual input on the system.
How you reply to the six points, will show how confident or not you are with the data your decisions are based upon.
To make a point, and with very simplistic reasoning, having 5% of errors in your data feeding your reports, KPIs, or optimization efforts, leads to a distortion of +/- 5% of the results. Assuming there are additional biases and assumptions made, and process inefficiencies on how data are treated, our decisions can be +/-20% off without even realizing it. Knowing where the data are coming from, how we input them to our systems, or how we manipulate them reveals any gaps or biases inflicted in the data.
Data cleansing is not just about correcting typos and other types of errors. After many years of experience, and after cleansing numerous data sets from many well-known companies, …
The three main stages someone should follow for proper data management are:
There is the false impression that by using expensive and well-known ERP systems or having digitized many parts of the supply chain, the data quality is great. Let’s compromise on that data quality will be better than before, yet not as great as you might think. Based on experience the data errors or lines that need to be excluded from a data set, vary between 3%-15% on average. Often there are cases of data sets with >38% of errors, making any data analysis or KPIs irrelevant.
According to Wikipedia “Data cleansing or data cleaning is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.”
Data cleansing is not just about correcting typos and other types of errors. After many years of experience, and after cleansing numerous data sets from many well-known companies, I concluded that data cleansing is one of the most key tasks, in delivering a successful project or making a risk-free decision. And as expected, the main goal in all of these projects was to identify areas for savings and performance improvement. Yet, in more than 90% of the projects, the data cleansing process usually points to the right direction of where to unlock hidden savings.
The main reasons why data cleansing is so important are
- You build confidence in the data set you to work on
- Make a critical evaluation of the seriousness of the errors identified
- Gain a better understanding of the operation & supply chain you investigate
- Identify what is causing the errors
- Decide on whether to include/exclude the errors from the overall study
- Gain an understanding of the % accuracy of your study, KPIs, reports, etc.
Data quality is also another key supply chain risk most frequently neglected and overlooked. The dominant understanding of SCRM is a VUCA business environment, with volatility, uncertainty, and fluctuations in the form of external information and data, radically shifting our models, forecasts, and plans. However, there is no excuse to add more risk to data and information under our own control.
Our data and data processes are 100% under internal control, and we can at least mitigate this supply chain risk. Of course, a supply chain comprises of multiple parties, suppliers, subcontractors, carriers, plants, customers, etc. However, mapping your visible supply chain with the data sources used, it’s a starting point to understanding from where these errors or the variability originates. Are the errors internal or external, from a supplier or the general environment? Which of these can we affect, measure, improve, and control?
Data cleansing is important, exactly because you remove errors and any other related noise that could skew the results and lead you to wrong conclusions. In almost every case, it is better to work on 95% of the data and be certain for 100% of the results, rather than the other way around. This is also the reason why you can identify more cost savings, using clean data.
Use case 1: in transportation planning & routing of vehicles, if you optimize the right set of data, ignoring the errors then the optimization algorithms will provide you with a 5% more optimum result, hence reducing your costs. A more optimum fleet, with fewer driven kilometers, means less CO2 emissions.
Use case 2: Using supply chain optimization software for modeling a company’s network or calculating multiple scenarios having a 5%-10% better accuracy, could lead to a different solution and decision.
Use case 3: Companies when issuing RFQs and tenders, share their data with multiple carriers. However, each carrier is cleansing the data received differently, making different assumptions, leading to proposals with a variance of 5%-15%. Hence, the proposals are not “like-for-like” comparable.
Standardizing, and cleansing data in a supply chain network, maybe a bit more challenging given that some carriers or 3PLs will have external data, with a different structure, and format, but complimentary to your data.
For this reason, we have developed GINUS, a B2B SaaS platform with an automated data-cleansing tool that tackles such complexities, and which can support you in:
Cleansing your data – AI supported
Multi-source data integration
Geocoding / Reverse Geocoding
Our platform can be used for passive data integration with data which can boost your supply chain visibility in a very simple, fast, and cost-efficient way.
In SCRM CENTRE we strongly believe in data quality, as the basis for making supply chains more cost-efficient, and more environmentally sustainable! It is also the first major step for supply chain risk management (SCRM).
Contact us to find out more about how we can support you to make better & greener decisions!
Dr. Leonidas Fotiou has more than 17 years of industry experience in various roles, and for well-known companies such as DHL Supply Chain, and Johnson & Johnson. He has lived in Greece, the United Kingdom, Switzerland, and Germany, and he has worked on more than 80 projects across Europe, the Middle East, and Africa. He holds a BSc in Total Quality Management, an MSc in Logistics Engineering & Logistics, and a Ph.D. in Supply Chain Risk Management.
Find Supply Chain Innovations through our Startup Database
Stay tuned with The Supply Chain Management Startups Handbook 2022
The Supply Chain Management Startup Handbook 2022 will be a comprehensive guide through the landscape of supply chain startups and will help to discover cutting edge technology and business model innovations.