Customer data onboarding is a difficult process and remains one of the most challenging problems that businesses face. Experienced IT managers consider it to be one of the most difficult challenges for their teams to handle and ensure it is done correctly, efficiently, and in a timely manner. Data onboarding engineers often find these tasks frustrating with unpredictable failures and feel they are just lurching from one daily crisis to another. Many Adeptia clients tell us that their engineers who work on these detailed data integration tasks find this work so thankless that they either prefer to quit or want to be reassigned to other areas.
From our experience of working on this problem with dozens of organizations ranging from $20M USD annual revenue startup to $150B Fortune 50 companies, these are the 10 common factors that make customer data onboarding an especially challenging problem to solve:
1. Sheer number of customer integrations to manage
The number of customer integrations for a company depends on the size of its business ecosystem and the type of data transactions between the company and its external entities.
These external organizations are typically categorized into different networks such as customers, partners, vendors and suppliers with whom the company exchanges business data electronically. The number of external organizations in a company’s business ecosystem typically depends on the size of the company and nature of its business activities. From Adeptia’s experience, our customers’ business ecosystems have ranged from scores of external entities to tens of thousands.
The second element impacting number of customer integrations is the type of business data and number of transactions involved in data exchange with each external organization. It takes a lot of effort to initially implement all these data flows and then it is a daily challenge to monitor, track and manage these data exchanges to ensure everything is running correctly.
2. Lack of control with what end customers can do
A company has limited ability to influence or mandate data standards and protocols in terms of setting up automated, recurring data flows with its business ecosystem made up of customers, partners, vendors, and suppliers. This means the company has to be flexible in handling a multitude of different channels, formats and protocols for ensuring streamlined data exchange. Ideally, a standards-based approach is highly scalable and easier to implement but that is not an option in many cases.
Usually, it is a mix of standards and non-standards-based approach often with standards-based data exchanges with suppliers and vendors while using non-standard approaches with customers and partners.
3. Lack of control of time
A factor that complicates customer data onboarding for a company is the lack of ability to control timelines with its external customers. Some end customers may be in a rush, and that requires the company’s IT staff to rearrange priorities to setup and configure new data integrations urgently with those customers. In other cases, customers may be slow or non-responsive, thus holding up precious IT engineering resources who are waiting for information. In other cases, there may be errors and exceptions that occur during normal daily operations and the external customer’s side is slow in fixing those errors.
4. Lack of control of data quality
As customer data onboarding relates to exchanging business data with external entities, there is limited ability to control what happens on the other side. This, as a result, impacts the quality of business data that is being received by the company negatively. Of course, poor quality of business data can have a huge negative impact on any business such as servicing wrong orders, shipping wrong products, not charging the correct amount, manufacturing wrong quantities, ordering wrong parts, and so on. So, maintaining business data quality is a paramount concern of any IT department (second only to data security) and this means that any data that is received from outside has to be considered as “suspect”. Such data needs thorough data integrity and validation checks before it can be processed further into internal systems. This requires a lot of extra work that is usually not needed for internal data and application integration scenarios.
5. Pressure from business delivery team to do it fast
In any company, the sales and marketing teams work very hard to generate business and win deals by acquiring new customers. When a new customer is acquired, there is a desire to quickly onboard that customer and start providing services to start generating revenue. Especially, in the case of companies that provide data-driven services, it is imperative that customer data be onboarded quickly, as opposed to the multi-week (in many cases multi-month) duration many companies consume. In these cases, the IT team tasked with setting up the automated, recurring data flows with new customers face pressure from business teams to get the data integrations completed quickly — in just hours or a few calendar days.
6. Lack of IT resources and expertise
Customer data onboarding as implemented these days is an engineering-intensive task and it requires deep knowledge of data integration technology along with business process knowledge related to the business information and data fields. As mentioned before, large business ecosystems require multiple customer onboarding flows to be implemented at any point of time in parallel as well as many other existing data flows may require changes and error- correction. This is a time-consuming and high effort work and many IT teams are just not staffed with enough resources to handle it. Further, in our experience of working with many companies, the level of expertise required is significantly high and there may not be enough available engineers with required skill sets.
7. Lack of right tools
Many companies continue to use typical data and application integration tools to address their needs for customer data onboarding. These are developer tools and they are designed for creating and managing internal integrations between a handful of applications or aggregating data for analytics. These tools are not designed to implement and manage thousands of data exchanges with external entities. These tools usually require code development environments and are not web-browser based. These tools are often code-centric solutions that do not allow easy re-use of existing objects and data maps. They also don’t allow collaboration with external users. These tools also lack the visibility needed to monitor data flows and troubleshoot errors. Lack of right tools makes it very difficult for IT staff to properly handle the challenges related to customer data onboarding.
8. Variances with different customers
Due to differences in technical IT capabilities on the side of the external customers and partners, there is usually a wide variety of channels, formats and protocols that are used to set up data exchanges with these organizations. This makes setup of data flows with different external entities typically one-off affairs, thus reducing the possibility of reusing data mappings, artifacts and objects from one customer to another. This increases the effort related to configuring new data flows.
9. Changes over time
A major challenge with customer data onboarding is that the problem of scale of exchanging data with a large number of customers and partners in the business ecosystem is compounded by the frequent need for changes to be made to the configured data flows over time. These data exchanges are not static as the business requirements may change for new data fields to be included, new data validations to be added, and other technical changes such as adding new transactions, changing expired certificates and changes to FTP servers and API credentials. This requires a lot of extra time and effort in maintaining existing data flows.
10. Difficulty with Exception Handling
Finally, a big problem associated with customer data onboarding is the difficulty associated with tracking errors, troubleshooting unexpected issues, and identifying fixes related to day-to-day problems in the running of data flows. There are too many moving parts to easily identify the source of problems because the issues may occur on the external side or in-transit or on the internal company side. And if the problem is external, then the resolution demands actions by people who are not in your control- making the resolution timeline unreliable. Exception handling makes customer data onboarding challenging as issues and errors happen every day.
Summary
These ten factors make customer data onboarding and the related intercompany data exchanges a complex problem for both technical IT and business teams to solve. However, this problem is faced by all companies and it can be turned into a competitive advantage by the ones that can handle it in a better and more efficient manner. These companies can use their strength and capabilities to make their customer data onboarding faster, facilitating better service delivery and delightful customer experience.
Adeptia Connect is a business application that has been designed to address the ten factors cited above and, in doing so, enabled customer integration onboarding up to 80% faster. Business user self-service also provides companies a compelling customer experience. Adeptia scales to manage a wide range of business ecosystems spanning up to tens of thousands of organizations with precision, making them easier to do business with.