One of the requirements in the application integration scenario is to apply data quality rules on the incoming data so that the source data is normalized, corrected before integrating it with target systems.
Suppose we are getting Purchase Orders or Customer data and the address column may contain incorrect or partial street addresses. Is there a way we can call an external data quality API that can fix these data discrepancies and send back correct data? The answer is Yes! and let’s show you how.
In this example, I’m going to use Lob.com Address Verification API that checks and corrects bad street address information. You can also use other APIs to achieve the same process.
I created a wrapper API service in Adeptia which would be easy to call through the Data Mapper. Refer to other videos in the forum on how to publish your own API in Adeptia.
This wrapper service takes the data from the source (such as file, stream, application data etc) and calls the external Data Quality API and gets the results which I then send back to the data mapper.
I’m using Adeptia’s Data Mapper as the interface to invoke my wrapper API service since this the most logical interface where users can apply data quality and other business rules on the source data before converting it and sending it to the target application.
Within Data Mapper, there is a function called invokeREST() which allows us to pass any source data and call a REST API (i.e., my wrapper API in this example) and get a response back and map that response to a target field. This function is available in the Global Methods > Mapper Utility Class.
invokeREST() consists of the following input parameters. As an example my invokeREST consisted of following input parameters.
java:MapperUtilityClass.invokeREST(‘http://eval.adeptia.com:80/adeptia/wsapi/v1/verify?’,
‘POST’,”,’text/html’,’text/html’,$varValue,” ,” )
1. URL of the rest web service
The endpoint is pointing to an API that I have published within Adeptia that in turn calls the actual data quality API.
2. Operation Type such as POST, GET, PUT, DELETE
Since we are posting address for verification, the option is set to POST
3. Data to be sent as POST.
For this API, the data is sent as variables and not as a payload so it is left as blank.
4. Request Media type
This can be text/xml, text/html. text/json
5. Response Media type
This can be text/xml, text/html, text/json
6. The key value pair data from the source that needs to be sent to verify the address.
In this case, I embedded the address key-value pairs in a local variable and assigned it to this parameter. The local variable contains the following. API needs this data to check the validity of the address.
concat(
‘address=’,$Input_OrderFulfillmentExcelSchema/Root/Record/ShipAddr1, ‘,’ ,
‘city=’,$Input_OrderFulfillmentExcelSchema/Root/Record/ShipCity, ‘,’ ,
‘state=’,$Input_OrderFulfillmentExcelSchema/Root/Record/ShipState, ‘,’ ,
‘zip=’,$varZip
)
7. If API requires basic authentication then provide the username
8. If API requires basic authentication then provide the password
In terms of publishing your own wrapper API that calls the Lob Data Quality API, one would need to publish the web service within Adeptia. Here I created a data integration API that takes the incoming parameters from the invokeREST() function call and maps it to the JSON structure of the Lob Address Verification API. Refer to the video on how the process is created. Once the flow is designed, you can publish this service as a REST API by going to Develop > Services > Web Services > Provider (API) interface.
I have attached the export zip file of the wrapper API for your reference. Please use the Adeptia Migration Utility to deploy the process flow in your local Adeptia instance.
Also, refer to the below video on how to call an API as part to your data mapping rule.