Accurate and Affordable Data Cleansing in the Cloud

Rich, high quality, trusted data is the foundation of the digital enterprise. To help its customers succeed in this business landscape, SAP has brought its proven on-premise data quality expertise to the cloud with a new offering that helps companies of all sizes tackle quality issues for addresses and locations. SAP Data Quality Management, microservices for location data, is an application programming interface (API) that runs as a service in SAP Cloud Platform. It provides cloud-based microservices that deliver address parsing, standardization, validation, cleansing, and enrichment functionality for global address data. Covering more than 240 countries worldwide, it eliminates the need for organizations to license and maintain their own global address and geocoding data directories.

SAP Data Quality Management, microservices for location data is sold under pay-per-use licensing, meaning you pay only for what you actually use. This is big news for small businesses, who may not be able to afford the cost of on-premise data quality software and annual directory subscriptions, but who face the same data quality issues as large enterprises. Even the smallest business can afford this zero-cost-of-ownership solution.

But why use this service when SAP already offers address-cleansing functionality on premise, and many other vendors offer these capabilities in the cloud? Because SAP Data Quality Management, microservices for location data is pre-integrated with many SAP applications, including SAP Business Suite, SAP S/4HANA, SAP Master Data Governance, SAP Cloud for Customer, and SAP Data Services. This integration makes it nearly effortless to implement a data quality firewall to prevent bad addresses from getting into critical business systems and causing business problems down the road, such as high shipping and mailing charges, high return rates, duplicate mailings, late payments, and low customer satisfaction.

SAP Data Quality Management, microservices for location data, can also be used in custom applications, since the API is implemented as a Representational State Transfer (RESTful) web service accessible through a simple JavaScript Object Notation (JSON) interface. This means it can be used for any use case that requires high quality address data or geocoding, even to augment existing on-premise data quality solutions. For example, a large manufacturer that already owns SAP data quality solutions can easily integrate address cleansing in customer-facing portals without having to call through its firewall to cleanse the address data input by its users.

How Does It Work?

SAP Data Quality Management, microservices for location data, uses REST-based API calls to perform address validation and cleansing; geocoding (return a latitude and longitude given an input address); and reverse geocoding (return an address or a list of addresses given a latitude/longitude pair). The service is called from an application via an HTTP POST request containing the address to be processed.

As with any service, the HTTP POST request must be properly formatted and sent to the correct URL and endpoint with properly formatted authorization headers and the required properties in the HTTP request body. Let’s examine each of the elements required to embed the service in an application.

Obtain the URL for Calling the Microservices

When you subscribe to SAP Data Quality Management, microservices for location data, you will be given a unique URL that looks like this:

https://dqmmicro<#########>-<consumeraccount>.
<landscapehost>/dq/<endpoint>

The anatomy of the URL breaks down as follows:

• dqmmicro represents SAP Data Quality Management, microservices for location data, and will not change
• ######### is the provider account name, which identifies the host for the service (see product documentation for details).¹
• <consumeraccount> represents the SAP Cloud Platform account that enabled (subscribed to) the service
• <landscapehost> represents the location of the data center hosting your organization’s landscape and is unique to each landscape (see product documentation for details)
• /dq/ represents SAP Data Quality Management, microservices for location data, and will not change
• <endpoint> represents the specific microservice being accessed

Specify an Endpoint to Call a Microservice

To call a specific microservice, you must request it by specifying a specific endpoint in the URL. Five different endpoints are currently supported, each of which corresponds to different set of actions:

• /addressCleanse parses, standardizes, corrects, validates, and enhances address data. Address cleansing also automatically includes geocode processing when you request at least one geocode-related field.
• /addressCleanse/batch cleanses multiple addresses in one request
• /reverseGeo takes latitude and longitude coordinates and returns one or more nearby addresses
• /refData/country returns a list of country names and codes
• /refData/region returns a list of region names and codes

As more features are added (for instance, phone number validation), more endpoints may be added.

Use the Correct Authorization Header in the HTTP Request

When calling a microservice, you must include the following authorizations in the HTTP request:

• Content-Type: This header must have the value application/json
• Authorization: Both Open Authorization (OAuth) tokens and client certificates are supported in productive landscapes.² If you are using client certificates, authentication will be handled by your client during SSL negotiation. If you are using OAuth tokens, the Authorization header field must include the value Bearer followed by a space and then the provided token value. For example:

Authorization: Bearer 107d2995fc87b6a625faf5440dea2d

When the service matches the address in the request to the depth available in the reference data for the locale, the response contains the cleansed address with diagnostic information. However, if the service does not match the address to the depth available, it returns a list of suggestions from which the user selects the best fit. This suggestion list functionality is useful for interactive applications when the user may not have complete information to begin with, but can infer the correct address from a list of candidates. This feature is enabled by default with SAP Business Suite integration, so users can be sure they are putting only correct and accurate address data into their SAP application system.

Include the Required Properties in the HTTP Request

A wide range of input properties and structures can be applied to the microservices. These properties control elements such as input formats, requested return fields, parsing options, and script conversion (Latin into non-Latin and vice versa). All the necessary input, output, and formatting properties may be included with the request or controlled via custom configurations.

Simplify the HTTP Request with a Custom Configuration

Rather than forcing you to include all the parameters for the service call in the body of the HTTP request, SAP Data Quality Management, microservices for location data, offers a simple configuration tool that you can use to easily create flexible, reusable service configurations (see Figure 1). Once you have created a configuration using the Configuration Editor, you can replace all the parameters in the service call with a single parameter — configurationName — which makes calling the service simpler and reduces the complexity of the HTTP request.

For more details on the properties specified in an HTTP request for address cleansing — including an example of an HTTP request that specifies the input fields for the address data and the output properties to return as well as the response to the request — visit SAP.com/dqheader.

The area marked My Input Model in the Configuration Editor tool allows you to map the fields of your application data model to the input fields of the service, while the My Output Model maps the output fields of the service to your application data model. In the Countries editor, you can set up variant configurations that apply only to specific countries, and in the Settings area, you define the service settings that control how the service parses, processes, and returns data.

Once the configuration is complete, you need only to send the input address and configuration name in the body of the HTTP request. Simply replace all the lines of the request body in the “addressSettings” section with a single line:

“configurationName”: “myConfiguration”

For more details on custom configurations, visit SAP.com/dqconfig. 

To see an example that shows an HTTP request with all parameters included compared to a request that replaces parameters with a reference to a custom configuration, visit SAP.com/dqrequest.

Get Started

Taking a test drive with SAP Data Quality Management, microservices for location data, is easy; just sign up for a free trial account on SAP Cloud Platform (cloudplatform.sap.com), log in, and go to your account via SAP Cloud Platform cockpit. You can find the service in the “Master Data” category of SAP Cloud Platform services in Data Center EU1. Once you have found the service, you only need to click the “Enable” button to start your trial. The trial account is free, good for 90 days, and limited to 1000 free addresses per country. You can find a wealth of how-to information and videos on how to use the service within the SAP community — a good place to start is Ron Dupey’s blog and video series, available at https://bit.ly/2LFaEfn.

For details on how to get a trial account, visit SAP.com/dqgettingstarted.

Once you’ve tried the service, it’s just as easy to subscribe. You can purchase a subscription directly from SAP Store (at SAP.com/dqstorelink or by searching for “data quality” at www.sapstore.com). You pay only a small up-front fee and receive a monthly invoice for your actual usage. You can also work with your SAP account executive to purchase the service directly.

^{1 The product documentation is available at SAP Help Portal. Visit SAP.com/dqhelp to learn more [back]}

^{2 Note that the trial landscape supports basic authentication. See the product documentation at SAP Help Portal for more information (SAP.com/dqauth). [back]}