Data migration projects are relevant for all companies that need to move data from one application to another. This includes companies that are: 

• Implementing a new SAP CRM system while replacing a legacy one
• Acquiring or merging with other companies
• Expanding into new lines of business
• Initiating green IT projects to reduce hardware and system environmental and dollar costs
• Harmonizing data and processes across organizational boundaries
• Conducting a global CRM rollout
• Performing a major upgrade of an SAP CRM system, replacing and integrating non-SAP source applications

Planning for a data migration project is a major task. Migration projects are risky. If not done correctly, they can cause a myriad of issues, including delayed production go-live, poor data quality that prevents core business processing, and poor user adoption of the new or upgraded SAP CRM system. Cleansing, transforming, and loading data into a new SAP CRM application can seem like a daunting task.

Add to that task the following common questions:

• How much of my data and related content do I take?
• Is my data ready to be migrated?
• What do I do with the legacy application after the migration?
• How long do I need to keep the data in the new application?
• How do I handle contracts and documents related to my data?

We’ll focus on the data migration process from the connection to the legacy system until the final data load to an SAP CRM system. For this purpose, we use the SAP BusinessObjects Data Services platform, which is part of SAP’s solutions for enterprise information management (EIM). Thanks to the pre-delivered content of the SAP Rapid Deployment solutions (RDS) and SAP Best Practices packages, the software product provides an out-of-the box solution for rapid data migration to an SAP CRM system. The RDS data migration package helps you migrate legacy data — such as supporting leads, opportunities, campaign management, interaction center services, and sales management — to SAP CRM. SAP RDS packages come with data migration tools and new SAP CRM packaged content.

Understanding SAP BusinessObjects Data Migration

SAP BusinessObjects Data Migration is a solution based on rapid deployment solutions, specifically rapid data migration targeted to specific SAP applications, such as SAP CRM. SAP’s data migration solution aims to reduce project risks and speed implementations. It has three major components:

• Software driven by SAP BusinessObjects Data Services (hereafter Data Services)
• Migration content developed by SAP’s RDS organization
• Consulting services provided by SAP Services and partners

The combination of the software, migration content, and optional consulting services provides a complete solution for migration to SAP CRM applications.

Data Services is a product that contains two parts: Data Integrator for data integration and Data Quality Management for data quality tasks. The piece for data integration is an extraction, transformation, and loading (ETL) tool for ETL aspects from one or many sources to one or many target systems. With these capabilities, including the data quality pieces, it is best suited for migration between non-SAP applications to SAP CRM.

As it is sitting on top of the SAP BusinessObjects BI platform, it also enables BI reporting and contains dashboard analysis. The migration content includes some SAP BusinessObjects Web Intelligence reports to get a better picture of the entire data migration process and its progress also for business users.

SAP BusinessObjects Data Migration software is used across the following steps:

1. Profile and extract data from the source system 
2. Map the data to the target data structures
3. Validate the data against the target business context (i.e., the business rules of the target system, such as if there are specific fields required or if the country codes are correct)
4. Load the data into the target CRM system
5. Reconcile the data between the target CRM system and the source system
6. Repeat the entire process iteratively until the data is ready for loading into the production target system

Let’s take a look at the architecture of data migration in Figure 1. We’ve called out six specific areas, which we discuss in more detail.

1. Source and target systems: First look at the target environment and legacy data environment. These show the source and target applications. When using the Rapid Data Migration for SAP CRM content, the environment is always an SAP CRM application, typically including the RDS for CRM package. The legacy environment can be any non-SAP environment, from connectivity to databases, applications (e.g., Siebel, Salesforce.com, or legacy applications), flat files, or XML.
2. Extract and profile: The staging area is provided using Data Services. In this staging area, you extract and profile data from the source systems. The profiling of data is a critical step as it provides insight into the state of the existing source systems. An example of important details you can check for is patterns across data. For example, in the U.S., ZIP codes are five digits plus an optional four-digit code. A data assessment can determine how many unique ZIP codes you have and how often the ZIP+4 extension is used across your source systems. Another example is to know the pattern of country designations. For example, for Germany the terms Germany, DE, or Deutschland could be used in the source system.
3. Cleanse, transform, and validate: This includes updating the data so that it meets specific patterns, transforming the data according to rules, and validating data against the SAP business context. This can involve combining two fields into one, splitting fields, updating the data within a field to match certain rules (for example, telephone number formats), and validating data against required fields and lookup values from the SAP business context and configuration. 
4. Extraction of SAP configuration: As part of an SAP implementation, SAP is configured with many values such as plants, material types and groups, and sales territories. Mapping of the source data normally requires mapping fields that comply with the SAP configuration. The extraction of SAP configuration data takes the settings in the SAP system so that the source data can conform to the required format in the target system.
5. Reconciliation: Reconciliation looks at what was actually loaded versus what was expected to be loaded.
6. Reporting and dashboards: Throughout the process, dashboards are available for people involved with the project to know the status of the migration. Additionally, the migration starts the process of data quality expectations and governance around data management.

The following is a list of all software components for data migration included in the RDS package:  

• Data Services: Data management platform to support all migration tasks
• SAP BusinessObjects Information Steward: Data profiling and assessment as well as data lineage and reconciliation from target to source
• SAP BusinessObjects BI platform: Includes SAP Crystal Dashboard Design (formerly known as Xcelsius Dashboards), SAP BusinessObjects Web Intelligence, and SAP Crystal Reports

The following is a list of content for data migration included in the RDS package:

• Migration content (available from https://service.sap.com/bp-datamigration) loaded into Data Services for mapping (field mapping) and validation
• SAP BusinessObjects Web Intelligence reports for monitoring the data migration
• Migration Services on the SAP BusinessObjects platform to map source data field values to SAP business context (value mapping)
• Migration content for reconciling what was loaded in the target to the source

As noted in Figure 1, the leading software for SAP BusinessObjects Data Migration is Data Services. Data Services is a well-known and scalable data integration platform for ETL. Some of the key features of Data Services include:

• A single interface to design data structures and build transformation rules
• A visualization of source-to-target metadata to analyze data lineage and impact
• Profile and analyze any source data 
• Database extraction to easily extract legacy systems
• Easy mapping (reusable)
• Reusable and sharable transforms and functions (drag and drop) to minimize own programming
• Web-based administration to manage job operations
• Operational dashboards to monitor job execution and trends

Data Services can connect to any source or target system via the adapter framework. While it does have native connectivity to SAP systems, it also has connections to files, ODBC, mainframe, XML, Excel, and many other connectivity options. Architecturally, Data Services resides in the orchestration and integration layer of your existing architecture.  

From a business perspective, you can reuse Data Services after the data migration project for ongoing integration (e.g., master data integration with multiple systems). You can also use it for ongoing data quality, including an on-entry quality check within the SAP CRM application. Additionally, it can help you drive data governance projects, especially as they relate to cleansing, quality, and the synthesis or removal of duplicate data.

Deep Dive into the RDS Content for Data Migration

SAP Rapid Data Migration provides ready-to-use content. The content becomes a key differentiation point compared to any other data migration approach because it accelerates data migration projects, and can save up to 30 percent of implementation time.

As part of our deep dive, let’s look into the predefined migration content before moving on to some documentation.

Predefined Migration RDS Content in Data Services

Data validation content enables the mapping of customer data to SAP data, including validation of key values. This content is specific for SAP CRM, including migration content specific to sales, service, and marketing. 

Documentation

Some examples of documentation include:

• A quick guide on how to install the content
• A process guide called “Business Process Document for Data Migration” that provides a process flow and the sequence in which the objects need to be loaded, including specific instructions
• An extension guide that provides the information you need to extend and modify the content according to your specific needs

The scope of objects provided is quite extensive. Figure 2 shows a list of objects available for SAP CRM.

Figure 2

Structural hierarchy of accrual object

Step-by-Step Demo on Using the Migration Content in Data Services

Now that you have an understanding about the solution, the software, and the content, the following is a step-by-step example of how to get started with SAP BusinessObjects Data Migration. Normally companies do not embark on a data migration project alone, especially the first data migration project to the SAP system. Use these steps to get started with the software and the content.

Step 1. Go to https://service.sap.com/bp-datamigration to check for an overview and https://service.sap.com/rds-dm4crm to download the package (both require login credentials). Optionally, you can review the overview presentation and watch the demo. Also you can download the DVD content in the Software Download Center even if you do not own the software.

Step 2. Once you complete the download, you can start to explore the content, accessing the getting-started guides and the documentation to get a complete picture about the content and the deliverables of the data migration package.

Step 3. At this point, even without the software installed, there are still key guides and places to start planning for the migration. The migration content has Word documents and Excel templates for each of the objects in the SAP system (e.g., a document on business partner contact data, and another on quotations). You can read the document to understand the object and use the Excel spreadsheet to start the research on how the source data maps to the target.

Step 4. When you do decide to install the software, you can get a temporary key from https://service.sap.com/licensekey. However, if you already have Data Services installed, you can use the guides to install the content manually. In addition, the RDS package contains a services component that helps you to deploy the entire package including the software and getting started with the first business objects within weeks.

Step 5 (optional). If you are new to Data Services, you might want to start by going through the Data Services tutorial that is included in the documentation when you install the product.

Step 6. Once everything is installed, you can see a project called AIO_BPFDM_IDOC_CRM that exists in Data Services. This project has jobs for each of the objects, including Job_DM_CRMBusinessPartner_IDoc for the CRM Business Partner data, which we use as an example now (Figure 3).

Figure 3

Accrual methods defined in SAP ECC for SAP CRM-IPM

Step 7. Drill into the CRM Business Partner job and you can see the mapping and validation that takes place (Figure 4). The validation is against required lookup fields in the SAP system, validating mandatory fields as well as the format (e.g., if the phone numbers all include country codes and if the U.S. ZIP code is in 5+4 format).

Figure 4

Assigning accrual methods with the IPM component

The mapping within the job, as shown in Figure 5, has the source fields on the left and the target fields on the right. This example shows the mapping of the country from the source to the target, which is done using drag and drop.

Figure 5

Accrual types defined in SAP ECC for the IPM component

Step 8. The migration content includes Word documents explaining each IDoc, as well as spreadsheets for mapping the legacy Business Partner data to the SAP IDoc structure. Use the Excel template to start the mapping research of the source to the target prior to the mapping in Data Services (Figure 6).

Figure 6

Defining derivation rules in SAP ECC for account determination

Step 9. The other component that is important during the migration is the Migration Services tool. This tool, which is delivered by SAP Best Practices, reads the IMG configuration data from the SAP system and enables mapping of source values to SAP configuration values. In this example, United State, United States, and US in the source system should map to US in the target system (Figure 7).

Figure 7

Derivation rule details

Using the steps above, you can get started using the migration content and Data Services for your data migration project. For a complete demo of data migration, go to <insert Link to Demo on the publishing house web page>.

Reporting, Visualization, and Governance with Data Migration Projects

In addition to the jobs, workflows, and mappings provided in Data Services, reports and visualizations keep you updated on the status of the data migration. The data migration project can lay the groundwork for data governance, data ownership, and ongoing data quality. Governance of data and information is a topic attracting increased interest, and data migration falls into the governance category, providing the first opportunity to implement governance processes around the management and data quality expectations. Now let’s look at some examples of the reports provided with the migration content.

Figure 8 shows example reports accessible via the Web-based SAP BusinessObjects InfoView that display the status of the migration objects.

Figure 8

Defining source and target G/L accounts for account determination

Figure 9 shows the number of valid and invalid records for addresses via the Business Partner Addresses folder in Figure 8.

Figure 9

Relationship between an accrual document and an accounting document (initial sales posting document

Frank Densborn

Frank Densborn is a product manager for packaged solutions at SAP. He is working in the technology area focusing on EIM, data migration, and cloud integration – specializing in SAP S/4HANA migration. He joined SAP in 2004 and has held various roles in development, education, support, and product management. You can view his profile at https://www.linkedin.com/in/frankdensborn. 

You may contact the author at frank.densborn@sap.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.

Ginger Gatling

Ginger Gatling works in solution management covering information management topics, focused on data migration. Prior to this, she covered business process management topics (including SAP Business Workflow, Universal Worklist, and SAP NetWeaver Business Process Management), integration topics (SAP NetWeaver Process Integration), and other SAP NetWeaver topics. She coauthored the second edition of Practical Workflow for SAP and has delivered a series of workshops on workflow for SAPinsider.

You may contact the author at ginger.gatling@sap.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.