Migration projects require iteration of data loads to better understand data, as well as close collaboration between business and IT. Beyond that, migration projects also need to be driven by software with functionality that allows you to convert legacy data into the appropriate format of your SAP solution. Discover what you need to know to leverage SAP BusinessObjects Data Migration for your data migration projects. Learn how to get started with its functionality, as well as the standard content that comes with it and the reports and governance it provides.
Key Concept
SAP BusinessObjects Data Migration allows you to better perform data migration projects. The solution is applicable for migration to any application. However, with the later release, key features have been introduced to specialize in migration from legacy data to SAP applications. SAP BusinessObjects Data Migration is primarily comprised of SAP BusinessObjects Data Services, but also includes SAP Best Practices content. This content helps you save time and effort by providing standard content that can map source-to-target data structures, as well as reports and dashboards that monitor the data migration jobs.
Migration projects are relevant for all types of companies, including those with SAP applications, non-SAP applications, or those who are upgrading — any company that needs to migrate data from one application to another. This includes companies that are:
- Acquiring or merging with other companies
- Expanding into new lines of business
- Initiating green IT projects to reduce hardware and systems environmental and dollar costs
- Harmonizing data and processes across organizational boundaries
- Conducting a global SAP rollout
Migration projects are risky, however, because if they are not done correctly they can cause a myriad of issues, including delayed production go-live, poor data quality that prevents core business processing, and poor user adoption of the new application system. The issues and concerns with data migration are well known, with research done by Gartner, Bloor Research, and others. Despite this research, migration projects can still be undervalued, underscoped, and poorly thought-out in terms of how the source data can ensure successful business processing in the new application.
Companies often migrate from multiple source applications to a central SAP ERP or other SAP Business Suite application, which is risky. SAP understands the critical nature of the data migration project, both for the success of the application implementation as well as for ensuring the data is cleansed, prepared, and sufficient to drive and support key business processes. SAP provides a solution for data migration that can be used for migration to SAP target applications, as well as non-SAP applications.
We’ll focus on the solution for SAP target applications, but will also discuss how you can use SAP BusinessObjects Data Migration for non-SAP migration as well.
Understanding SAP BusinessObjects Data Migration
SAP BusinessObjects Data Migration aims to reduce project risks and speed implementations. It has three major components:
- Software components, driven by SAP BusinessObjects Data Services
- Migration content, developed by SAP’s Best Practices organization
- Consulting services, provided by SAP Services and partners
The combination of the software, migration content, and consulting services provides a complete solution for migration to SAP applications.
The following are examples of when this solution should be used:
- Implementing a new SAP application system, such as SAP ERP or SAP Customer Relationship Management (SAP CRM)
- Example: Project to implement SAP ERP Central Component (SAP ECC) 6.0 or SAP CRM 7.0
- Adding a new line of business that requires migrating data to an SAP system
- Example: Adding new plants to an existing system or adding a new country to SAP ERP Financials
- Implementing a new application within the existing SAP ERP system
- Example: Implementing SAP CRM and Billing for Utilities onto SAP ERP
- Example: Implementing materials management on the existing SAP ERP system
- Reducing the number of SAP systems as part of total cost of ownership (TCO) or other green IT project
- Example: Moving from 20 to four production systems globally, with data being migrated to the four systems
- Decommissioning non-SAP systems as part of TCO or other green IT project
- Example: After implementing a new SAP application, the previous system can be decommissioned
SAP BusinessObjects Data Migration is best suited for migration between non-SAP applications and migration to SAP Business Suite. There are also times when the migration could be from one SAP system to another SAP system. The following are examples of typical SAP-to-SAP migrations:
- You need to carve out a company code and all related data from one SAP system and move it to another SAP system
- You need to convert data directly in the productive database (for example, currency conversion, chart of accounts, and legal compliance issues) while preserving the business context
- You are merging with other companies that have existing SAP systems and you need to merge multiple SAP systems
Note
When performing SAP-to-SAP migrations, the leading solution is SAP Landscape Transformation (SAP LT). SAP LT is focused on migrations and transformations within SAP landscapes. You can get more details at
https://service.sap.com/slo. There are specific use cases when both SAP LT and SAP BusinessObjects Data Migration are used. One example is when migrating companies from one SAP release to another SAP release and you need to have all the company names checked and addresses cleansed. For this, you should use SAP BusinessObjects Data Migration.
SAP BusinessObjects Data Migration software is used across the following steps:
- Data is profiled and extracted from the source system
- Data is mapped to the target data structures
- Data is validated against the target business context (i.e., the business rules of the target system, such as if there are plants required or if the country codes are correct)
- Data is loaded into the target system
- Data is reconciled between the target system and the source system
- The entire process is done iteratively until the data is ready for load into the production target system
Note
Steps 2, 3, and 5 — mapping, validating, and reconciling data — include content provided by SAP Best Practices.
Let’s take a look at the architecture of data migration in Figure 1. We’ve called out these six specific areas, which we’ll now discuss in more detail.
Figure 1
Architecture of data migration using SAP software and SAP Best Practices components
- Source and target systems: First look at the target environment and legacy data environment. These show the source and target applications. When using the SAP Best Practices content for data migration, the content environment is always an SAP application, normally SAP ERP or SAP CRM. The legacy environment can be any non-SAP environment, from connectivity to databases, applications (e.g., JD Edwards, Salesforce, or legacy applications), flat files, or XML.
- Extract and profile: The staging area is provided using SAP BusinessObjects Data Services. In this staging area, you extract and profile data from the source systems. The profiling of data is a critical step as it provides insight into the state of the existing source systems. Examples of important details you can check for are patterns across data. For example, in the US ZIP codes are five digits plus an optional four-digit code. A data assessment can determine how many unique ZIP codes you have and how often the ZIP+4 extension is used across your source systems. Another example is to know the pattern of country designations. For example, for Germany the terms Germany, DE, or Deutschland could be used in the source system.
- Cleanse, transform, and validate: This includes updating the data so that it meets specific patterns, transforming the data according to rules, and validating data against the SAP business context. This can involve combining two fields into one, splitting fields, updating the data within a field to match certain rules (for example, telephone number formats), and validating data against required fields and lookup values from the SAP business context and configuration.
- Extraction of SAP configuration: As part of an SAP implementation, SAP is configured with many values such as plants, material types and groups, and sales territories. Mapping of the source data normally requires mapping fields that comply with the SAP configuration. The extraction of SAP configuration data takes the settings in the SAP system so that the source data can conform to the required format in the target system.
- Reconciliation: Reconciliation looks at what was actually loaded versus what was expected to be loaded.
- Reporting and dashboards: Throughout the process, dashboards are available for people involved with the project to know the status of the migration. Additionally, the migration starts the process of data quality expectations and governance around data management.
The following is a list of all software components used for data migration that come as part of the SAP BusinessObjects Data Migration bundled package:
- SAP BusinessObjects Data Services: Data management platform to support all migration tasks
- SAP BusinessObjects Data Insight: Data profiling and assessment
- SAP BusinessObjects Metadata Manager: Data lineage and reconciliation of data from target to source
- SAP BusinessObjects Enterprise: Includes SAP Crystal Dashboard Design (formerly known as Xcelsius), SAP BusinessObjects Web Intelligence, and SAP Crystal Reports
Note
A new capability called SAP BusinessObjects Information Steward will be generally available in 2011. Once it is released, it will replace the Data Insight and Metadata Manager components. By the end of 2011, SAP BusinessObjects Data Migration will be available on the 4.0 release of the software.
The following is a list of software components using SAP Best Practices content:
- Migration content (available from https://service.sap.com/bp-datamigration) loaded into SAP BusinessObjects Data Services
- SAP BusinessObjects Web Intelligence reports for monitoring the data migration
- Migration services on SAP BusinessObjects Enterprise to map source data to SAP business context
- Migration content for reconciling what was loaded in the target to the source
As noted in Figure 1, the leading software for SAP BusinessObjects Data Migration is SAP BusinessObjects Data Services. SAP BusinessObjects Data Services is a well-known and scalable data integration platform for extraction, transformation, and loading (ETL). Some of the key features of SAP BusinessObjects Data Services include:
- Easy mapping (reusable)
- Database extraction to easily extract legacy systems
- Reusable and sharable transforms and functions (drag and drop) to minimize hand coding
- A single interface to design data structures and build transformation rules
- A visualization of source-to-target metadata to analyze data lineage and impact
- Profile and analyze any source data
- Web-based administration to manage job operations
- Operational dashboards to monitor job execution and trends
SAP BusinessObjects Data Services can connect to any source or target system via the adapter framework. While it does have native connectivity to SAP systems, it also has connections to files, ODBC, mainframe, XML, Excel, and many other connectivity options. Architecturally, SAP BusinessObjects Data Services resides in the orchestration and integration layer of your existing architecture.
From a business perspective, you can reuse SAP BusinessObjects Data Services after the data migration project for ongoing integration (e.g., master data integration with multiple systems). You can also use it for ongoing data quality, including on-entry quality check within the SAP ERP and SAP CRM applications. Additionally, it can help you drive data governance projects, especially as they relate to cleansing, quality, and the synthesis or removal of duplicate data.
Deep Dive into the SAP Best Practice Content for Data Migration
The SAP BusinessObjects Data Migration content is provided by the SAP Best Practices development group. SAP Best Practices development is not part of the software development team. They are a separate team with the mission to develop content and configuration to speed implementation time. They do not change the underlying software; they build content to load or plug in to the existing software. The content becomes a key differentiation point because it can save up to 30 percent of implementation time.
As part of our deep dive, let’s look into the predefined migration of the SAP Best Practices content before moving on to some documentation.
Predefined Migration Best Practices Content in SAP BusinessObjects Data Services
Data validation content enables the mapping of customer data to SAP data, including validation of key values. Content for SAP ERP and SAP CRM is included. Examples are:
Documentation
Some examples of documentation that is included are:
- A quick guide on how to install the content
- A process guide called Business Process Document for Data Migration that provides a process flow and the sequence in which the objects need to be loaded, including specific instructions
- An extension guide that provides the information you need to extend and modify the content according to your specific needs
The scope of objects provided is quite extensive. Figure 2 shows a list of objects available for SAP ERP, while Figure 3 shows objects for SAP CRM.
Figure 2
Available objects for SAP ERP
Figure 3
Available objects in SAP CRM
In addition to these objects, there is always new content being developed, with new content released a couple of times each year. For example, in 2011 there will be new content for data quality for the customer objects, extended SAP ERP HCM content, retail article master (for the retail industry), and new capabilities for project management and data governance.
In addition to this content, SAP is releasing Rapid Deployment Solutions. These include Rapid Data Migration, in which additional migration content will be provided. Planned content in this area includes migration content for utilities, banking, and extended SAP CRM content. We are currently evaluating the content that should be provided for 2012.
Step-by-Step Demo on Using the Migration Content in SAP BusinessObjects Data Services
Now that you have an understanding about the solution, the software, and the content, the following is a step-by-step example of how to get started with SAP BusinessObjects Data Migration. Normally companies do not embark on a data migration project alone, especially the first data migration project to the SAP system. Most likely SAP services or an implementation partner will be involved in the migration. Use these steps to get started with the software and the content.
Step 1. Go to https://service.sap.com/bp-datamigration (requires login credentials). Optionally, you can review the overview presentation and watch the demo. Then select Data Migration DVD. This takes you to the location to download the content. You can download the DVD even if you do not own the software. If you do have a key for the software, the newest version has a silent installer that installs both SAP BusinessObjects Data Services and the migration content.
Step 2. Once you complete the download and unzip the file, select index.htm. This brings up a page with access to the getting-starting guides and the documentation. Alternatively, you can get the same information in online help. To use the guides from online help, go to https://help.sap.com/ and follow menu path SAP Best Practices > Cross-Industry Packages > Data Migration. Online help provides quick guides for SAP ERP and SAP CRM, walking you through manually loading the migration content into SAP BusinessObjects Data Services.
Step 3. At this point, even without the software installed, there are still key guides and places to start planning for the migration. The migration content has Word documents and Excel templates for each of the objects in the SAP system (e.g., a document on materials, and another on customers). You can read the document to understand the object and use the Excel spreadsheet to start the research on how the source data maps to the target.
Step 4. When you do decide to install the software, you can get a temporary key from https://service.sap.com/licensekey. As mentioned above, the newest download includes a silent installer that installs both the software and the content. However, if you already have SAP BusinessObjects Data Services installed, you can use the guides to install the content manually. Depending on your experience, the manual installation of SAP BusinessObjects Data Services and the content could take four to eight hours.
Step 5 (optional). If you are new to SAP BusinessObjects Data Services, you might want to start by going through the SAP BusinessObjects Data Services tutorial that is included in the documentation when you install the product.
Step 6. Once everything is installed, you can see a project called AIO_BPFDM_IDOC that exists in SAP BusinessObjects Data Services. This project has jobs for each of the objects, including Job_AIO_CustomerMaster_IDOC for the customer master and Job_AIO_MaterialMaster_IDOC (Figure 4).
Figure 4
Customer master and material master jobs in AIO_BPFDM_IDOC
Step 7. Drill into the customer job and you can see the mapping and validation that takes place (Figure 5). The validation is against required lookup fields in the SAP system, validating mandatory fields as well as the format (e.g., if the phone numbers all include country code and if the US ZIP code is in 5+4 format).
Figure 5
Mapping and validation in the customer job
Step 8. The mapping within the job, as shown in Figure 6, has the source fields on the left and the target fields on the right. The pop-up screen in Figure 6 is produced by double-clicking MapLegacy. This example shows the mapping of the country from the source to the target.
Figure 6
Mapping within the job
Step 9. The migration content includes Word documents explaining each IDoc, as well as spreadsheets for mapping the legacy customer data to the SAP IDoc structure. You can use the Excel template to start the mapping research of the source to the target (i.e., the source system table customer and FirstName field will be mapped to SAP table KNA1 Name1) prior to the mapping in SAP BusinessObjects Data Services (Figure 7).
Figure 7
Use an Excel template
Step 10. The other component that is important during the migration is the Migration Services tool. This tool, which is delivered by SAP Best Practices, reads the IMG configuration data from the SAP system and enables mapping of source values to SAP configuration values. In this example, U.S.A., US, and Vereinigte Staaten von Amerika in the source system should map to US in the target system (Figure 8).
Figure 8
Migration Services tool
Using the steps above, you can get started using the migration content and SAP BusinessObjects Data Services for your data migration project. For a complete demo of data migration, go to https://service.sap.com/bp-datamigration and select demo.
Reporting, Visualization, and Governance with Data Migration Projects
In addition to the jobs, workflows, and mappings provided in SAP BusinessObjects Data Services, there are also reports and visualizations to keep you updated on the status of the data migration. The data migration project can lay the groundwork for data governance, data ownership, and ongoing data quality. Governance of data and information is a topic attracting increased interest, and data migration falls into the governance category, providing the first opportunity to implement governance processes around the management and data quality expectations. Now let’s look at some examples of the reports provided today. (In Q3 of 2011 further data governance processing will be provided in the SAP Best Practices content.)
Figure 9 shows example reports accessible via the Web-based SAP BusinessObjects InfoView that display the status of the migration objects.
Figure 9
Example reports in SAP BusinessObjects InfoView
Figure 10 shows the number of valid and invalid records for materials via the Material Master folder in Figure 9.
Figure 10
Material Master Summary report
Figure 11 is an example reconciliation report that is accessed via Folders > Data Migration Objects > Management Tables, showing which records have failed.
Figure 11
Example reconciliation report
Using SAP BusinessObjects Data Migration When Migrating to Non-SAP Applications
There are several situations in which companies may need to migrate to non-SAP systems or load data from one non-SAP system to another — loading a data warehouse, for example. Extraction, transformation, and loading of non-SAP systems are supported by SAP BusinessObjects Data Services. Of course, the migration content discussed above cannot be leveraged, but SAP BusinessObjects Data Services does provide blueprint examples.
These blueprints provide examples of quality rules, workflows, and jobs that can be used. These are used to depict best practice settings for common data quality issues. With these blueprints, example data is used to illustrate how a job and quality rules should be designed. There are also blueprints for text analysis functions in SAP BusinessObjects Data Services. You can access the blueprints and related documentation on SDN at https://www.sdn.sap.com/irj/boc/blueprints.
Ginger Gatling
Ginger Gatling works in solution management covering information management topics, focused on data migration. Prior to this, she covered business process management topics (including SAP Business Workflow, Universal Worklist, and SAP NetWeaver Business Process Management), integration topics (SAP NetWeaver Process Integration), and other SAP NetWeaver topics. She coauthored the second edition of Practical Workflow for SAP and has delivered a series of workshops on workflow for SAPinsider.
You may contact the author at ginger.gatling@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.
Oren Shatil
Oren Shatil leads product management for technology, Big Data, and analytics solution packages at SAP. He has more than 12 years’ experience in development, consulting, and management, and has a passion for creating packages that combine best practices content and winning solution delivery methodology. You can view his profile at https://www.linkedin.com/in/orenshatil.
You may contact the author at oren.shatil@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.
Premium
