For a recent client, I had to migrate business partner data from a homegrown legacy system to SAP CRM 2005. The requirement was to load business partner data, marketing attributes, external identifiers, contacts and their marketing attributes, employee relationships, and various partner functions. This involved loading about one million business partners with their various attributes in a short span of time.

The initial evaluation revealed that the Data Transfer Workbench (transaction SXDB_TOOLS) with object type BUS1006 was not a viable option because we needed to load many repeating attributes. We decided to use a Business Application Programming Interface (BAPI) approach with a custom load program to load each of the attributes separately. This option provided more flexibility in the data load process.

SAP’s benchmark for loading a business partner using a direct data load program is approximately 35,000 records into the system. In this case, the custom data load program using BAPIs was loading fewer than 5,000 records into the system. I’ll take you through the tasks we performed to improve the system performance and show you some key lessons we learned in this project.

The Custom Program and Enhancements

For the BAPI approach, we used a custom program to load the business partner data into SAP CRM and attach the attributes to the business partner. You can view the foundation of this code in the Downloads section at the end of this article.

After you create the business partner in SAP CRM, the custom program adds the industry sector, tax type and number, sales area, shipping, and billing address data to the new business partner. The number of attributes that you need to add to the business partner depends on your organization’s business process.

After you create the business partner, you migrate the attributes maintained in your legacy environment into the new SAP CRM environment using BAPIs. Table 1 provides the list of BAPIs we used in the business partner upload. The process involved a total of 15 BAPI calls. The first BAPI call created the business partner, using a legacy identification number from the legacy system, and transferred it into SAP CRM for reference purposes. Subsequent BAPI calls used the business partner number created in SAP CRM to create marketing attributes and contacts.

Even though the program was tuned for performance efficiency, we still had room for improvement. The following section describes issues we encountered during the data migration and how we resolved them.

Issues and Resolutions

Issue 1. BAPIs called multiple times during the transfer process. Per standard protocol, we placed a COMMIT and WAIT command after each BAPI call, which degraded the system performance.

Resolution: We developed a compromise, placing a COMMIT command after each BAPI call to create a business partner (BAPI_BUPA_CREATE_FROM_DATA). After that, we executed only one COMMIT command for the remaining BAPI calls. We also only executed COMMIT and WAIT commands for every 100 records.

Issue 2. Too much memory consumption. Each record had about 150 fields, with each field approximately 15 bytes long. This put a heavy demand on memory requirement, and resulted in short dumps for large files.

Resolution: We modified the program so it used the extract data set rather than the internal table (Figure 1). This reduced the amount of memory necessary to load the long data record.

Figure 1

Area of custom code modified to extract data set

Issue 3. Heavy input/output (I/O) load. Figure 2 shows the system log (transaction code SM50 [single server]/SM51 [multiple servers]) in which the program accesses table CRMD_AUSP_HIST to retrieve one record. The read statement on this table seemed to be causing heavy I/O load because the context switches and system CPU use (Time) was high.

Figure 2

The figure above shows the process overview as you see it in transaction code SM50/SM51

The main problem was that the function module CRM_MKTBP_UPDATE_AUSP_EX_DB was performing a full table scan on table CRMD_AUSP_HIST, as shown in the code snippet in Figure 3. Even though CRM_MKTBP_UPDATE_AUSP_EX_DB is a standard SAP function module, the primary index is not sufficient for good performance. No suitable index exists for the ABAP statement in the program, which results in the poor system performance.

Figure 3

The program code shows the ABAP statements that were causing the problem

Resolution: Figures 4 and 5 confirm that the SELECT statement in CRMD_AUSP_HIST took a great deal of run time. To overcome this, in transaction SM11 we created a new secondary index on table CRMD_AUSP_HIST with fields MANDT, PARTNER_GUID, and ADZHL.

Figure 4

Load on SQL tables

Figure 5

Execution plan for SQL statement

Issue 4. Old business partner data in table CRMD_AUSP_HIST. If the value of a marketing attribute changes for a business partner, the system stores the old value in table CRMD_AUSP_HIST. This causes the table to contain multiple data records from table AUSP. In this case, the marketing attributes in table CRMD_AUSP_HIST become large with old business partner data and cause a performance bottleneck while loading marketing attributes.

Resolution: Delete old data from this table using report CRM_MKTBP_DELETE_HISTORY. For more information see SAP Note 696274 “Change history marketing attributes.” Note that you shouldn’t run this report if you want to see the change history for the marketing attributes, such as how many times a customer segment changed.

Issue 5. Custom program runs with old data. This is a common issue when the organizational model changes often.

Resolution: Make sure the organizational model is buffered. In SAP CRM, schedule the report HRBCI_ATTRIBUTES_BUFFER_UPDATE in transaction code SE38/SM36 (job scheduling). For further information, refer to SAP Note 110909 “Composite SAP note on generic attributes.” If the organization model changes daily, you should run this report daily. Check for system performance in the development environment first. If the report doesn’t slow the system down, schedule it in production.

Issue 6. Need to reorganize data in CRM Middleware. Sometimes the CRM Middleware data is scattered, which can slow data load performance.

Resolution: Schedule report SMO6_REORG2 in transaction code SE38/ SM36 to clean up the CRM Middleware. Further information is available in SAP Note 713173 “Update of the CRM Middleware reorganization SMO6_REORG.” If relevant for your scenario, schedule the report with the options Reorganization of processed BDoc type messages and Reorganization of trace data.

Issue 7. Performance slowdowns caused by generation of business partner Business Documents (BDocs) BUPA_MAIN, BUS_TRANS_MSG, or BUPA_REL. This could happen when you perform an initial data load from R/3 into the CRM Server, from legacy systems into SAP CRM, or if SAP CRM is a standalone system. In other cases, distributing a BDoc type is undesirable for objects that are irrelevant for distribution.

Resolution: You can temporarily or permanently suppress the generation of these business partner BDocs. Information about how to set this up is available in three SAP Notes:

If you meet the prerequisites to use these SAP Notes, start transaction SMW3_00 and select the Do Not Snd indicator for every relevant BDoc type.

Issue 8. Updating multiple business partners at a time. To update each individual business partner can take a significant amount of time.

Resolution: Use function module CRM_MKTBP_ASSIGN_ATTRIBUT to update up to 5,000 business partners in one BAPI call. Refer to SAP Note 717624 “Mass evaluation of business partners” for more information about mass loading marketing attributes.

In addition to the above, investigate if SAP Note 626037 “Performance marketing characteristics” is relevant for your scenario. This SAP Note provides pointers on performance issues in loading marketing characteristics.

Use these lessons learned to streamline your own migration project when using the BAPI method. For a summary of all the SAP Notes we used in this project, refer to Table 2.

Pandari Kubendran

Pandari Kubendran has been working on SAP NetWeaver technology for the past 13 years. He has architected and implemented various SAP solutions, including SAP NetWeaver Master Data Management, SAP NetWeaver Process Integration, SAP NetWeaver Portal, SAP Solution Manager, and SAP BusinessObjects Data Services.

If you have any comments about this article or SAP Professional Journal, or want to submit and idea for an article, contact the SAP Professional Journal editor.

You may contact the author at kpn.pandari@hotmail.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.