Know Every Aspect of the ABAP Test Cockpit: Part 1

Over time your collection of custom ABAP programs grows. Their quality is inevitably inconsistent. The SAP NetWeaver ABAP Test Cockpit (ATC) is a freely available automated tool to help you find those quality issues so that you can clean them up. The robustness of ABAP programs is important for smooth production operations.

The ideal time to ensure the robustness of an ABAP program is during its first-time development in a project. However, maintaining the balance of possible deadline delays and the cost of priorities in a project may cause the manual quality check processes to suffer. Once these ABAP programs are in the productive environment, they join the pool of already existing ABAP programs, and finding quality issues in the overall ABAP portfolio becomes expensive. For example, performance-related errors on the SELECT statement without WHERE clause are some of the critical errors for hefty size tables in production. These errors have the potential to slow down the day-to-day operations of transactions involving that program.

To prevent this expensive problem, you need a proactive and automated quality check process not only in production support but also at the beginning of the development of ABAP programs. ATC is one tool that can help find such errors along with the line number in the program where these errors exist. Projects can leverage ATC’s fully or partially automated quality checks based on their requirements.

Overview of ATC

ATC verifies the quality of ABAP objects in multiple areas, including performance, security, syntax, programming conventions, robust programming, and general checks. Priorities are assigned to these different categories of checks based on their impact. These checks are customizable and transportable.

ATC is a combination of the Syntax Check (transaction codes SE38 and SE80) tool, Extended Syntax Check (SLIN) tool, Code Inspector (SCI) tool, and the Code Vulnerability tool (paid feature). Except for the last feature, the rest of ATC is free. ATC does not just integrate these tools, but also acts as a complete governance platform for code quality checks due to its reporting, transport block, and exemption workflow features.

ATC reporting is covered in detail in part 3 of this article. Transport block, which is a functionality of the ATC tool, has both fully and partially automated check options. In the fully automated transport block, before release a transport is checked automatically for ATC errors in the ABAP programs under it. If there are ATC errors found in those ABAP programs, the transport is not allowed for release, and it remains in that system.

I cover the setup of ATC in depth in this article. See the section “Insight from Experiences” at the end of this article for advice on how to use ATC effectively.

ATC Setup and Use Scenarios

ATC can be used in multiple ways via its different setup options and integrated features. The ATC setup can happen in two ways—local and central:

• Local setup happens in the development system. This setup checks the code quality of all ABAP programs in that development system locally. In this case the programs are in work-in-progress status as well as in completed status.
• Central setup happens in the quality or testing system. This central setup can check the code quality of ABAP programs of different development systems that are transported to the quality system. The results of the central setup are sent back to the respective development systems. In this case, only the completed programs that are transported to the quality system are considered for given filter criteria in this setup execution.

These two options are explained in detail later in this article. Table 1 shows all possible usage options of ATC in any project or landscape. 

Table 1
ATC use scenario with required setup option

(Note: Technical experts recommend that developers do a self-review and arrange for a peer review of the quality of ABAP objects as part of the ABAP development process. This process can be included in the ABAP code review checklist. In a self-review the developer who created the code checks the quality, and a peer review by a different developer can provide a quality perspective from a fresh pair of eyes. An ATC report specific to a given developer ID can be the first item in a peer review checklist for verification. Quality managers, technical leads, and project managers can view the ATC checks of the entire development by centrally run reports for a given transport layer, package, user, or software component.

In SAP Solution Manager, ATC setup is a mandatory prerequisite for the Custom Code Lifecycle Management (CCLM) quality setup. ATC setup is required for all the managed systems having ABAP development so that the results of an ATC central run can be automatically fetched by the CCLM utility in the quality (of ABAP programs) feature. The ATC results are shown graphically in CCLM. This is useful for managers for observing quality trends in the ABAP portfolio, especially in the case of large-scale implementations involving hefty ABAP developments.

As mentioned earlier, some percentage of errors observed by ATC will be ignored as their rectification won’t add value. If this decision to ignore an error happens outside the SAP system manually, then Solution Manager won’t be able to show the ignored errors distinction versus the errors planned for rectification in its graph showing the results trend. Technical leads and quality managers will need to pay attention to this fact during the use of the CCLM quality feature.)

Local Setup in the Development System

ATC local setup (for local run setup, local run, and periodic scheduling in a background job) is carried out in the development system where ABAP programs are being developed. Once this setup is completed, you can run ATC reports locally manually as well as periodically. Background jobs can be scheduled as part of the setup for automated periodic reports execution.

Below are the steps for an ATC local setup:

1. Log in to the development system of SAP ERP Central Component (ECC), Master Data Management (MDM), or any other SAP system in which ABAP programming happens. This log-in ID should have authorization for the ATC transaction including admin rights.

2. From the SAP Easy Access screen, go to transaction code ATC. This transaction has two major sections as shown in Figure 1: ATC Administration and Quality Governance. The ATC Administration section includes initial setup-related options, report execution scheduling and monitoring, and the exemption approver listing. The Quality Governance section offers an exemption browser for handling approval or rejections and helping manage check variants.

Figure 1

ATC transaction screen

3. For the setup of ATC, click the Configure ATC option under Setup in Figure 1. This action opens the setup screen (Figure 2).

Figure 2

ATC setup screen

4. To do the setup, click the change icon shown in Figure 2. Enter the values as shown in Figure 3. I describe these values below.

Figure 3

ATC setup values

Click the save icon after all the values are entered in Figure 3. This is an important step as it decides the behavior of the ATC tool in different usage scenarios. Below is the meaning of each set-up value.

a. Global Check Variant – DEFAULT. This is the Code Inspector’s global check variant that is used in checking the ABAP code on different parameters. DEFAULT is the SAP-provided standard check variant. It has predefined values set on different areas such as performance and usability. This is explained in detail in the “Customizing Check Variant” section.

b. Do You Want to Enable ATC Exemptions in the System? Yes. This step confirms the decision to use ATC exemption workflow. Whenever a program is reported with an ATC error, there are two ways of dealing with it. You either rectify the error by making the necessary changes in the program or ignore the error by justifying it. This ignore option happens by requesting the exemption of a given error to the approver (i.e., the technical lead or quality manager). As part of ATC exemption workflow, the exemption approver can ask for more explanation or reject or approve the exemption request. This is explained in the “ATC Exemption Workflow” section in part 3 of this series.

c. For Which Kind of Results? (exemption applicability) – For Central Results Only. This step confirms to which results the exemption should be applicable. This setup selected Central Results. There are two kinds of results outcomes in ATC:

• Local results – ATC is run locally within the same system, and those results are referred for exemption workflow. This fits in the category of For Any Results in Figure 3.
• Central results – ATC is run via a quality or test system, and the results are sent back to the development system. In ATC, this quality or test system is called the master system. This kind of ATC run is applicable when there is a need to analyze the completed ABAP program that has been transported to the quality or test system.

d. Master System – Quality system with a trusted Remote Function Call (RFC) connection to it. This refers to the quality or test system in which the testing happens for ABAP programs. In the case of multiple systems, decide thoughtfully which one system to make the master. Generally, the quality system in the landscape is good to consider as the master system. As mentioned earlier, the master system is important as the ATC runs of this system are sent back to the development system and are referred to as central results.

e. ATC Behavior on Release – No ATC check. This refers to the Transport Block scenario. It guides how ATC should react to a transport release that contains ABAP programs with ATC errors. In this example, no check would be performed. There are other options in the dropdown such as Inform on Errors (Priority 1 or 2) or Block on Errors (Priority 1 or 2). An additional setting is needed in case of blocking of a transport due to ATC errors in the program the transport contains. I describe this scenario in the “Transport Block due to ATC Errors” section of part 3 of this series.

5. Once this setup is done, an ATC run can be scheduled. To do so, go back to the home screen of the ATC transaction. Click the Schedule Runs option under Runs (Figure 4). This action displays the screen in Figure 5.

Figure 4

The Schedule Runs option of ATC

6. There are two ways to schedule an ATC run:

• Ad hoc (i.e., execute the ATC run any time manually)
• Periodic (i.e., execute an ATC run automatically in a background job in a periodic manner)

For an ad hoc run click the Create button (Figure 5) to open a pop-up screen in which you specify the name of the Schedule run (Figure 6).

Figure 5

Schedule ATC runs

Enter a name and click the enter icon as shown in Figure 6.

Figure 6

Name of the ATC schedule run

7. In the next screen, enter the values shown in Figure 7. This screen is populated with the Check Variant name, which was saved in the ATC configuration setup in step 4. This step decides which set of ABAP programs are checked in the ATC tool. There are two options:

• By Query. ABAP objects can be specified via their package names, via the transport layer, or via software components. In this example, the transport layer is used and its value is mentioned. It means that all the transportable objects of this development system, which are attached in a transport request, are checked in the ATC tool.
• By Object Set. In this option, the Code Inspector (CI) object set variant can be used. This option is used when a few selective ABAP objects require an ATC check. This object set variant is prepared in CI (transaction code SCI).

Save the setup by clicking the save icon in Figure 7.

Figure 7

Enter values for the schedule run

8. After you save your entries, the schedule run name mentioned in Figure 6 appears in the list of Figure 8. Select the schedule run and click the Schedule button (Figure 8).

Figure 8

Schedule the ATC run

9. In the next screen (Figure 9), the execution parameters appear. In this case, there is no need to change these parameters.

It is generally recommended by technical experts that you save a variant of these values by clicking the save icon. In this screen in the header block, there is a Set to Active Result check box. If this option is selected, then the result of this run is marked as the latest run and is set as Active Result in the report. Active Result is represented by a yellow bulb icon in the report. There is an input field for Life Span in Days. By default, it appears as 120 days. It means the result has a life of 120 days. In case the project requires ATC results to be available for a longer duration, this input can be adjusted according to the number of days needed. Click the execute icon shown in Figure 9 to schedule the run for ATC for an immediate run.

Figure 9

Schedule the run execution

The system prompts a pop-up noting a successful scheduling, as shown in Figure 10. Click the continue icon to continue in this screen.

Figure 10

Success message

10. With step 9, the ATC tool starts running its check for the development system transport layer with the Default check variant. To see the status of the execution, go back to the home screen of transaction code ATC and double-click Runs – Monitor and Control Runs as shown in Figure 11. 

Figure 11

The Monitor and Control Runs option of ATC

11. A selection screen appears (Figure 12). Click the execute icon in this screen.

Figure 12

Selection screen for monitoring runs

12. In the next screen, double-click the schedule run. In the bottom screen, all the execution details appear as shown in Figure 13.

Figure 13

ATC execution with steps

13. This was the manual execution of a report in an ad hoc way whenever demanded. There is a way to schedule automatic execution of the ATC tool in a periodic manner via a background job. To do this setup, go to the home page of the ATC transaction and double-click the Schedule Runs option as shown in Figure 4. In case the required schedule run doesn’t exist, create one by following steps 5, 6, and 7.

In Figure 8, select this newly created run that is required for periodic execution (e.g., AUTOMATED_RUN in this case) and click the Schedule button as shown in Figure 8.

14. In the next screen, click the Program menu button and select the Execute in Background option as shown in Figure 14. This option means that a background job will be scheduled for the selected ATC run.

Figure 14

Background job scheduling of an ATC run

15. In the next screen, set the print parameters by entering locl in the Output Device field as shown in Figure 15. Click the enter icon.

Figure 15

Print parameters setup for the background job

16. In the next screen, enter the conditions for the periodic job to execute. This screen is for scheduling a background job in an SAP system. Enter the Scheduled Start Date and Time as required and select the Periodic Job check box. Now click the Period values button as shown in Figure 16.

Figure 16

Background job scheduling screen

17. A small pop-up for period selection appears (Figure 17). Click the Weekly button so that the ATC run happens every week. Click the save icon in the pop-up and then click the save icon in Figure 16.

Figure 17

Period values selection for the background job

18. A message appears at the bottom of the screen on the successful scheduling of the ATC run background job. This job can be found after you execute transaction code SM37. In this transaction, you can enter the values as shown in Figure 18 and click the Execute button.

Figure 18

Enter values for the background job selection screen

19. In the next screen, a finished job appears for the scheduled run (Figure 19).

Figure 19

Job status for an ATC run

This job screen shows that the periodic scheduling happened successfully.

Central Setup in a Quality or Test System

ATC central setup (ATC master system for central run setup, central run, and periodic scheduling in a background job) is carried out in a quality or test system where ABAP programs are transported. Once this setup is completed, you can run ATC reports centrally in the quality system manually as well as periodically. You can schedule background jobs as part of the setup for automated periodic reports execution.

In the case of a quality system setup, most of the tasks are similar to those of a local setup described in the “Local Setup in the Development System” section. This section covers the unique steps and refers to the common steps from the local setup section.

20. Log in to the quality or test system with an ID that has ATC admin access. In the ATC setup, this system is referred to as the master system. From SAP Easy Access, go to transaction ATC.

21. Perform steps 2, 3, 4, 5, 6, 7, and 8 for configuring the ATC basic setup and creating a schedule run. The difference in a local and central run happens in the scheduling part. In the case of a central run, the run results are distributed back to the development system via an RFC. Also, the results of a central run are preferred as an active result set in this example setup. The result of ATC run via this setup is called an active setup if its check box is selected. This is the way you can make a distinction between any manual run versus the run via set up automatically.

The preferred active result set has a yellow bulb icon in the Object Navigator (transaction code SE80) report of ATC results. This is helpful when multiple ATC runs happen (manually and automatically or both) and you want to see the result set that you would consider for review and decision with regard to rectification.

22. To set up this ATC run as an active result set in the Schedule Run screen (Figure 20), add the highlighted configuration values (the boxed check boxes). In this case, select both the Central Check Run and Set to Active Result check boxes. Selecting both these boxes means the result of this run is set as an active result in a report with a bulb icon.

In the Postprocessing section at the bottom of this screen, enter the name of the development system’s trusted RFC. This setting makes this setup a central run or master run. Click the save icon to save it as a variant if needed, which I recommend. Click the execute icon to run the ATC tool.

Figure 20

Master system setup

23. The execution is the same as a local run execution. Follow steps 10 through 19 for monitoring and controlling the runs and for automatic periodic scheduling of ATC runs via a background job.

24. The difference in this run is that the results of ATC run are sent back in the development system. Developers and tech leads use these results for reporting purposes and corrective actions.

With the above steps, an ATC quality check on ABAP programs is ready to run. Both the above types of setup (local and central) used the DEFAULT check variant of Code Inspector in this example. Customization of a check variant is carried out specific to project needs. After the setup of ATC, you can view ATC reports by executing transaction code SE80 of the development system. All the details of ATC reporting and the customization of a check variant are explained in “Know Every Aspect of ABAP Test Cockpit (ATC) Check Variant Customization and Reporting: Part 2.”

Insight from Experiences

In production support scenarios, the effective use of ATC is a challenge. The support team is not responsible for ATC errors of existing programs that were developed by other project teams. Rather, the support team is responsible only for the part of the code of a program that it has modified during the support process. This scenario holds true for almost all organizations and projects. This model is a challenge for ATC use. This also poses hurdles for the transport block functionality of ATC. In this case, transports will be blocked due to existing errors in a program whose rectification is not the responsibility of the support team. The default check variant of the ATC Code Inspector tool doesn’t understand this behavior and shows all the errors. Therefore, you need to tailor a check variant via custom development so that ATC executes the quality check only for the version of the program after a certain predefined date (e.g., after the start date of the support contract).

All the tools integrated in the ATC utility are related to a technical quality check of ABAP programs. None has any element to check the functional correctness or logic verification of programs. To verify these, you have to wait for the testing phase in the project.

For projects using extensive ATC checks in the development phase, a typical trend is observed on the percentage of ATC errors that are corrected versus errors that are ignored. Of all the ATC errors, around 35 to 40 percent are rectified by the development team, whereas the remaining 60 to 65 percent of ATC errors are ignored. Generally, this percentage is observed after the manual analysis of all ATC errors. This information is based partially on my experiences with large-scale implementation and enhancement projects.

Some ATC errors are ignored as their rectification won’t add much value. For example, a  SELECT* clause for a custom database table that has very few fields is OK although ATC will pose it as a high-priority error. Its rectification won’t add much value due to its negligible risk of slowing down production, whereas the production-critical errors are picked up for rectification. The percent of error rectification provides confidence in the ability of ATC functionality to find issues on the technical quality front. Again, in this case, use of the ATC transport block functionality will be challenging. If errors are ignored via an ATC exemption process, then transports won’t be blocked for release. However, if the decision to ignore errors is planned outside the system via a manual review, say in Excel, then transports will be blocked.

A manual code review process generally follows a predefined checklist. This checklist may not be as detailed as ATC checks. Also, manual errors cannot be avoided, while in ATC there is no manual intervention, so human errors are avoided.

There are deadline and cost priorities for each project. A project needs to find a balance on how much effort to spend on code quality checks during the realization, development, and testing phases. Otherwise, rectification may affect the project timeline. If rectification does not add value, you can let it go. However, if a critical technical error moves from the development to the testing phase without rectification, then rectification becomes expensive. The same error becomes more expensive if it moves from testing to a pre-go live check phase and certainly is a costly matter if it gets into production. The decisions on code quality processes are based on a mix of cost, timeline, potential delays in the project, and the overall call of the program management team.