Discover how SAP Business Connector C/C++ services work internally. Find out when it’s best to use them and when you should avoid them. Uncover tips and tricks for creating and debugging SAP BC C/C++ services smoothly.
Key Concept
The four basic rules for
C/C++ services are:
- Keep all C/C++ services (and the interfaces used in their input/output definitions) in a separate package. Don’t mix in Java services and don’t use folder names that are also used for Java services.
- Keep the number of “frontier crossings” (function calls from one language to the other or data copy operations from one heap to the other) between Java and C/C++ to a minimum.
- Use simple data types (primitive arrays, strings, or simple records) for the data exchange between Java and C/C++.
- Don’t use a C/C++ service for ordinary mappings that could equally well be done in a Java service.
Assume you want to exchange electronic documents with a business partner, and these documents need to be digitally signed to become legally binding. The obvious choice for performing this kind of data exchange easily is the SAP Business Connector. However, you don’t want to endow SAP Business Connector with the ability to sign documents automatically. Instead, an authorized human person with full power of attorney needs to sign each document individually before it is sent out.
You have purchased the necessary hardware for that (e.g. a crypto-token that can be plugged into a card reader or USB device during the sending process), but this hardware needs to be accessed via a device driver written in C. How can you use it in your business process implemented on top of the SAP Business Connector?
One little-known feature of SAP Business Connector is the ability to implement services in C/C++ and to seamlessly consume these services from within Flow mappings or Java services. In the above scenario, you could write one such C/C++ service that reads the crypto-token from the card reader and then use the signature in the rest of your business process that is implemented in Flow.
In the following I’ll explain how C/C++ services work and in which situations it makes sense to use them. I’ll provide recommendations about what to pay attention to when designing and implementing C/C++ services and what pitfalls to avoid. I’ll then finish with an actual example illustrating the process of creating, developing, and debugging a C/C++ service.
Note
Early versions of BCStarter.exe had a bug that prevents SAP Business Connector from loading the DLLs containing the C/C++ services automatically. Therefore, make sure you have at least CoreFix 2 installed on SAP Business Connector 4.8. Otherwise, some steps in the article will not work as described.
I assume that you are familiar with the standard C/C++ development process on the platform where your SAP Business Connector is running. Also you should know the basic SAP Business Connector documentation about how the SAP Business Connector Developer works, and understand the package, folder, and service concept for managing SAP Business Connector services. If you need instructions for this, refer to the “
SAP Business Connector Developer Guide” (login credentials required).
Finally, you should also review my recent article “
Develop, Debug, and Manage Java Services in SAP Business Connector 4.8.” Some of the key concepts about code revision and setup of your development landscape can be applied to C/C++ services as well. I won’t repeat these points in the current article.
How C/C++ Services Work
SAP Business Connector uses Java Native Interface (JNI) to execute C functions within Java code. JNI is quite complex and SAP Business Connector services pose additional constraints on the functions that you can use. Therefore, SAP Business Connector generates the necessary Java coding that executes the C function when the C/C++ service is invoked. It also generates a C function stub with the correct signature and a few helper functions.
Each C/C++ service corresponds to one C function. Normally, all services belonging to one folder are combined into one shared library, but it is also possible to combine all services of one package into one shared library.
When to Use C/C++ Services
In normal situations you should not use C/C++ Services for the following reasons:
- It takes much more effort to create a C/C++ service than a Flow or Java service.
- C/C++ is a difficult language. It is easier to make mistakes in it than in Java, and usually a mistake in C coding results in grave consequences. If you make a mistake in a Java service, the worst that can happen is that the service does not work. One mistake in a C/C++ service can crash your entire SAP Business Connector server or introduce a memory leak that will bring down the server in the long run. Only experienced developers should create C/C++ Services.
- A change in a Flow or Java service is active immediately. A change in a C/C++ service requires a restart of the SAP Business Connector server before it becomes visible.
Nevertheless C/C++ services are a valuable tool when used in the appropriate situations and with care. They are the perfect solution, if:
- You want to reuse existing C libraries instead of having to implement everything again from scratch. With a bit of effort, it is also possible to reuse existing libraries written in other languages such as Fortran, Cobol, or Pascal.
- You need to access OS features or hardware.
- Stress testing your solution has revealed a performance bottleneck and you want to replace that particular step or operation by an optimized and fast C implementation.
- Your development team is comfortable with C/C++, but does not have Java experience.
Design and Implementation Recommendations
The first basic rule you need to know is: Always isolate your C/C++ services into a separate package of their own. There are two reasons for this:
- Because a Java process cannot unload a native JNI library, the entire SAP Business Connector process needs to be restarted if a C/C++ service (and thus the JNI library) is changed. Therefore, SAP Business Connector cannot reload a package that contains C/C++ components. Consequently, Java components in the same package can also not be changed without restarting the entire SAP Business Connector server.
- Normally each SAP Business Connector package has its own ClassLoader class, which loads the Java classes for the services contained in that particular package. This allows you to use the same folder name (corresponding to a Java class name) in several different Packages. However, the JNI library and the Java stubs invoking the C functions in that library need to be loaded by the global system class loader. Therefore, a package containing C/C++ services uses the SystemClassLoader class. This leads to the following problem: if the package containing the C/C++ services also contains Java services, and if other packages use the same folder name for their Java services, then the services of the other packages cannot be found. The Java class from the C/C++ package that corresponds to the folder name is globally visible and prevents the identically named Java classes from the other packages from being loaded.
The next important concept concerns performance. Usually, communication between two programming languages is slow, and JNI is no exception in this regard. Therefore, you need to make sure that the amount of data copied between Java and C — as well as the number of function calls across the Java and C boundary — is kept as small as possible. Otherwise, the overhead involved in JNI easily consumes the performance improvement gained from implementing the task in C — up to the point where it might become even slower than a pure Java implementation. A few examples illustrate this:
- Within Flow and Java you loop over a list of data records. For each record you call a C/C++ service, which processes that record and returns the result back to Flow and Java. If the list is long, this loop becomes extremely slow. In this case, you should design your C/C++ service so that you call it only once, passing the entire list over to C/C++, then do the loop in C/C++, collect the results in another list, and in the end pass that list back to Flow and Java. Now instead of 2n “frontier crossings” you only have two. (Here n denotes the length of the list.)
- The C API allows you to create similar mappings as you would implement in a Java service using the IData or Values API. However, the pipeline object (and thus the data on which the mapping is performed) continues to live on the Java heap, even when passed to a C/C++ service. Therefore, each mapping step and every little field access carried out in a C/C++ Service results in one JNI call made back to the Java heap. This is definitely a performance killer, so it is not a good idea to use C for implementing a mapping between two records that live on the Java heap. Similarly, a C/C++ service performs best if you use a byte array, a string, or a simple flat record as input/output instead of nested records or other complex Java objects. Otherwise, the C/C++ service requires many JNI calls to extract all its input values from the pipeline.
One use case in which a C/C++ service yields amazing results is if you are dealing with a simple, well-known data format that you need to process or convert. Sometimes, applying a standard multi-purpose tool (such as a standard XML parser) to a simple data structure is overkill because multi-purpose tools are always slower than customized ones.
Multi-purpose tools are the right choice if either the input is so complex that writing specialized code for it would drive you crazy (and it would not result in more efficient code anyway) or the input is not a clearly defined format, but can be varied. For example, you are writing a service that can process IDocs in general, but you do not know beforehand whether the IDocs will be ORDERS, MATMAS, or INVOIC IDocs.
Steps to Create and Debug a C/C++ Service
Let’s take a look now at an example, which is simplified so that I can outline the general ideas without getting lost in too much special-case complexity. Nevertheless, this example is meaningful enough to have some kind of real-life relevance. Suppose you need to synchronize address data between two different systems. Both systems have an XML interface, but use a different data format. [Note: version in print issue ends here.] Therefore one step in your data processing consists of converting an XML structure from the format shown in
Figure 1 to the one shown in
Figure 2. Of course, real address data has more fields than just a name, ID, and telephone number, but for demonstrating the general idea, this should be sufficient.
<?xml version="1.0" encoding="UTF-8"?>
<PersonsFlat>
<Person>
<Name>Mr. X</Name>
<ID>1</ID>
<Telephone>08/15</Telephone>
</Person>
<Person>
<Name>Mr. X</Name>
<ID>1</ID>
<Telephone>04711</Telephone>
</Person>
<Person>
<Name>Mrs. Y</Name>
<ID>2</ID>
<Telephone>12345</Telephone>
</Person>
...
</PersonsFlat>
|
Figure 1 |
Source XML structure |
<?xml version="1.0" encoding="UTF-8"?>
<PersonsStruct>
<Person>
<Name>Mr. X</Name>
<ID>1</ID>
<Telephones>
<Telephone>08/15</Telephone>
<Telephone>04711</Telephone>
</Telephones>
</Person>
<Person>
<Name>Mrs. Y</Name>
<ID>2</ID>
<Telephones>
<Telephone>12345</Telephone>
</Telephones>
</Person>
...
</PersonsStruct>
|
Figure 2 |
Target XML structure |
Note
My SPJ_CService_Solution package violates the first basic rule for C/C++ services: it mixes Java, Flow, and C/C++ services in one package. However, this code is for explanation purposes only. I wanted to show examples of the types of services needed for this section close together for easy reference.
Furthermore, suppose input and output are required as byte arrays. This makes it easy to pass the data between Java and C/C++. Assume that originally you had implemented the conversion step as a Java service using the standard approach for tasks of this type:
- The data is run through the SAP Business Connector XML parser (stringToDocument + documentToRecord)
- An algorithm converts the record the parser created into a record with the desired structure
- The new record is converted back to XML using the SAP Business Connector XML renderer (recordToDocument)
See the service spj.csolution:xmlConversion_Java in the download for an example of what this original service might have looked like. (In the end, you use this service for the performance comparison with the C/C++ service.)
However, assume that during your end-to-end testing you identified this XML conversion step to be a performance bottleneck. Now you want to replace it with an optimized implementation. In the following section, I show you how to optimize the implementation two ways:
- By replacing the XML parser and renderer with custom code that works directly on the XML stream (taking advantage of the simple structure of that stream)
- By using fast C code for the conversion
Create the C/C++ Service
Log on to the development SAP Business Connector using the developer tool. First, create a package with the name SPJ_CService. In that package create a folder called spj, which contains another folder called csample. In the csample folder, create a specification named xmlConversionSignature, which has one input and one output of type object named xmlIn and xmlOut. Then create a C service named xmlConversion.
While the system generates the xmlConversion service, choose the platform on which the SAP Business Connector server is running (I use Windows 32bit in my example). Then, select the specification you just created. The end result should look like
Figure 3. If you have a problem during any of these or the following steps, you can always peek at the SPJ_CService_Solution package. It contains finished versions of all elements created in this section with the only difference that the folder csample is named csolution. This prevents any conflicts between the SPJ_CService package you are creating and the solution package.
Figure 3
The generated C/C++ Service and its signature
At this point, the xmlConversion service is not yet operational because you still need to compile the shared library containing the underlying C function of that service. (If you were to try and execute the service at this point, you’d receive an UnsatisfiedLinkError: cxmlConversion error.) Here is the best way to set up the C/C++ services.
The following steps need to be performed on the OS level. If the SAP Business Connector server is running on a different host from the one you are working on, then log on to the SAP Business Connector host before proceeding. Open the directory sapbc48ServerpackagesSPJ_CServicecodesourcespj. Here SAP Business Connector has generated three files for you:
- A Java stub named JSBC_csample.java, which has been compiled and is not of interest at the moment
- A make file named xmlConversion.mak. This is the first C/C++ service in that package, so copy this file and name it csample.mak. You will extend this make file whenever you add a new C/C++ service to the package. You can then use this make file to compile the shared library if you don’t intend to use an Integrated Development Environment (IDE) for development.
- A C stub named xmlConversion.c. Copy this file and name it xmlConversionImpl.cpp. This prevents the system from overriding your changes, if the C/C++ service is regenerated.
Note
Microsoft Visual Studio 2005 SP1 compiles a file with extension .c as plain C, even if the project properties have Compile as C++ Code (/TP) set. Because you are going to use some C++ features, you need the extension .cpp. Remember to change the SRC file name in csample.mak accordingly and to add the compiler switches necessary for C++ if you are going to compile with make/nmake. Sample make files for all seven platforms that SAP Business Connector supports are contained in the solution package, but if you want to use them at the current state, you need to remove all source files from the SRC variable except for xmlConversionImpl.cpp.
In the generated source file make two changes:
- In the function wrapupInvoke_xmlConversion() change the line
jobject oOut = (*((*out)->env))->NewLocalRef((*out)->env, (*out)->ref);
to
jobject oOut = ((*out)->env)->NewLocalRef((*out)->ref);
- Change the first line of the Java_spj_JSBC_1csample_cxmlConversion() function declaration to
extern “C” JNIEXPORT jobject JNICALL
Note
If you plan to use plain C only, leave the file extension at .c and don’t make the two changes above.
Now create the directory sapbc48ServerpackagesSPJ_CServicecodelibs. If a C compiler is installed on the SAP Business Connector server host, you can create the necessary shared library by running the csample.mak file — for example on Windows use nmake –f csample.mak. Afterwards, restart SAP Business Connector once. Now you can execute your spj.csample:xmlConversion service without error (although it doesn’t do anything yet).
To be able to debug the service later, use an IDE instead of the make file to create the shared library. I use Microsoft Visual Studio 2005 SP1. Linux/Unix users should adjust the settings accordingly.
If the SAP Business Connector server is running on a different host from the one you are using for development, copy the file xmlConversionImpl.cpp to your host (using your code control system’s check-in/check-out process, if you are using code control). Start Visual Studio and follow menu path File > New > Project From Existing Code. The first screen should look like
Figure 4, which shows an example of development carried out directly on the host where the SAP Business Connector server is running.
Figure 4
Settings for the Visual Studio project
Click the Next button. On the next screen choose Dynamically linked library (DLL) project. Click the Next button and enter the include search paths as follows (adjust accordingly if SAP Business Connector is not installed on the D: drive):
- D:sapbc48Serverjvminclude
- D:sapbc48Serverjvmincludewin32
- D:sapbc48Serverlib
(if compiling on the host where the SAP Business Connector server is running)
- D:sapbc48Developerjvminclude
- D:sapbc48Developerjvmincludewin32
- D:sapbc48Developersdkcsrc
(if compiling on a host where only the SAP Business Connector developer tool is installed)
Click the Finish button and then add the xmlConversionImpl.cpp file to the project. Now you are almost done. You only need to tell the linker the location of one helper library required by all C/C++ Services.
Open the Properties pane of the project and under Linker > General set the Additional Library Directories field to D:sapbc48Serverlib (or to D:sapbc48Developersdkclib, if compiling on a machine where only the SAP Business Connector developer is installed). Then under Linker > Input add win_wmJNI.lib as an Additional Dependency. (Replace the prefix win_ with win64_, lnx_, or lnx64_ if compiling for Windows 64bit or one of the Linux platforms.) Finally under Linker > General, change the Output File name from $(ProjectName).dll to spj_csample.dll.
Now you can build the shared library using Visual Studio. Whenever you want to test your changes on the server, you need to perform the following procedure. It requires you to restart SAP Business Connector, making the development process for C/C++ services somewhat awkward compared to developing Java services. However, this procedure allows you to test the effects of your changes immediately and thus facilitates a kind of incremental development process.
- Build the DLL
- Shut down SAP Business Connector
- Copy the file SPJ_CServicecodesourcespjDebugspj_csample.dll into the directory SPJ_CServicecodelibs, overwriting the previous version
- Start SAP Business Connector again
Implement the Service
Now open the source file xmlConversionImpl.cpp in Visual Studio. It contains a number of internal helper functions and the function extern “C” JNIEXPORT jobject JNICALL Java_spj_JSBC_1csample_cxmlConversion(). This is the one in which you can implement the service’s logic.
First, you need to access the byte array passed in from Java. This step is a bit tricky. The above function contains a variable WmRecord* in, which corresponds to the com.wm.util.Values object passed from Java side. However, the WmRecord API defined in wmJNI.h and documented in D:sapbc48DeveloperdocapiCindex.html (which operates on these WmRecord objects) supports only string-based data (strings, string arrays, string tables, and records, whose leaves are strings). Fortunately the original jobject that corresponds to the Values object on Java side is still available in that function, so you can use some standard JNI techniques to extract the byte array. After you access the byte array from Java side, you write it into a file as a quick check to see whether everything worked OK. For implementing this logic up to here, just change the function Java_spj_JSBC_1csample_cxmlConversion() as indicated in
Figure 5.
1 /********** Declare your variables here ************/
2 /***************************************************/
3 static jmethodID mGet = NULL;
4 jbyteArray inArray;
5 char* buf = NULL;
6 FILE* outFile = NULL;
7 jsize length = 0;
8
9//... part omitted ...
10
11 /****** Add your custom service logic here *********/
12 /***************************************************/
13
14 /* [access inputs] */
15 if (mGet == NULL){
16 jclass cValues = env->FindClass("com/wm/util/Values");
17 if (cValues == NULL){
18 out = throwWmServiceErrorMsg(&con,
19 "Unable to initialize Values class reference");
20 goto end;
21 }
22
23 mGet = env->GetMethodID(cValues, "get",
24 "(Ljava/lang/String;)Ljava/lang/Object;");
25 if (mGet == NULL){
26 out = throwWmServiceErrorMsg(&con,
27 "Unable to initialize Values class get methodID");
28 goto end;
29 }
29 }
30
31 inArray = (jbyteArray)env->CallObjectMethod(oIn, mGet, Input_xmlIn);
32 if (inArray == NULL){
33 out = throwWmServiceErrorMsg(&con, "Unable to get 'xmlIn' array");
34 goto end;
35 }
36 length = env->GetArrayLength(inArray);
37
38 outFile = fopen("testresult.txt", "wbS");
39 if (outFile == NULL){
40 out = throwWmServiceErrorMsg(&con, _strerror(NULL));
41 goto end;
42 }
43
44 buf = (char*)(env->GetByteArrayElements(inArray, NULL));
45 if (buf == NULL){
46 out = throwWmServiceErrorMsg(&con, "Unable to get 'xmlIn' contents");
47 goto end;
48 }
49
50 /* [process inputs here...] */
51 if (fwrite(buf, sizeof(char), (size_t)length, outFile) != length)
52 out = throwWmServiceErrorMsg(&con, _strerror(NULL));
53
54 /* [set outputs] */
55
56 /* [cleanup] */
57 end:
58 if (outFile) fclose(outFile);
59 if (buf) env->ReleaseByteArrayElements(inArray, (jbyte*)buf, JNI_ABORT);
60
61 /***************************************************/
62 /****** End of custom service logic ****************/
63 /***************************************************/
64 return wrapupInvoke_xmlConversion(&con, &in, &out, &session);
|
Figure 5 |
How to access a byte array in a C/C++ service |
Note
If you plan to use plain C only, in the sample code shown in Figure 5 you need to change all appearances of env->SomeFunction(/* some arguments */); to (*env)->SomeFunction(env, /* some arguments */);
After you have recompiled the library and restarted SAP Business Connector, you can test the C/C++ service from a Flow service, such as the one in
Figure 6. Define one input parameter named string and convert it to a byte array using pub.string:stringToBytes before invoking the xmlConversion service. When you execute the test service, it should write the input string into the file sapbc48Servertestresult.txt.
Figure 6
Wrapper Service for testing the xmlConversion Service
Now that the C/C++ service and the transmission of the input byte array work as expected, you need to implement the actual functionality. The algorithm is a bit complex, so I won’t go into the details. It uses fast pointer access to scan the input data and transform it into the desired output format. You can see the code in the download file SPJ_CService_SolutioncodesourcespjxmlConversionImpl.cpp. Copy this file to the corresponding location in your SPJ_CService Package, rename the function Java_spj_JSBC_1csolution_cxmlConversion() to Java_spj_JSBC_1csample_cxmlConversion(), recompile, and restart as usual. The finished service should now be operational.
Note two improvements I added compared to the test version of the C/C++ service:
- I moved the code for initializing the jmethodID for the Java method com.wm.util.Values.get() (and put() as well) into the function init_xmlConversion().
- I replaced GetByteArrayElements() with GetPrimitiveArrayCritical(). In most JVM implementations, GetByteArrayElements() creates a temporary copy of the original array. This can take up quite a bit of memory if the processed XML documents are large. GetPrimitiveArrayCritical() usually returns a direct pointer to the original array on the Java heap. However, because it temporarily turns off garbage collection, you cannot use it if any potentially blocking code is executed, before the critical pointer is released again. Therefore, I could not use GetPrimitiveArrayCritical() in the test version (Figure 5), because the fwrite() call in line 51 could potentially block, causing a deadlock of the entire Java Virtual Machine (JVM). The final version performs only byte shifting between the Get… and Release… calls, so this is safe.
A third alternative you can use — if you are processing huge byte arrays, but need to execute blocking code while processing the array — is to copy the byte array over to C block-wise using a fixed sized buffer and GetByteArrayRegion() in a loop. This avoids the temporary copy of the entire byte array and allows the system to execute blocking code while processing the byte array. Nevertheless, it has the disadvantage that many calls from C to Java need to be made, slowing everything down. (This is the usual tradeoff between execution time and memory consumption.) The writeFile service that I’ll demonstrate in the last section “Add More C/C++ Services to the Shared Library,” follows this strategy so you can save larger files without too much additional memory consumption.
Debug the Service
If you add a step bytesToString to the callXmlConversion service and then run it with a small input such as the one shown in
Figure 1, you can see that the output is not quite correct: a bunch of zero characters is added at the beginning of the output. This is a mistake I actually committed when first implementing the xmlConversion service and I debugged it as follows.
I assume that so far you have compiled the spj_csample.dll in debug mode. If not, do so and replace the DLL as described above. Then start Visual Studio, open the xmlConversionImpl.cpp file, and set a breakpoint at the beginning of the function Java_spj_JSBC_1csample_cxmlConversion(). Then follow menu path Debug > Attach to Process… and select the BCStarter.exe process.
Note
If the SAP Business Connector server runs on a different host, you need to install the Microsoft Remote Debugging Tool first, but basically it works the same. Linux/Unix users need to log in to the SAP Business Connector server host and attach gdb or a similar debugger to the Java process.
Then log in to SAP Business Connector with the developer and start the callXmlConversion service. Visual Studio should now appear and stop at the breakpoint. When you step through the code and get to the line
outBuffer->insert(outBuffer->end(), temp, temp+strlen(temp));
you should see that the variable outBuffer surprisingly is already filled up to the end with zeros. Apparently the vector(int size) constructor does not create an empty vector with a capacity of size, as I assumed, but a vector filled with size zeroes. After realizing this, the bug fix is clear: stop debugging and replace the line
outBuffer = new std::vector<char>(length);
with the following two lines:
outBuffer = new std::vector<char>();
outBuffer->reserve(length);
Recompile the library and restart SAP Business Connector as usual. The service should work now.
Performance Comparison Between the Java and the C++ Implementations
Now you should check for any performance improvements that resulted from the work you have done so far. For performance measurement, use the service spj.test:performanceComparison, which is contained in SPJ_CService_Solution. The service loads a test file and then runs it either through the Java implementation or through the C++ implementation. On my 32bit Windows XP machine with 1GB RAM and a 1.86GHz Pentium the processing of the 42MB sample file, packagesSPJ_CService_Solutionpubtestdata.xml, took 37s with the Java service and 1.5s with the C++ service.
I had started SAP Business Connector with JAVA_MIN_MEM = JAVA_MAX_MEM = 700M to prevent swapping. If the process starts swapping, the numbers become even more impressive. Here you can see the most important benefit of the C++ implementation: the Java service needs much more temporary memory and consequently it starts swapping much earlier, leading to a dramatic increase in processing time. For example, when starting SAP Business Connector with 1100M Java memory and trying to process a 63MB file, the C++ service takes between 2s and 11s and the Java service between 400s and 1000s, most of which the CPU was idle because the machine was swapping like crazy. (The large variation probably depends on what other processes currently consume resident RAM and consequently how big a portion of the BCStarter.exe process is swapped out into the pagefile.) When trying to process a 73MB file, the Java service went out of memory after 1:10h, whereas the C++ service was still able to process a 161MB file successfully in 33s.
Admittedly, the xmlConversion_Java service is doing a bit more than the xmlConversion Service. The Java service detects any syntax error in the XML input, whereas the C++ implementation does not detect errors unless they completely mess up the XML structure. However, this example serves only as an illustration of the potential that a C/C++ service can offer you. If you know for certain that you will only receive valid XML input, the missing syntax check is not a problem. If you need to be prepared for invalid input, you can easily add a syntax check to the implementation and it will still be faster than the Java implementation.
This is a very common scenario of how a C/C++ implementation can improve performance. Often in C/C++ you can reuse existing memory more easily than in Java. For example, suppose your task is to iterate recursively through a directory and all its subdirectories and sum up the file sizes of all files contained in these directories. In Java you need to create one java.io.File object for each file and directory because the File object is immutable. This means you need to create hundreds of thousands of temporary objects, all of which need to be cleaned up again later by the garbage collector.
In an analogous implementation in C, I needed only one single structure of type WIN32_FIND_DATA (or dirent on Linux). I passed this structure on in the recursion and successively filled the details of all files and directories into it using the OS function FindNextFile() (or readdir_r() on Linux). The performance improvement achieved by this was incredible. For a large directory containing a few hundred thousand files, the processing time went down from around 10s using the Java implementation to less than a second using the C implementation, so more than a factor 10 faster. The temporary memory consumption went down from a few hundred thousand File objects to just one single C structure.
Add More C/C++ Services to the Shared Library
Here are a few more tips on how to set up your make environment for multiple C/C++ services.
Adding another C/C++ service in the same folder as the first one is not difficult — just use the developer to create it like the first one. SAP Business Connector adds another entry to the existing Java stub (instead of creating a new one) and creates the other two files (the make file and the C file) as usual.
As an example, I add the service helloWorld to the csample folder. After creating the service, delete the helloWorld.mak file (it won’t be needed) and copy the helloWorld.c file as usual. Open the csample.mak file and add the file name helloWorldImpl.cpp to the SRC line. Then open the cpp file in your Visual Studio project, make the two changes necessary for compiling with C++ (see the section “Create the C/C++ Service”), and in the function Java_spj_JSBC_1csample_chelloWorld() find the line /* [set outputs] */. Change the string response in the following line to “Hello World!” After compiling and exchanging the spj_csample.dll as usual, the service already works as expected.
However, adding a service in a different folder is a bit trickier. Let’s walk through this in a bit more detail. First create a folder named io in the csample folder. Create a specification with two inputs, a string filename and an object bytes. Use this specification to create a C/C++ service writeFile inside the io folder. As usual, SAP Business Connector creates the three files JSBC_io.java, writeFile.mak and writeFile.c in the directory SPJ_CServicesourcespjcsample.
I’ll first try my previous approach: delete the .mak file, add an entry csamplewriteFileImpl.cpp to the SRC parameter of the csample.mak file in the parent directory, and copy the writeFile.c file to a new file named writeFileImpl.cpp. You can copy the source code for this service from the SPJ_CService_Solution package. Don’t forget to rename the implementing function from …_csolution_... to …_csample_... Rebuild and replace the spj_csample.dll as before and try to execute the writeFile service. You will receive the error message “Link Error: cwriteFile”, and the server.log will contain the error message [B2BSERV.0028.9999] no spj_csample_io in java.library.path.
Aha! Apparently SAP Business Connector is looking for the C function that corresponds to the writeFile service in a library called spj_csample_io.dll, but you put it into the spj_csample.dll! How can you tell SAP Business Connector to use your library instead? Now comes time to take a look at the Java stub corresponding to our C/C++ service (
Figure 7).
Figure 7
Java stub corresponding to a C/C++ Service
In the Shared tab, you can see some static code that tries to load the JNI library spj_csample_io. You don’t need that, so delete the entire static block (don’t delete the method native static Values cwriteFile(), though), save the code, and restart SAP Business Connector. Then the service should work, right? Wrong. You still get the same link error.
Now try the following: execute the helloWorld service once and then execute writeFile again. As if by magic, it works this time. The explanation is not difficult — the Java stub JSBC_csample.java has a similar static block as the one you just deleted in JSBC_io.java. The system executes this Java stub when the class JSBC_csample is loaded, which is when one of the methods in JSBC_csample.class is invoked for the first time. When the Java method corresponding to the C function for the helloWorld service was invoked, the static block loaded the spj_csample.dll, and from that point on, all three C functions in that DLL (including the one for the writeFile service) are known to the JVM.
Using this knowledge, you can now devise a simple mechanism that ensures that the writeFile service (and all other C/C++ services in subfolders of the csample folder) are operational right from the start. In the csample folder, create a new Java service named startup. That service does not need to do anything. Then mark the SPJ_CService Package and in the Startup/Shutdown/Replication Services tab define spj.csample:startup as a startup service (
Figure 8). (You may need to refresh the view before that service becomes visible in the Available services screen.)
Figure 8
Define spj.csample:startup as a start up service for SPJ_CService
From now on, whenever SAP Business Connector starts, it executes that startup service, which means the JVM needs to load the Java class spj.JSBC_csample.class first. The loading of this class triggers execution of the static block contained in that class, which in turn loads the JNI library spj_csample.dll. Consequently, all C functions in that DLL are available to the JVM, before anybody can try to execute a service in the SPJ_CService Package (in particular the writeFile service).
Ulrich Schmidt
Ulrich Schmidt joined SAP in 1998 after working in the field of computational algebra at the Department of Mathematics, University of Heidelberg. Initially, he was involved in the development of various products used for the communication between SAP R/3 systems and external components. These products include the SAP Business Connector, which translates SAP’s own communications protocol RFC into the standard Internet communications protocols HTTP, HTTPS, FTP, and SMTP, as well as pure RFC-based tools, such as the SAP Java Connector and RFC SDK. Ulrich gained insight into the requirements of real-world communications scenarios by assisting in the setup and maintenance of various customer projects using the above products for RFC and IDoc communications.
You may contact the author at
u.schmidt@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the
editor.