Thursday, October 11, 2012

OSB alert and report activities

I normally use "alert" to trace my message flows. Occasionally, I use "report".

Here is my personal view of pros and cons.

with "alert", you can drill down to the detail in 2 clicks. Once on the main "home" or "pipeline alerts" tab on the main console, then click on the actual alert. However, the "alert" title can't show any variable values. If you have 20 calls coming in, you have an "alert" that shows "id", then the subject of all those alerts will all show "id". You have to drill down to see the detail.

With report, you can put a key value, and report main page will show the entry with the actual value of the "id" from your payload. The good thing is you can see multiple entries with different ID's. However, in order to see the detail, you have to drill it twice more. Click on the "id" entry, then click on the "details" to see more.

Here is an example of my report entry:

I have a payload body looks like:


<ins:CisPidsRtCollection xmlns:ins="http://xmlns.oracle.com/pcbpel/adapter/db/top/insertRTPids">
     <ins:CisPidsRt>
        <ins:pid>123</ins:pid>
    </ins:CisPidsRt>
</ins:CisPidsRtCollection>

in my report, I can add expression like $body, then above payload will show in the "detail" link. I can also add a key "pid". Look closely, "pid" is an actual element inside the payload, then in the value section, i put in path as "ins:CisPidsRtCollection/ins:CisPidsRt/ins:pid", and variable as "body", then the report entry will show "pid=123".

OSB "publish" activity

Long story short, here is how I made my "publish" to work.

Assign "body" with this


<soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <myBizPayload xmlns="blah">
        <blah>and</blah>
    <myBizPayload >
</soapenv:Body>

where <myBizPayload>...</myBizPayload> is the actual payload when you invoke your business service. You can find the biz service payload by popping up the OSB debug console on the Biz service, it will fill in a default payload for you.

Make sure you include soap <Body> as the wrapper, no more and no less. If you assign the wrong payload, OSB won't spit out a clue anywhere, it simply ignores your call and move on to the next activity!!!

Keep in mind that all variables inside "publish" block are local. You can make changes to any variable: body, inbound, outbound etc, they are all local copies. Any changes you make do not affect the variables outside "publish".

It's simple but very important to remember, you "publish" with the "body" variable. It beats why Oracle (or BEA doc for that matter) doesn't mention that anywhere. I had to figure out it trial and errors for hours: twice and a few years apart. Hence, this post. And I don't want to relearn this again.

Thursday, September 27, 2012

Adding custom java classes to SOA

1. go to SOA ext folder, in my case: C:\Oracle\Middleware\Oracle_SOA1\soa\modules\oracle.soa.ext_11.1.1

2. you have two ways to do it:

2.1 explode your jars under "classes" sub folder here
2.2 copy the jar files in this folder, then in command prompt, run "ant" (this folder should have a default build.xml file), it basically does some magic and your jars will be included in SOA. As for the "magic", I believe it simply updates oracle.soa.ext.jar file and modified the "Manifiest" under "META-INF", set something like "Class-Path: yourCustomer.jar classes/"

Keep in mind, that push to shove, you may also have to add the same jar to  "$DOMAIN_DIR/lib", that you  just drop the file in.

I experienced problem before that merely putting jar in "racle_SOA1\soa\modules\oracle.soa.ext_11.1.1" works fine for SOA run time (with your BPEL referencing the class in the jar). However, if you don't copy the same file under "$DOMAIN_DIR/lib", you cannot re-deploy your BPEL composite that referencing the custom class. Because BPELCompiler, 2nd pass compilation (during the composite deployment),  seems to require the jar to be in "$DOMAIN_DIR/lib" folder. Go figure...

Friday, September 21, 2012

File and FTP adatper


Working with Oracle File and FTP adapter is definitely like black magic! I have SOA 11.1.1.6.0. I need to use FTP adapter to copy a file from a remote server. Both FTP and File adapters are designed to read a file, not to copy a file. If you want to copy a file and not process it, you have to jump through the hoops.

I sifted through 3 versions  of "Oracle® Fusion Middleware, User's Guide for Technology Adapters, 11g Release" (11.1.1.6.0, 11.1.1.6.2 and 11.1.1.6.3). Part of the document is inaccurate to say the least.I even tried to look up the adapter java classes (Oracle_OSB1\lib\external\adapters\ftpAdapter.jar) looking for clues. In the end, I resorted to trial and error, and with the help of network sniffing to sort out the mess.

The main challenge is to follow the instruction in section "4.5.11.5 Moving a File from One Remote Directory to Another Remote Directory on the Same FTP Server". Here are the instructions with my comments:

1. Create an empty BPEL process 
I created a synchronous BPEL, selected all default.
2. Drag and drop FTP Adapter from the Component Palette to the External
References swim lane. The Adapter Configuration Wizard Welcome page is displayed.
3. Click Next. The Service Name page is displayed.
4. Enter a service name in the Service Name field.
5. Click Next. The Adapter Interface page is displayed.- 
Default "Define from operation and schema (specified later) is checked.
6. Click Next. The FTP Server Connection page is displayed.
7. Enter the JNDI name for the FTP server, and click Next. The Operation page is
displayed. – 
JDev default might be "eis/ftp/FtpAdapter", however, SOA server default may be eis/Ftp/FtpAdapter.Look closely, you may need to change the 2nd part from "ftp" to "Ftp" (upper case "F").
8. Select Synchronous Get File, enter FTPMove in the Operation Name field, and
then click Next. The File Directories page is displayed. – 
do not overlook that "FTPMove". This is part of the black magic, it has to be precisely spelled like that, or you will be doomed. Why can't there be a drop down selection of "FTPMove"? If you have to enter specific text, what's the point of using an IDE.
9. Enter a dummy physical path for the directory for incoming files, and then click Next. The File name page is displayed.
Note: The dummy directory is not used. You must manually change the directory in a later step.
This is misleading. You can put real path in here, and the adapter uses it if you don’t overwrite it dynamically in BPEL code. Same applies to the other parameters.

10. Enter a dummy file name, and then click Next. The File Name page is displayed.
11. Click Next. The Messages page is displayed.
12. Select Native format translation is not required (Schema is opaque), and then
click Next. The Finish page is displayed.
13. Click Finish. The outbound Oracle File Adapter is now configured.
14. Drag the small triangle in the BPEL process in the Components area to the drop
zone that appears as a green triangle in FTPMove in the External References area.
The BPEL component is connected to the Oracle FTP Adapter outbound service.
15. Click File, Save All.
16. Create an invoke activity for the FTPMove service that you just created.
The next step is to modify the generated WSDL file for FTPMove service and
configure it with the new interaction specification for the move operation.
17. Open the FTPMove_ftp.jca file and modify the interaction-spec, as shown
in the following example.
You must configure the JCA file with the source and target directory and file details. You can either hardcode the source and target directory and file details in the JCA file or use header variables to populate them. In this example, header variables are used.
<adapter-config name="FTPMove" adapter="Ftp Adapter"
xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
<connection-factory location="eis/Ftp/FtpAdapter" adapterRef=""/>
<endpoint-interaction portType="FTPMove_ptt" operation="FTPMove">
<interaction-spec
className="oracle.tip.adapter.ftp.outbound.FTPIoInteractionSpec">
<property name="SourcePhysicalDirectory" value="foo1"/>
<property name="SourceFileName" value="bar1"/>
<property name="TargetPhysicalDirectory" value="foo2"/>
<property name="TargetFileName" value="bar2"/>
<property name="Type" value="MOVE"/>
</interaction-spec>
</endpoint-interaction>
</adapter-config>

18. Map the actual directory and file names to the source and target file parameters by performing the following procedure:
a. Create 4 string variables with appropriate names. You must populate these variables with the source and target directory details. The BPEL source view shows you this:
<variable name="sourceDirectory" type="xsd:string"/>
<variable name="sourceFileName" type="xsd:string"/>
<variable name="targetDirectory" type="xsd:string"/>
<variable name="targetFileName" type="xsd:string"/>

b. Create an assign activity to assign values to sourceDirectory,
sourceFileName, targetDirectory, and targetFileName variables.
The assign operation appears in the BPEL source view as in the following example:
 I use BPEL2.0, my assignment looks like:
   <assign name="Assign1">
      <copy>
        <from>'/test/input'</from>
        <to>$sourceDirectory</to>
      </copy>
      <copy>
        <from>'input.txt'</from>
        <to>$sourceFileName</to>
      </copy>
      <copy>
        <from>'/test/output</from>
        <to>$targetDirectory</to>
      </copy>
      <copy>
        <from>'output.txt'</from>
        <to>$targetFileName</to>
      </copy>
</assign>
c. Pass these parameters as headers to the invoke operation. The values in these
variables override the parameters in the JCA file.

The code snippet has apparent errors, here is my code in BPEL2.0 format:
    <invoke name="Invoke" partnerLink="mvFile" portType="ns1:FTPMove_ptt"
            operation="FTPMove" inputVariable="Invoke_FTPMove_InputVariable"
            outputVariable="Invoke_FTPMove_OutputVariable"
            bpelx:invokeAsDetail="no">
      <bpelx:toProperties>
        <bpelx:toProperty name="jca.file.SourceDirectory"
                          variable="sourceDirectory"/>
        <bpelx:toProperty name="jca.file.SourceFileName"
                          variable="sourceFileName"/>
        <bpelx:toProperty name="jca.file.TargetDirectory"
                          variable="targetDirectory"/>
        <bpelx:toProperty name="jca.file.TargetFileName"
                          variable="targetFileName"/>
      </bpelx:toProperties>
    </invoke>

A few notes here, do NOT use the UI and select the “properties”, these properties do not show up in the list, AND the properties show up in the list do NOT work L (for example, jca.ftp.FileName would look like a perfect choice, but it is misleading, it does not work!).
Additionally, there are “To” and “From” properties I can’t find any documents about these choices.  I finally figured out (network sniffing) that “to” appears to be what used to be called “input” properties. I would assume “from” would be the same as old “output” properties.


19. Finally, add an initial receive or pick activity.
You have completed moving or renaming a file from a remote directory to another remote directory on the same FTP server.
If you created a default BPEL, you may already have your receive activity. Run your test, that should take move “/test/input/input.txt” to “/test/output/output.txt” on the same FTP server.

If you want to move the file from remote “/test/input/input.txt” to local “/test/output/output.txt”, you need to add this property to the jac file:

<property name="TargetIsRemote" value="false"/>

Wednesday, September 12, 2012

Extracting and Passing WSS Name Token Security Header in BPEL

How to extract and insert SOAP headers in BPEL 2.0 is documented in the Oracle doc here http://docs.oracle.com/cd/E12839_01/integration.1111/e10224/bp_manipdoc.htm#SOASE415 However, like all Oracle docs, it takes some hands on finessing to figure out the specifics.

Additionally, for WSS security tokens, I have to dance around the WSS security XSD files in order to get it to work. It may not be the best way, but it seems to work.

Let's look at how to extract SOAP headers in BPEL. As Oracle document indicates, it takes two steps. You need to be very precise with each step, otherwise, you won't see the headers inside BPEL.

Step I -  update the WSDL - the Oracle document provides an excellent sample with multiple headers, it just needs some more detailed descriptions. Let me copy the sample here:
<!-- custom header -->
   <message name="CustomHeaderMessage">
    <part name="header1" element="tns:header1"/>
    <part name="header2" element="tns:header2"/>
  </message>
  <binding name="HeaderServiceBinding" type="tns:HeaderService">
    <soap:binding style="document"
      transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="initiate">
      <soap:operation style="document" soapAction="initiate"/>
        <input>
          <soap:header message="tns:CustomHeaderMessage"
            part="header1" use="literal"/>
          <soap:header message="tns:CustomHeaderMessage"
            part="header2" use="literal"/>
          <soap:body use="literal"/>
        </input>
    </operation>
  </binding>

Step 1.1 - define the messages to enclose the header elements to be passed in SOAP header. 
Notice that in the <!-- custom header --> section, The CustomHeaderMessage has two parts.  In order to pass in multiple SOAP headers, each SOAP header needs to be included as a part in this message.

The part names are "part1" and "part2". Although not critical, in the real world, you should give them some more meaningful part names.

Although obvious, it is still worth noting that these elements need be defined somewhere or imported from somewhere. For example, in your schema "header1" may hypothetically defined as:

<element name="header1">
<complexType>
<sequence>
<element name="fristName" type="string"/>
<element name="lastName" type="string"/>
</sequence>
</complexType>
</element>
In the case of WSS security header, I have to import a few related XSD's - see details below.

Step 1.2 - update the operation in the binding section

The Oracle sample may not look like the typical BPEL WSDL, so here is the changes I made to my WSDL:

<wsdl:operation name="process">
<wsdl:input  message="client:fooBPELRequestMessage">
<soap:header message="client:securityHeaderMsg"  part="payload" use="literal"/>
<soap:header message="client:fooBPELResponseMessage"  part="payload" use="literal"/>
</wsdl:input>
<wsdl:output message="client:fooBPELResponseMessage"/>
</wsdl:operation>

Here I am passing two SOAP headers. Keep in mind, very importantly, in Step 1.1, these two headers need be declared in a single message as two parts.

Step II - Receive (extract) headers in BPEL

Step 2.1 - define a message-type variable using the CustomHeaderMessage

    <variable name="customHeaderVar" messageType="client:customHeaders"/>

Step 2.2 - update receive activity
  <receive name="receiveInput" 
partnerLink="foobpel_client" 
portType="client:fooBPEL" 
operation="process" variable="inputVariable" 
createInstance="yes" 
bpelx:headerVariable="customHeaderVar"/>
There you go, customHeaderVar will contain the SOAP header values if you pass them in. That's a big "if", because you do not pass the headers in your SOAP call, or do not pass them in correctly, you still won't receive the headers, although it causes no harm to the receive activity even if you don't pass the headers in yours SOAP call.

In my case, I am only interested in the WSS security header, so I first define a "security" variable:
   <variable name="SecurityVar" element="ns2:Security"/> 
where ns2 is xmlns:ns2="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
then I pick up the WSS header:
    <assign name="AssignCustomHeader">
      <copy>
        <from>$customHeaderVar.payload1</from>
        <to>$SecurityVar</to>
      </copy>
    </assign>

Step III - passing SOAP headers in BPEL invoke activity

Finally, I can pass in the WSS security header in the invoke:

     <invoke name="InvokeBar"
                partnerLink="fooCallsBar" portType="ns1:barBPEL"
                operation="process" inputVariable="Invoke_process_InputVariable"
                outputVariable="Invoke_process_OutputVariable"
                bpelx:invokeAsDetail="no" bpelx:inputHeaderVariable="SecurityVar"/>

The WSS Security XSD Conundrum 

I couldn't effectively importing WSS security XSD remotely, it takes forever for JDEV to drill down the WSS security element structure, so I ended up copying these 3 files locally:
"oasis-200401-wss-wssecurity-secext-1.0.xsd", 
"oasis-200401-wss-wssecurity-utility-1.0.xsd" 
"xmldsig-core-schema.xsd"
I had to comment out this line in the first file, so it won't bring my house down 
<!--xsd:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="http://www.w3.org/2001/xml.xsd"/-->

Here is my entire WSDL:

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions name="fooBPEL"
             targetNamespace="http://xmlns.oracle.com/cis/foo/fooBPEL"
             xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
             xmlns:client="http://xmlns.oracle.com/cis/foo/fooBPEL"
             xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
             xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
             xmlns:plnk="http://docs.oasis-open.org/wsbpel/2.0/plnktype">
<wsdl:types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://xmlns.oracle.com/cis/foo/fooBPEL" schemaLocation="xsd/fooBPEL.xsd" />
<import namespace="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
                        schemaLocation="xsd/oasis-200401-wss-wssecurity-secext-1.0.xsd" />
</schema>
</wsdl:types>
<wsdl:message name="fooBPELRequestMessage">
<wsdl:part name="payload" element="client:process"/>
</wsdl:message>
<wsdl:message name="fooBPELResponseMessage">
<wsdl:part name="payload" element="client:processResponse"/>
</wsdl:message>
<wsdl:message name="securityHeaderMsg">
<wsdl:part name="payload" element="wsse:Security"/>
</wsdl:message>
<wsdl:message name="customHeaders">
<wsdl:part name="payload1" element="wsse:Security"/>
<wsdl:part name="payload2" element="client:processResponse"/>
</wsdl:message>
<wsdl:portType name="fooBPEL">
<wsdl:operation name="process">
<wsdl:input  message="client:fooBPELRequestMessage">
                                <soap:header message="client:securityHeaderMsg"  part="payload" use="literal"/>
                                <soap:header message="client:fooBPELResponseMessage"  part="payload" use="literal"/>
                        </wsdl:input>
<wsdl:output message="client:fooBPELResponseMessage"/>
</wsdl:operation>
</wsdl:portType>
<plnk:partnerLinkType name="fooBPEL">
<plnk:role name="fooBPELProvider" portType="client:fooBPEL"/>
</plnk:partnerLinkType>
</wsdl:definitions>

Here is the SOAP UI test payload:

soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:foob="http://xmlns.oracle.com/cis/foo/fooBPEL">
   <soapenv:Header>
    <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <wsse:UsernameToken>
         <wsse:Username>weblogic</wsse:Username>
         <wsse:Password>welcome1</wsse:Password>
      </wsse:UsernameToken>
   </wsse:Security>
   <processResponse xmlns="http://xmlns.oracle.com/cis/foo/fooBPEL">
      <result>this is my header</result>
   </processResponse>
   </soapenv:Header>
   <soapenv:Body>
      <foob:process>
         <foob:input>SOAPUI invokes foo</foob:input>
      </foob:process>
   </soapenv:Body>
</soapenv:Envelope>

Snippet of BPEL source:


         xmlns:ns2="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
  ...
  <variables>
    <variable name="inputVariable" messageType="client:fooBPELRequestMessage"/>
    <variable name="outputVariable" messageType="client:fooBPELResponseMessage"/>
    <variable name="Invoke_process_InputVariable"
              messageType="ns1:barBPELRequestMessage"/>
    <variable name="Invoke_process_OutputVariable"
              messageType="ns1:barBPELResponseMessage"/>
    <variable name="SecurityVar" element="ns2:Security"/>
  </variables>

  <sequence name="main">

    <!-- Receive input from requestor. (Note: This maps to operation defined in fooBPEL.wsdl) -->
    <receive name="receiveInput" partnerLink="foobpel_client" portType="client:fooBPEL" operation="process" variable="inputVariable"
    createInstance="yes" bpelx:headerVariable="customHeaderVar"/>
    <assign name="AssignCustomHeader">
      <copy>
        <from>$customHeaderVar.payload1</from>
        <to>$SecurityVar</to>
      </copy>
    </assign>
    <assign name="AssignBarInput">
      <copy>
        <from>"knock, knock, foo's calling"</from>
        <to>$Invoke_process_InputVariable.payload/ns1:input</to>
      </copy>
    </assign>

    <invoke name="InvokeBar"
            partnerLink="fooCallsBar" portType="ns1:barBPEL"
            operation="process" inputVariable="Invoke_process_InputVariable"
            outputVariable="Invoke_process_OutputVariable"
            bpelx:invokeAsDetail="no" bpelx:inputHeaderVariable="SecurityVar"/>






Thursday, August 30, 2012

count() vs. countNodes() in BPEL 2.0


I have wrestled with these two since 10g . However, I still mix them up when I use them in BPEL 2.0. The function signature of count() has changed since BPEL1.1.

For the record, here is the scoop on these two functions in BPEL 2.0:

I have created a simple test BPEL based on the sample schema (at the end of this post).

I tested count() and countNodes() like below:

#1 count($outputVariable.payload/client:result)

It shows 0. Please note "result" is optional based on the XSD. Additionally, please note that if you enter an invalid path, such as “$outputVariable.payload/foo/bar”, JDev won’t compile it. So count() only allows valid xpath for the variable type, check #5 of countNodes() below to see more on this.

#2 count($inputVariable.payload/client:input)

It shows 1 as expected.

#3 ora:countNodes('outputVariable', 'payload', '/client:processResponse/client:result')

It shows 0, same as case #1. However, pay close attention to two things: first argument is the variable name as a string, not “$variable” as it appears in count(). Additionally, the 3rd argument uses the full path “/client:processResponse/client:result”. It includes “client:processResponse” section.

 #4 ora:countNodes('inputVariable', 'payload', '/client:process/client:input')

     It shows 1, just like in case #2. However, pay attention that first argument is the string name of the variable. Second argument is the “part” of the message type. Third argument needs the full path like “/client:process/client:input”. In the case of count() function, it appears that it skipped the “root” element “client:process” after payload.

#5 ora:countNodes('outputVariable', 'payload', '/client:input/test_invalid_path)

Finally, this one shows 0 as expected. The point here is to show that you can enter a path that is not defined in your XSD. This may come in handy if you have a payload that does not conform to your input schema, or you simply do not know the schema beforehand.

Here is the schema, please note that I have made "result" optional:

<?xml version="1.0" encoding="UTF-8"?> 
<schema attributeFormDefault="unqualified"
            elementFormDefault="qualified"
            targetNamespace="http://xmlns.oracle.com/cis/foo/BPELProcess1"
            xmlns="http://www.w3.org/2001/XMLSchema">
            <element name="process">
                        <complexType>
                                    <sequence>
                                                <element name="input" type="string" minOccurs="0"/>
                                    </sequence>
                        </complexType>
            </element>
            <element name="processResponse">
                        <complexType>
                                    <sequence>
                                                <element name="result" type="string" minOccurs="0"/>
                                    </sequence>
                        </complexType>
            </element>
</schema>

Sunday, August 26, 2012

Search and Replace in Composite Configuration Plan File

SOA 11g provides a configuration plan file for composite deployment. A common use is to search and replace hosts and ports depends on the platform (dev, test, qa, prod) you are deploying to. The question is when do those search / replacement take place?

If you wonder why do I ask this seemingly obscured question, let's take another look at it.

Suppose you do development on server "soaDEV.mybusiness.com", and SOA runs on port 8001. Then you need to migrate your work to "soaPRD.mybusiness.com", and SOA runs on port 80 over there.

So your plan file may contain something like:

  <search>soaDEV.mybusiness.com:8011</search>
  <replace>soaPRD.mybusiness.com:80</replace>

When you click on "Deploy" and pick your configuration plan file, a common misconception is that "soaDEV.mybusiness.com:8001" will be magically replaced, and "soaDEV.mybusiness.com" is almost irrelevant. However, if you shut down "soaDEV" server, you will notice that JDEV will fail the compilation.


When you click "Deploy" in JDev, it's actually a two-step process. First Jdev generates the jar (sar) file. Then it uploads (deploys) the jar to the SOA server (you can actually do these two steps manually. First compile / generate the jar file. Then go to EM console to deploy the composite.)

The search / replacement only takes place during the 2nd stage. When SOA server receives the composite jar file, it does a second compilation using BPEL Compiler. So when JDEV compiles the BPEL source code, it still requires your original "soaDEV.mybusiness.com" to be up and running. It still needs to reference the WSDL from port 8001on "soaDEV".

Now we know the facts, I still have this question that bugs me all the time. Why can't (won't) JDEV do search and replacement before the compilation? I don't truly know the answer. I'm sure there is a fundamental answer to this question.

I always have this eerie feeling that 9 months after you have finished your production deployment, you still need to keep your soaDEV server running in order to support your soaPRD environment. Because Jdev needs to reference soaDEV in order to compile. What about the potential inconsistency between soaDEV and soaPRD 9 months after or even longer?