Monday, December 10, 2012

Stuck: cannot undeploy SOA composite

You ever run into the problem that you cannot redeploy or even undeploy an SOA composite? It can be frustrating. The most common reason is that the composite references an URL doesn't exist anymore. I think it's almost "stupid" to require that a composite referenced URL to exist when you need to undeploy a composite, come on, i don't even want the composite anymore, why do i care if it references a non-existent URL.

Anyway, here is a sample scenario how you might get stuck, and how you can try to un-stuck yourself.


1. an previously deployed SOA composite references an URL, say on OSB, http://localhost:8011/blah, Now that URL doesn't exist on osb anymore.

2. When you try to redeploy the composite from jdev, it won't re-deploy, complains http://localhost:8011/ext/blah not exist

3. try un-deploy bpel from EM console, get the same error, so BPEL needs 8011/blah to exist even for straight un-deploy or retire

4. fix: go to osb to temporarily (remember to change it back after!!!) enable 8011/blah, then un-deploy BPEL from EM console

5. deploy BPEL from JDEV

6. if it still doesn't work, switch back the temporary change on OSB, and try to deploy to a new composite version, such as 2.0.

Experiment JSON with OSB


Honestly, I never used json, other than hearing about it all the time. I am creating a Rest service with OSB, my colleague would like me to send the response back as json. So game is on. After trial and error, here is how I did it.
1.      Create a proxy service, select “Message Service”.
2.      For request message Type, you have multiple choices. Select “XML”. (if you want post up free text, then select “Text”)
3.      For response message type, select “Text”
4.      For transport, select “http” protocol
5.      Keep the default for the rest
Message flow:
1.      Process input:
Pick up your input XML from $body. Here is a quick note on parsing the $body, say you have payload like this:
<soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <foo xmlns="foo">
    <bar xmlns="bar">car</bar>
  </foo>
</soapenv:Body>

To get “car”, use this xpath: $body/f:foo/b:bar/text()
Send Json response back:
[
    {{
        "PREM_ID":"111111",
        "PREM_TYPE_CD":"HOUSE",
        "ADDRESS1":"2222 Cypress Ave",
        "CITY":"Mountain Mesa",
        "POSTAL":"12345",
        "DESCR":"House",
        "operationArea":"KRV",
        "district":"MMS",
        "isMultiple":"N",
        "allowSelfService":"Y",
        "needAppointment":"N"
    }}
]

The only special thing I did was to escape “{“ with “{{“, and “}” with “}}”.
That’s it.



Monday, November 26, 2012

more notes on java embedding

I noted in the past on BPEL java embedding http://yuanmengblog.blogspot.com/2012/04/few-bpel-java-embedding-notes.html

here are some additional notes:

1. importing syntax:

I'm using BPEL 2.0 this time. This is the importing syntax I'm using now, not sure if it's BPEL 2.0 thing. For sure it's different from my previous post:

add imports right above partner links and right below name spaces.


  <import location="oracle.xml.parser.v2.XMLElement" importType="http://schemas.oracle.com/bpel/extension/java"/>      
  <import location="java.io.File" importType="http://schemas.oracle.com/bpel/extension/java"/>      

2. Two ways to get and set variables:

2.1. getting and setting string variables:
   nothing fancy, you can do straight:
    String foo = (String) getVariableData("foo");
    setVariableData("foo", "bar");

2.2. If you need to get to an Xpath in an element, then you have to do something like

XMLElement srcElem = (XMLElement) getVariableData("inputVariable", "payload", "/client:foo/client:bar");      
String bar = srcElem.getTextContent();      

remember you need to add "import" like in step 1. 

3. Gotcha's

Although it's very tempting, don't do this:
   String srcElem = (String) getVariableData("inputVariable", "payload", "/client:foo/client:bar");   
it doesn't work that way.

If you don't want to deal with the hassles of "importing" and casting, here is a recommended way to go around it. 
Using above example:
 a) create a BPEL string variable, call it "bar"
 b) inside BPEL, assign ("inputVariable", "payload", "/client:foo/client:bar") to the "bar" variable
 c) inside your java embedding, you can use get/set freely on "bar", just like in step 2.1 above.

Wednesday, November 21, 2012

"Teach" JDEV where to look for MDS artifacts

A common task with JDEV is to tell it where to find the MDS connection. If JDEV fails to find MDS, then you may get errors like:

  • Error: Error in getting XML input stream: oramds:/apps/xsd/foo.xsd: oracle.mds.exception.MDSException: MDS-00054: The file to be loaded oramds:/apps/xsd/foo.xsd does not exist. 
There might be many reasons for this error. One possible reason could be that JDEV does not know where to look for MDS. JDev uses ad-config.xml to look for MDS connections.


Assume you use DB based MDS, here is how I give JDev a "jolt" and make it to learn:

1. make sure you have a DB connection points to your MDS schema, create one if needed (default schema name is "DEV-MDS")
2. make sure you have your MDS connection in the resource panel, create a new MDS connection if needed and point to your DB connection above

3. finally, create a dummy SOA project, called it "junk", "dummy" or anything, create it with a BPEL process, on the BPEL process screen do these steps:
  1) click on the search glass for input schema
  2) click on search schema in "Type Chooser" popup window
  3) click on search glass in "Import Schema File" popup window
  4) select "Resource Palette" in "SOA Resource Browser" popup window
  5) find your MDS connection, and drill down to where you stored your XSD
  
pick a XSD and any element, pretend you need to use it for your "junk" project input, save your project. That should do it. We just gave JDev a "jolt", it should know where to find your missing MDS artifacts now.

 Now go back and open your adf-config.xml again, you should see the file has picked up a new entry for your MDS connection.

Thursday, November 1, 2012

OSB, http basic authentication and OWSM policy

OWSM and OSB are bundled together after-fact. So OSB doesn't support some of the security policies that comes with OWSM.

However, it is misleading that OSB allows you to attach any OWSM policy to your proxy,but  that doesn't mean it will actually work. It took me a while to find out that the simplest of all: OWSM "wss_http_token_service_policy" does not work in OSB!

However, not all is lost. OSB actually supports http basic authentication out of box without OWSM. You just need to go to "HTTP Transport Configuration" tab in your proxy configuration page, check that "basic authentication" checkbox.

Then the caller needs to add a HTTP header like:
     Authorization: Basic d2VibG9naWM6d2VsY29tZTE=
where d2VibG9naWM6d2VsY29tZTE= is base64 encoded user:password, in this example, it's "weblogic:welcome1". You are not required to encode it, but it's the common practice.

You can base64 encode your stuff online from here: http://www.base64decode.org/

that's all.

Thursday, October 25, 2012

importing OSB projects into Eclipse

Sometimes our OSB Eclipse projects get corrupted, or sometimes you need to create new workspace from existing source code (from project source code control).

Here are steps I use:

I'll assume your entire OSB source is under a folder call "c:/osb".

0. back up your osb folder (if you used it for an existing workspace), e.g. rename it osb.bak
1. create new folder "c:/osb"
2. check out your source files under "c:/osb", e.g. ("my config proj", "my proj1", "my proj2")
    or copy your backed project files if you backed them in step 0
3. fire up eclipse, switch to OSB pespective, and create a new workspace under "c:/osb"
4. select "file", "import", "general", then "existing projects into workspace"
5. pick your "c:/osb" folder, check and uncheck the projects you want
6. import them
7. it may whine about your .svn, or .vss files, you can ignore them, or you can google it up, i believe there is way to mask off those warnings in Eclipse preference, i just don't remember how
8. if you happen to see one of the projects showing error saying it doesn't have an associated OSB configuration project, just drag that project in Eclipse "project explorer" panel, and drop it into your OSB configuration project.

hope that takes care of everything.

Friday, October 19, 2012

PGP and SOA

My environment:
      SOA 11.1.1.6
      Java 1.6.x
Goal:
      sftp adapter to grab a PGP encrypted file, then decrypt the file in the SOA composite then process the data. You will be supplied with a PGP private key file (binary key ring, or ascii key), and pass phrase to decode the file.

My solution:
     use java embedding to decrypt PGP file. In essence, it's a Java solution.

Java part:

Here is how to do it in Java:
1. go to Bounce Castle to download the latest jars


Download
 bcpg-jdk15on-147.jar 
bcprov-jdk15on-147.jar 

2. online resource to use the package to encrypt/decrypt:


3. I added a simple main() to the above PGPFileProcessor class:

public static void main(String args[]) throws Exception
{
   PGPFileProcessor pgp = new PGPFileProcessor();

   //pgp.setAsciiArmored(true); // if you want dump ascii file
   // hard code for my test
   pgp.setInputFileName("c:/pgp/java/sample_file.txt");
   pgp.setOutputFileName("c:/pgp/java/sample_file.txt.pgp");
   pgp.setPublicKeyFileName("c:/pgp/java/mypgp-pub.key"); //can be either binary or text key
   pgp.encrypt();

// decrypt the same file
   pgp.setInputFileName("c:/pgp/java/sample_file.txt.pgp");
   pgp.setOutputFileName("c:/pgp/java/sample_file-decrypted.txt");
   pgp.setSecretKeyFileName("c:/pgp/java/mypgp-pri.key"); // can be either binary or text key
   pgp.setPassphrase("mypassword");
   pgp.decrypt();
}

3. set your class path to include the jars in step 1, and compile java files from step 2

4. if you get java exceptions like:
PGPKeyRingTest: exception: java.security.InvalidKeyException: Illegal key size

You may need to download

Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 6


save the jars to “C:\java\jdk1.6.0_31\jre\lib\security”. Backup local_policy.jar, and US_export_policy.jar first!


5. package the compiled classes into a jar, let's call it mypgp.jar

Add Java to the composite:

1. copy  bcpg-jdk15on-147.jar, bcprov-jdk15on-147.jar and mypgp.jar (you just generated) to two locations (you need to figure out your own soa environment)
C:\Oracle\Middleware\Oracle_SOA1\soa\modules\oracle.soa.ext_11.1.1
and
C:\Oracle\Middleware\user_projects\domains\soa_domain\lib


For the first copied location, there is build.xml in that directory already, you need to run "ant" command in that directory. If you don't want to run "ant" command, you can also unjar your files into the "classes:" sub directory.
After you finish that, you need to bounce the entire SOA suite (including weblogic server).

2. code embedded java:
            <![CDATA[System.out.println("**********new pgp");      
pgp.test.com.PGPFileProcessor pgp = new pgp.test.com.PGPFileProcessor();      
      
System.out.println("*end *********new pgp");      
     
XMLElement srcElem = (XMLElement) getVariableData("inputVariable", "payload", "/client:myReqeust/client:srcFile");    
String srcFile = srcElem.getTextContent();    
String tgtFile = srcFile.substring(0, srcFile.lastIndexOf(".pgp"));    
      
args[0] = ;      
args[1] = ;      
      
      
pgp.setInputFileName(new String("/soaIntegration/pgp/receive/") + srcFile);      
pgp.setOutputFileName(new String("/soaIntegration/pgp/process/") + tgtFile);      
pgp.setSecretKeyFileName("/soaIntegration/pgp/pri-key.key");      
pgp.setPassphrase("mypassword");      
      
try {      
pgp.decrypt();      
}      
catch(Exception e)      
{      
  System.out.println("###PGP decryption failed:"+e.getMessage());      
}]]>

Additional: generate your own key pairs for test or for fun

What if you want to generate your own key pairs for test?  Here is what I did:
download "bcpg-jdk15on-147.zip" with source, then find this file PGPKeyRingTest.java in package org.bouncycastle.openpgp.test.

I modify generatetest() like below:
    public void generateTest()         throws Exception
    {
System.out.println("********* intercepted ********");
        char[]              passPhrase = "hello".toCharArray();
        KeyPairGenerator    dsaKpg = KeyPairGenerator.getInstance("DSA", "BC");
        dsaKpg.initialize(512);
...
PGPPublicKeyRing        pubRing = keyRingGen.generatePublicKeyRing();
        PGPPublicKey            vKey = null;
        PGPPublicKey            sKey = null;

  byte[] b = null;
            ByteArrayOutputStream baos = null;
            ArmoredOutputStream aos = null;
            try {
                    baos = new ByteArrayOutputStream();
                    aos = new ArmoredOutputStream(baos);
                    // get public key
                    pubRing.encode(aos);
                    aos.flush();
                    baos.flush();
                    aos.close() ;
                    b = baos.toByteArray();
                    System.out.println(new String(b));

                    baos = new ByteArrayOutputStream();
                    aos = new ArmoredOutputStream(baos);
                    // get private key
                    keyRing.encode(aos);
                    aos.flush();
                    baos.flush();
                    aos.close() ;
                    b = baos.toByteArray();
                    System.out.println(new String(b));

            } catch (Exception e) {
                    System.out.println("Exception caught while exporting SecretKeyRing"+ e);
            } finally {
                    aos.close();
                    baos.close();
            }
of course, i modified performTest() only to run generatetest().

you need to add bctest-jdk15on-147.jar to the classpath to run the above class. 

If you get java exceptions compiling or running, check step 4 above in the Java section.

There you go, you'll get your own public and private keys to play with (remember your password is "hello" in the above code, you can change it if you like).

PGP Command Line

BTW, if you happen to have the PGP command line from old PGP Corporation (currently owned by Symatec), you can generate your own keys, and test encryption and decryption from the command line.

To generate key:
   pgp --gen-key "foo@bar.com" --key-type rsa --bits 2048 --passphrase car

export the key:
     pgp --export-key-pair foo

To test encryption/decryption:
  pgp -e test.txt --recipient foo (or pgp -e test.txt --recipient foo --armor)

  pgp --decrypt test.txt.pgp  --passphrase "car"--output foo.txt

if you wonder where pgp command line stores key ring files, they are under here on windows: C:\Documents and Settings\yourUserName\My Documents\PGP. I was using the binary key rings initially before I figured out the export commands.

That's all

I assumed anyone reading this post has the basic idea how PGP works. What I demonstrated is how to use Java to decrypt a PGP file, and add the java solution to SOA composite. 





Thursday, October 11, 2012

OSB alert and report activities

I normally use "alert" to trace my message flows. Occasionally, I use "report".

Here is my personal view of pros and cons.

with "alert", you can drill down to the detail in 2 clicks. Once on the main "home" or "pipeline alerts" tab on the main console, then click on the actual alert. However, the "alert" title can't show any variable values. If you have 20 calls coming in, you have an "alert" that shows "id", then the subject of all those alerts will all show "id". You have to drill down to see the detail.

With report, you can put a key value, and report main page will show the entry with the actual value of the "id" from your payload. The good thing is you can see multiple entries with different ID's. However, in order to see the detail, you have to drill it twice more. Click on the "id" entry, then click on the "details" to see more.

Here is an example of my report entry:

I have a payload body looks like:


<ins:CisPidsRtCollection xmlns:ins="http://xmlns.oracle.com/pcbpel/adapter/db/top/insertRTPids">
     <ins:CisPidsRt>
        <ins:pid>123</ins:pid>
    </ins:CisPidsRt>
</ins:CisPidsRtCollection>

in my report, I can add expression like $body, then above payload will show in the "detail" link. I can also add a key "pid". Look closely, "pid" is an actual element inside the payload, then in the value section, i put in path as "ins:CisPidsRtCollection/ins:CisPidsRt/ins:pid", and variable as "body", then the report entry will show "pid=123".

OSB "publish" activity

Long story short, here is how I made my "publish" to work.

Assign "body" with this


<soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <myBizPayload xmlns="blah">
        <blah>and</blah>
    <myBizPayload >
</soapenv:Body>

where <myBizPayload>...</myBizPayload> is the actual payload when you invoke your business service. You can find the biz service payload by popping up the OSB debug console on the Biz service, it will fill in a default payload for you.

Make sure you include soap <Body> as the wrapper, no more and no less. If you assign the wrong payload, OSB won't spit out a clue anywhere, it simply ignores your call and move on to the next activity!!!

Keep in mind that all variables inside "publish" block are local. You can make changes to any variable: body, inbound, outbound etc, they are all local copies. Any changes you make do not affect the variables outside "publish".

It's simple but very important to remember, you "publish" with the "body" variable. It beats why Oracle (or BEA doc for that matter) doesn't mention that anywhere. I had to figure out it trial and errors for hours: twice and a few years apart. Hence, this post. And I don't want to relearn this again.

Thursday, September 27, 2012

Adding custom java classes to SOA

1. go to SOA ext folder, in my case: C:\Oracle\Middleware\Oracle_SOA1\soa\modules\oracle.soa.ext_11.1.1

2. you have two ways to do it:

2.1 explode your jars under "classes" sub folder here
2.2 copy the jar files in this folder, then in command prompt, run "ant" (this folder should have a default build.xml file), it basically does some magic and your jars will be included in SOA. As for the "magic", I believe it simply updates oracle.soa.ext.jar file and modified the "Manifiest" under "META-INF", set something like "Class-Path: yourCustomer.jar classes/"

Keep in mind, that push to shove, you may also have to add the same jar to  "$DOMAIN_DIR/lib", that you  just drop the file in.

I experienced problem before that merely putting jar in "racle_SOA1\soa\modules\oracle.soa.ext_11.1.1" works fine for SOA run time (with your BPEL referencing the class in the jar). However, if you don't copy the same file under "$DOMAIN_DIR/lib", you cannot re-deploy your BPEL composite that referencing the custom class. Because BPELCompiler, 2nd pass compilation (during the composite deployment),  seems to require the jar to be in "$DOMAIN_DIR/lib" folder. Go figure...

Friday, September 21, 2012

File and FTP adatper


Working with Oracle File and FTP adapter is definitely like black magic! I have SOA 11.1.1.6.0. I need to use FTP adapter to copy a file from a remote server. Both FTP and File adapters are designed to read a file, not to copy a file. If you want to copy a file and not process it, you have to jump through the hoops.

I sifted through 3 versions  of "Oracle® Fusion Middleware, User's Guide for Technology Adapters, 11g Release" (11.1.1.6.0, 11.1.1.6.2 and 11.1.1.6.3). Part of the document is inaccurate to say the least.I even tried to look up the adapter java classes (Oracle_OSB1\lib\external\adapters\ftpAdapter.jar) looking for clues. In the end, I resorted to trial and error, and with the help of network sniffing to sort out the mess.

The main challenge is to follow the instruction in section "4.5.11.5 Moving a File from One Remote Directory to Another Remote Directory on the Same FTP Server". Here are the instructions with my comments:

1. Create an empty BPEL process 
I created a synchronous BPEL, selected all default.
2. Drag and drop FTP Adapter from the Component Palette to the External
References swim lane. The Adapter Configuration Wizard Welcome page is displayed.
3. Click Next. The Service Name page is displayed.
4. Enter a service name in the Service Name field.
5. Click Next. The Adapter Interface page is displayed.- 
Default "Define from operation and schema (specified later) is checked.
6. Click Next. The FTP Server Connection page is displayed.
7. Enter the JNDI name for the FTP server, and click Next. The Operation page is
displayed. – 
JDev default might be "eis/ftp/FtpAdapter", however, SOA server default may be eis/Ftp/FtpAdapter.Look closely, you may need to change the 2nd part from "ftp" to "Ftp" (upper case "F").
8. Select Synchronous Get File, enter FTPMove in the Operation Name field, and
then click Next. The File Directories page is displayed. – 
do not overlook that "FTPMove". This is part of the black magic, it has to be precisely spelled like that, or you will be doomed. Why can't there be a drop down selection of "FTPMove"? If you have to enter specific text, what's the point of using an IDE.
9. Enter a dummy physical path for the directory for incoming files, and then click Next. The File name page is displayed.
Note: The dummy directory is not used. You must manually change the directory in a later step.
This is misleading. You can put real path in here, and the adapter uses it if you don’t overwrite it dynamically in BPEL code. Same applies to the other parameters.

10. Enter a dummy file name, and then click Next. The File Name page is displayed.
11. Click Next. The Messages page is displayed.
12. Select Native format translation is not required (Schema is opaque), and then
click Next. The Finish page is displayed.
13. Click Finish. The outbound Oracle File Adapter is now configured.
14. Drag the small triangle in the BPEL process in the Components area to the drop
zone that appears as a green triangle in FTPMove in the External References area.
The BPEL component is connected to the Oracle FTP Adapter outbound service.
15. Click File, Save All.
16. Create an invoke activity for the FTPMove service that you just created.
The next step is to modify the generated WSDL file for FTPMove service and
configure it with the new interaction specification for the move operation.
17. Open the FTPMove_ftp.jca file and modify the interaction-spec, as shown
in the following example.
You must configure the JCA file with the source and target directory and file details. You can either hardcode the source and target directory and file details in the JCA file or use header variables to populate them. In this example, header variables are used.
<adapter-config name="FTPMove" adapter="Ftp Adapter"
xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
<connection-factory location="eis/Ftp/FtpAdapter" adapterRef=""/>
<endpoint-interaction portType="FTPMove_ptt" operation="FTPMove">
<interaction-spec
className="oracle.tip.adapter.ftp.outbound.FTPIoInteractionSpec">
<property name="SourcePhysicalDirectory" value="foo1"/>
<property name="SourceFileName" value="bar1"/>
<property name="TargetPhysicalDirectory" value="foo2"/>
<property name="TargetFileName" value="bar2"/>
<property name="Type" value="MOVE"/>
</interaction-spec>
</endpoint-interaction>
</adapter-config>

18. Map the actual directory and file names to the source and target file parameters by performing the following procedure:
a. Create 4 string variables with appropriate names. You must populate these variables with the source and target directory details. The BPEL source view shows you this:
<variable name="sourceDirectory" type="xsd:string"/>
<variable name="sourceFileName" type="xsd:string"/>
<variable name="targetDirectory" type="xsd:string"/>
<variable name="targetFileName" type="xsd:string"/>

b. Create an assign activity to assign values to sourceDirectory,
sourceFileName, targetDirectory, and targetFileName variables.
The assign operation appears in the BPEL source view as in the following example:
 I use BPEL2.0, my assignment looks like:
   <assign name="Assign1">
      <copy>
        <from>'/test/input'</from>
        <to>$sourceDirectory</to>
      </copy>
      <copy>
        <from>'input.txt'</from>
        <to>$sourceFileName</to>
      </copy>
      <copy>
        <from>'/test/output</from>
        <to>$targetDirectory</to>
      </copy>
      <copy>
        <from>'output.txt'</from>
        <to>$targetFileName</to>
      </copy>
</assign>
c. Pass these parameters as headers to the invoke operation. The values in these
variables override the parameters in the JCA file.

The code snippet has apparent errors, here is my code in BPEL2.0 format:
    <invoke name="Invoke" partnerLink="mvFile" portType="ns1:FTPMove_ptt"
            operation="FTPMove" inputVariable="Invoke_FTPMove_InputVariable"
            outputVariable="Invoke_FTPMove_OutputVariable"
            bpelx:invokeAsDetail="no">
      <bpelx:toProperties>
        <bpelx:toProperty name="jca.file.SourceDirectory"
                          variable="sourceDirectory"/>
        <bpelx:toProperty name="jca.file.SourceFileName"
                          variable="sourceFileName"/>
        <bpelx:toProperty name="jca.file.TargetDirectory"
                          variable="targetDirectory"/>
        <bpelx:toProperty name="jca.file.TargetFileName"
                          variable="targetFileName"/>
      </bpelx:toProperties>
    </invoke>

A few notes here, do NOT use the UI and select the “properties”, these properties do not show up in the list, AND the properties show up in the list do NOT work L (for example, jca.ftp.FileName would look like a perfect choice, but it is misleading, it does not work!).
Additionally, there are “To” and “From” properties I can’t find any documents about these choices.  I finally figured out (network sniffing) that “to” appears to be what used to be called “input” properties. I would assume “from” would be the same as old “output” properties.


19. Finally, add an initial receive or pick activity.
You have completed moving or renaming a file from a remote directory to another remote directory on the same FTP server.
If you created a default BPEL, you may already have your receive activity. Run your test, that should take move “/test/input/input.txt” to “/test/output/output.txt” on the same FTP server.

If you want to move the file from remote “/test/input/input.txt” to local “/test/output/output.txt”, you need to add this property to the jac file:

<property name="TargetIsRemote" value="false"/>

Wednesday, September 12, 2012

Extracting and Passing WSS Name Token Security Header in BPEL

How to extract and insert SOAP headers in BPEL 2.0 is documented in the Oracle doc here http://docs.oracle.com/cd/E12839_01/integration.1111/e10224/bp_manipdoc.htm#SOASE415 However, like all Oracle docs, it takes some hands on finessing to figure out the specifics.

Additionally, for WSS security tokens, I have to dance around the WSS security XSD files in order to get it to work. It may not be the best way, but it seems to work.

Let's look at how to extract SOAP headers in BPEL. As Oracle document indicates, it takes two steps. You need to be very precise with each step, otherwise, you won't see the headers inside BPEL.

Step I -  update the WSDL - the Oracle document provides an excellent sample with multiple headers, it just needs some more detailed descriptions. Let me copy the sample here:
<!-- custom header -->
   <message name="CustomHeaderMessage">
    <part name="header1" element="tns:header1"/>
    <part name="header2" element="tns:header2"/>
  </message>
  <binding name="HeaderServiceBinding" type="tns:HeaderService">
    <soap:binding style="document"
      transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="initiate">
      <soap:operation style="document" soapAction="initiate"/>
        <input>
          <soap:header message="tns:CustomHeaderMessage"
            part="header1" use="literal"/>
          <soap:header message="tns:CustomHeaderMessage"
            part="header2" use="literal"/>
          <soap:body use="literal"/>
        </input>
    </operation>
  </binding>

Step 1.1 - define the messages to enclose the header elements to be passed in SOAP header. 
Notice that in the <!-- custom header --> section, The CustomHeaderMessage has two parts.  In order to pass in multiple SOAP headers, each SOAP header needs to be included as a part in this message.

The part names are "part1" and "part2". Although not critical, in the real world, you should give them some more meaningful part names.

Although obvious, it is still worth noting that these elements need be defined somewhere or imported from somewhere. For example, in your schema "header1" may hypothetically defined as:

<element name="header1">
<complexType>
<sequence>
<element name="fristName" type="string"/>
<element name="lastName" type="string"/>
</sequence>
</complexType>
</element>
In the case of WSS security header, I have to import a few related XSD's - see details below.

Step 1.2 - update the operation in the binding section

The Oracle sample may not look like the typical BPEL WSDL, so here is the changes I made to my WSDL:

<wsdl:operation name="process">
<wsdl:input  message="client:fooBPELRequestMessage">
<soap:header message="client:securityHeaderMsg"  part="payload" use="literal"/>
<soap:header message="client:fooBPELResponseMessage"  part="payload" use="literal"/>
</wsdl:input>
<wsdl:output message="client:fooBPELResponseMessage"/>
</wsdl:operation>

Here I am passing two SOAP headers. Keep in mind, very importantly, in Step 1.1, these two headers need be declared in a single message as two parts.

Step II - Receive (extract) headers in BPEL

Step 2.1 - define a message-type variable using the CustomHeaderMessage

    <variable name="customHeaderVar" messageType="client:customHeaders"/>

Step 2.2 - update receive activity
  <receive name="receiveInput" 
partnerLink="foobpel_client" 
portType="client:fooBPEL" 
operation="process" variable="inputVariable" 
createInstance="yes" 
bpelx:headerVariable="customHeaderVar"/>
There you go, customHeaderVar will contain the SOAP header values if you pass them in. That's a big "if", because you do not pass the headers in your SOAP call, or do not pass them in correctly, you still won't receive the headers, although it causes no harm to the receive activity even if you don't pass the headers in yours SOAP call.

In my case, I am only interested in the WSS security header, so I first define a "security" variable:
   <variable name="SecurityVar" element="ns2:Security"/> 
where ns2 is xmlns:ns2="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
then I pick up the WSS header:
    <assign name="AssignCustomHeader">
      <copy>
        <from>$customHeaderVar.payload1</from>
        <to>$SecurityVar</to>
      </copy>
    </assign>

Step III passing SOAP headers in BPEL invoke activity

Finally, I can pass in the WSS security header in the invoke:

     <invoke name="InvokeBar"
                partnerLink="fooCallsBar" portType="ns1:barBPEL"
                operation="process" inputVariable="Invoke_process_InputVariable"
                outputVariable="Invoke_process_OutputVariable"
                bpelx:invokeAsDetail="no" bpelx:inputHeaderVariable="SecurityVar"/>

The WSS Security XSD Conundrum 

I couldn't effectively importing WSS security XSD remotely, it takes forever for JDEV to drill down the WSS security element structure, so I ended up copying these 3 files locally:
"oasis-200401-wss-wssecurity-secext-1.0.xsd", 
"oasis-200401-wss-wssecurity-utility-1.0.xsd" 
"xmldsig-core-schema.xsd"
I had to comment out this line in the first file, so it won't bring my house down 
<!--xsd:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="http://www.w3.org/2001/xml.xsd"/-->

Here is my entire WSDL:

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions name="fooBPEL"
             targetNamespace="http://xmlns.oracle.com/cis/foo/fooBPEL"
             xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
             xmlns:client="http://xmlns.oracle.com/cis/foo/fooBPEL"
             xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
             xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
             xmlns:plnk="http://docs.oasis-open.org/wsbpel/2.0/plnktype">
<wsdl:types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://xmlns.oracle.com/cis/foo/fooBPEL" schemaLocation="xsd/fooBPEL.xsd" />
<import namespace="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
                        schemaLocation="xsd/oasis-200401-wss-wssecurity-secext-1.0.xsd" />
</schema>
</wsdl:types>
<wsdl:message name="fooBPELRequestMessage">
<wsdl:part name="payload" element="client:process"/>
</wsdl:message>
<wsdl:message name="fooBPELResponseMessage">
<wsdl:part name="payload" element="client:processResponse"/>
</wsdl:message>
<wsdl:message name="securityHeaderMsg">
<wsdl:part name="payload" element="wsse:Security"/>
</wsdl:message>
<wsdl:message name="customHeaders">
<wsdl:part name="payload1" element="wsse:Security"/>
<wsdl:part name="payload2" element="client:processResponse"/>
</wsdl:message>
<wsdl:portType name="fooBPEL">
<wsdl:operation name="process">
<wsdl:input  message="client:fooBPELRequestMessage">
                                <soap:header message="client:securityHeaderMsg"  part="payload" use="literal"/>
                                <soap:header message="client:fooBPELResponseMessage"  part="payload" use="literal"/>
                        </wsdl:input>
<wsdl:output message="client:fooBPELResponseMessage"/>
</wsdl:operation>
</wsdl:portType>
<plnk:partnerLinkType name="fooBPEL">
<plnk:role name="fooBPELProvider" portType="client:fooBPEL"/>
</plnk:partnerLinkType>
</wsdl:definitions>

Here is the SOAP UI test payload:

soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:foob="http://xmlns.oracle.com/cis/foo/fooBPEL">
   <soapenv:Header>
    <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <wsse:UsernameToken>
         <wsse:Username>weblogic</wsse:Username>
         <wsse:Password>welcome1</wsse:Password>
      </wsse:UsernameToken>
   </wsse:Security>
   <processResponse xmlns="http://xmlns.oracle.com/cis/foo/fooBPEL">
      <result>this is my header</result>
   </processResponse>
   </soapenv:Header>
   <soapenv:Body>
      <foob:process>
         <foob:input>SOAPUI invokes foo</foob:input>
      </foob:process>
   </soapenv:Body>
</soapenv:Envelope>

Snippet of BPEL source:


         xmlns:ns2="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
  ...
  <variables>
    <variable name="inputVariable" messageType="client:fooBPELRequestMessage"/>
    <variable name="outputVariable" messageType="client:fooBPELResponseMessage"/>
    <variable name="Invoke_process_InputVariable"
              messageType="ns1:barBPELRequestMessage"/>
    <variable name="Invoke_process_OutputVariable"
              messageType="ns1:barBPELResponseMessage"/>
    <variable name="SecurityVar" element="ns2:Security"/>
  </variables>

  <sequence name="main">

    <!-- Receive input from requestor. (Note: This maps to operation defined in fooBPEL.wsdl) -->
    <receive name="receiveInput" partnerLink="foobpel_client" portType="client:fooBPEL" operation="process" variable="inputVariable"
    createInstance="yes" bpelx:headerVariable="customHeaderVar"/>
    <assign name="AssignCustomHeader">
      <copy>
        <from>$customHeaderVar.payload1</from>
        <to>$SecurityVar</to>
      </copy>
    </assign>
    <assign name="AssignBarInput">
      <copy>
        <from>"knock, knock, foo's calling"</from>
        <to>$Invoke_process_InputVariable.payload/ns1:input</to>
      </copy>
    </assign>

    <invoke name="InvokeBar"
            partnerLink="fooCallsBar" portType="ns1:barBPEL"
            operation="process" inputVariable="Invoke_process_InputVariable"
            outputVariable="Invoke_process_OutputVariable"
            bpelx:invokeAsDetail="no" bpelx:inputHeaderVariable="SecurityVar"/>






Thursday, August 30, 2012

count() vs. countNodes() in BPEL 2.0


I have wrestled with these two since 10g . However, I still mix them up when I use them in BPEL 2.0. The function signature of count() has changed since BPEL1.1.

For the record, here is the scoop on these two functions in BPEL 2.0:

I have created a simple test BPEL based on the sample schema (at the end of this post).

I tested count() and countNodes() like below:

#1 count($outputVariable.payload/client:result)

It shows 0. Please note "result" is optional based on the XSD. Additionally, please note that if you enter an invalid path, such as “$outputVariable.payload/foo/bar”, JDev won’t compile it. So count() only allows valid xpath for the variable type, check #5 of countNodes() below to see more on this.

#2 count($inputVariable.payload/client:input)

It shows 1 as expected.

#3 ora:countNodes('outputVariable', 'payload', '/client:processResponse/client:result')

It shows 0, same as case #1. However, pay close attention to two things: first argument is the variable name as a string, not “$variable” as it appears in count(). Additionally, the 3rd argument uses the full path “/client:processResponse/client:result”. It includes “client:processResponse” section.

 #4 ora:countNodes('inputVariable', 'payload', '/client:process/client:input')

     It shows 1, just like in case #2. However, pay attention that first argument is the string name of the variable. Second argument is the “part” of the message type. Third argument needs the full path like “/client:process/client:input”. In the case of count() function, it appears that it skipped the “root” element “client:process” after payload.

#5 ora:countNodes('outputVariable', 'payload', '/client:input/test_invalid_path)

Finally, this one shows 0 as expected. The point here is to show that you can enter a path that is not defined in your XSD. This may come in handy if you have a payload that does not conform to your input schema, or you simply do not know the schema beforehand.

Here is the schema, please note that I have made "result" optional:

<?xml version="1.0" encoding="UTF-8"?> 
<schema attributeFormDefault="unqualified"
            elementFormDefault="qualified"
            targetNamespace="http://xmlns.oracle.com/cis/foo/BPELProcess1"
            xmlns="http://www.w3.org/2001/XMLSchema">
            <element name="process">
                        <complexType>
                                    <sequence>
                                                <element name="input" type="string" minOccurs="0"/>
                                    </sequence>
                        </complexType>
            </element>
            <element name="processResponse">
                        <complexType>
                                    <sequence>
                                                <element name="result" type="string" minOccurs="0"/>
                                    </sequence>
                        </complexType>
            </element>
</schema>

Sunday, August 26, 2012

Search and Replace in Composite Configuration Plan File

SOA 11g provides a configuration plan file for composite deployment. A common use is to search and replace hosts and ports depends on the platform (dev, test, qa, prod) you are deploying to. The question is when do those search / replacement take place?

If you wonder why do I ask this seemingly obscured question, let's take another look at it.

Suppose you do development on server "soaDEV.mybusiness.com", and SOA runs on port 8001. Then you need to migrate your work to "soaPRD.mybusiness.com", and SOA runs on port 80 over there.

So your plan file may contain something like:

  <search>soaDEV.mybusiness.com:8011</search>
  <replace>soaPRD.mybusiness.com:80</replace>

When you click on "Deploy" and pick your configuration plan file, a common misconception is that "soaDEV.mybusiness.com:8001" will be magically replaced, and "soaDEV.mybusiness.com" is almost irrelevant. However, if you shut down "soaDEV" server, you will notice that JDEV will fail the compilation.


When you click "Deploy" in JDev, it's actually a two-step process. First Jdev generates the jar (sar) file. Then it uploads (deploys) the jar to the SOA server (you can actually do these two steps manually. First compile / generate the jar file. Then go to EM console to deploy the composite.)

The search / replacement only takes place during the 2nd stage. When SOA server receives the composite jar file, it does a second compilation using BPEL Compiler. So when JDEV compiles the BPEL source code, it still requires your original "soaDEV.mybusiness.com" to be up and running. It still needs to reference the WSDL from port 8001on "soaDEV".

Now we know the facts, I still have this question that bugs me all the time. Why can't (won't) JDEV do search and replacement before the compilation? I don't truly know the answer. I'm sure there is a fundamental answer to this question.

I always have this eerie feeling that 9 months after you have finished your production deployment, you still need to keep your soaDEV server running in order to support your soaPRD environment. Because Jdev needs to reference soaDEV in order to compile. What about the potential inconsistency between soaDEV and soaPRD 9 months after or even longer?


Reading XMLType with DB Adapter

My Oracle table has a column with XMLType. I have found out it can be very tricky when reading the XMLType column with DB Adapter.

First of all, if you use TopLink to map the data, BPEL will read in the XMLType column as a string. 

That string variable is no longer an XML element inside BPEL, and you can no longer process it. In fact, it's very trickery to the eyes if you view your DB read output in the EM console, it appears that the XML payload is read in successfully, but that's not true. If you check the raw XML view, you'll see the "fake" XML element is a string with special escaped characters "&lt;". So you can not use this scrambled string for much of anything.

<messagePayload>&lt;cws:cwsPaymentEventService xmlns:cws="http://xmlns.calwater.com/cwsPaymentEvent.xsd"> &lt;cws:cwsPaymentEventDetails PayorAccountID="0956488888" PaymentAmount="12" PaymentDate="2012-08-12T01:02:03" TenderSource="KUBRA-RT"> &lt;/cws:cwsPaymentEventDetails> &lt;/cws:cwsPaymentEventService></messagePayload>

One way to tell if your variable is an XML structure or a string (with "fake" XML structure) is to see if the XML elements are shown as expanded or on one line.

So what if I need to read in the XMLType column and still want to process it as an XML structure? The work around is to use "pure SQL" to read the element in. Then it preserves the XML structure. You can assign the "pure SQL" read output into a variable of any XML structure. It's up to you to make sure that DB field (XMLType) and your BPEL variable have the same structure! Great flexibility, but use with care.

What if you need to do a DB poll with DBAdapter? DB poll doesn't have an option of "pure SQL". My solution: do two reads. First the poller loads the row in, then you need do another regular DB read for the same row (with your self defined DB key), this type you use "pure SQL". This is definitely inefficient to say the least, but it's one way to get around the limitation of DBAdapter and toplink.

I think the alternative to avoid two reads is to use parseEscapedXML() function. My colleague reminded me on that today, I vaguely remember we used that function on a previous project. But I haven't tried it this time.  I believe it should work as the function name indicated. You parse that string, and place the result into your XML element.

Additionally, if it is an option for you, you can use AQAdapter. AQ adapter can load in XMLType just fine. It's a one step solution.