Thursday, August 29, 2013

Custom JCA Adapter and Dynamic log4j Logging Level Control with Weblogic

Recently I worked on dynamical log4j logging level control in a custom JCA adapter. I still don’t know much about JCA adapter. But I managed to make it work, and in the process I picked up a few more interesting things about Weblogic and log4j configurations.

At the core, the solution is the same as in http://yuanmengblog.blogspot.com/2013/08/weblogic-log-files-and-log4j-tricks.html i.e. use this code snippet in a "central" location of your code:

if (notInitialized) {
   String fileName = System.getProperty("jca.log4j.configFile");
   DOMConfigurator.configureAndWatch(fileName, 3000);
}

I stumbled upon a few subtle things about log4j and Weblogic before I made it to work. Here are a few things to pay close attention if you want to do it:

1. By default, Weblogic uses its own logging mechanism (it’s based on log4j)
2. setDomainEnv.sh script may set a specific Java system variable "-Dlog4j.configuration". Based on my test, this variable is sort of a "reserved" by Weblogic. The value of this variable is "file:yourAbsoluteFilePath". Be extra careful with the variable name and the "file:" prefix. 
3. If you place log4j.xml in the domain home directory, then Weblogic will pick it up and uses that one. It confused me for a while. I only found it out by trial and error.

The common thing about these scenarios is that Weblogic will internally pick up the log4j.xml file. When that happens, Weblogic initializes log4j, and you have no control of the file whatsoever. You can modify the log4j.xml all you want; weblogic is not going to do anything with the changes unless you restart the server.

So the solution is :
1. Do not to place "log4j.xml" in the domain folder, or use a different file name, such as "mylog4j.xml". 
2. Use a different Java system variable name, such as "jca.log4j.configFile" (just make sure do not "log4j.configuration").
3. Since you pick your own variable name, you should not use the "file:" prefix for the variable value. Instead, just use the full path name of you log4j configuration file.

That’s about it. I named my file "my-test-log4j.xml". I picked a variable name like "jca.log4j.confiureFile". I was able to change "my-test-log4j.xml" and watch the logging levels change on the fly.

Also, make sure if you start the server (such as AdminServer) with the script, then you need to update the script to pass the variable in. If you use nodeManager (for managed servers), then you should use weblogic console, server, startup argument field to pass in the variable.

Finally, in the JCA rar file, there is a weblogic-ra.xml that allows you to name your "<log-filename>" element, I tested that, it merely generated a file with that name; nothing gets logged into that file. I’m not sure how to use that file. So I simply skipped this element in the weblogic-ra.xml file.

Thursday, August 15, 2013

OSB 11g XSLT multiple input parameters and the conversion error of org.apache.xmlbeans.impl.store.Xobj$DocumentXobj

OSB 11g XSLT expression editor allows you to enter multiple input parameters. The developer guide http://docs.oracle.com/cd/E23943_01/dev.1111/e15866/toc.htm (page 4-84) even contain a precise definition of each field with example. However, it doesn't work. You may see errors similar to this:
java.lang.RuntimeException: Invalid conversion from 'node-set' to 'org.apache.xmlbeans.impl.store.Xobj$DocumentXobj'

I see many frustrated bloggers posting questions. After my own long frustrating hours, i concluded it's an OSB bug. Good news is i also figured out a solution.

At the core, the issue is OSB passes in the parameter to XLST as org.apache.xmlbeans.impl.store.Xobj$DocumentXobj object (with the exception if the parameter is a simple string). But XSLT is expecting a NodeList.

Here is a quick solution. Compile this class, jar it up and copy it to your domain/lib directory:
package org.example.xslt;
import org.w3c.dom.NodeList;
import org.w3c.dom.Node;
public class XObjUtil
{
  public NodeList getNodeList(Object o)
  { System.out.println("### getNodeList o class name="+o.getClass().getName());
org.w3c.dom.Document xo = (org.w3c.dom.Document)o;
Node n = (Node) xo;
NodeList nl = n.getChildNodes();
       return nl;
// lazy way:  return (((org.w3c.dom.Document)o)).getChildNodes();
 }
}

Here is my cheap script to compile, jar and copy the file:

javac -d . XObjUtil.java
del myxlstext.jar
jar cvf myxlstext.jar org
copy myxlstext.jar C:\Oracle\Middleware\user_projects\domains\osbserver\lib

On the OSB side, here is the XSLT what will parse the input parameter:

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0"
xmlns:client="http://xmlns.oracle.com/JCA_ResourceAdapter/foo/BPELProcess1"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:XObj="http://www.oracle.com/XSL/Transform/java/org.example.xslt.XObjUtil"
exclude-result-prefixes="xsl client ">
<xsl:param name="car" />
<xsl:template match="/">
<client:process>
<xsl:variable name="x" select="XObj:new()" />
<xsl:variable name="obj" select="XObj:getNodeList($x, $car)" />
<client:input>
<xsl:value-of select="$obj/client:car1" />
</client:input>
<client:foo1>
<xsl:value-of select="$obj/client:car2" />
</client:foo1>
<client:foo2>
<xsl:value-of select="/client:barElem/client:bar2" />
</client:foo2>
</client:process>
</xsl:template>
</xsl:stylesheet>

My foo/bar/car sample is based on this cheap XSD
<?xml version="1.0" encoding="UTF-8" ?>
<schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://xmlns.oracle.com/JCA_ResourceAdapter/foo/BPELProcess1"
     xmlns:this="http://xmlns.oracle.com/JCA_ResourceAdapter/foo/BPELProcess1" xmlns="http://www.w3.org/2001/XMLSchema">
    <complexType name="fooType">
        <sequence>
            <element name="input" type="string"/>
            <element name="foo1" type="string"/>
            <element name="foo2" type="string"/>
        </sequence>
    </complexType>
    <complexType name="barType">
        <sequence>
            <element name="input" type="string"/>
            <element name="bar1" type="string"/>
            <element name="bar2" type="string"/>
        </sequence>
    </complexType>
    <complexType name="carType">
        <sequence>
            <element name="input" type="string"/>
            <element name="car1" type="string"/>
            <element name="car2" type="string"/>
        </sequence>
    </complexType>
    <element name="process" type="this:fooType"/>
    <element name="carElem" type="this:carType"/>
    <element name="barElem" type="this:barType"/>
    <element name="processResponse">
        <complexType>
            <sequence>
                <element name="result" type="string"/>
            </sequence>
        </complexType>
    </element>
</schema>

Here is a sample output from the OSB test console:

client:process xmlns:client="http://xmlns.oracle.com/JCA_ResourceAdapter/foo/BPELProcess1"xmlns:XObj="http://www.oracle.com/XSL/Transform/java/org.example.xslt.XObjUtil">
<client:input>car18</client:input>
<client:foo1>car29</client:foo1>
<client:foo2>bar23</client:foo2>
</client:process>

One interesting fact is this sample shows you a cheap way of extending XSLT without a full blown Xpath function project, see here http://yuanmengblog.blogspot.com/2013/04/break-comma-separated-string-into-array.html

BTW, very importantly, SOA and OSB uses different XSLT framework. OSB uses more org.w3 XML, SOA use "Oracle" own XML stuff. When SOA invokes XSLT, it passes in the parameter as type oracle.xml.parser.v2.XMLDocumentFragment, if ever want to play with that, the conversion is this:
 public NodeList getNodeList(oracle.xml.parser.v2.XMLDocumentFragment o)
 {
org.w3c.dom.NodeList nl = ((org.w3c.dom.Node)o).getChildNodes();
return  nl;
 }

Two other things you can explore:

1. instead of tossing the jar file into domain/lib directory, try if you can merely upload the jar to the proxy folder in OSB, treats it like a java callout jar. If that works, then you don't need to reboot OSB when change your jar file.

2. implement this as an Xpath function, then your JDev XSLT editor can pick it up. Of course, your XSLT syntax will be modified slightly as well. Have fun!

Saturday, August 10, 2013

OWSM SAML Policy Tests


This post assumes readers have the basic understanding of OWSM policy framework.
In this post, I’ll explore two examples of OWSM SAML policies. There are more than a dozen SAML policies that come with OWSM. One reason for the numerous SAML policies is OWSM provides two collections of SAML policies for WSS 1.0 (WS Security 1.0) and WSS 1.1. You can explore the difference between the WSS 1.0 and 1.1 that is outside the scope of this discussion. Suffice to say that when you pick policies on the client and service sides, you need to make sure that WSS versions match.

Nearly all of the OWSM SAML policies require some kind of encryption, except two wss10_saml(20)_token_service_policy. We will experiment with the non-encryption policy first, then we’ll discuss wss10_saml(20)_token_with_message_protection_service_policy.

For the experiments, I’ll create two local weblogic domains. “Domain 1” runs on port 7001. “Domain 2” runs on 7011. (Refer to previous post on how to createmultiple domains on your localhost).

For demonstration purpose, I use the following services:
  •         A JavaEE web service – deployed to domain 2 and locked by a SAML security service policy
  •         An OSB business service – on domain 1, it hooks up to the JavaEE web service. It has an attached SAML client policy.
  •         OSB proxy service – on domain 1, locked up by WS name token security, which routes to the business service.
You can find jar files for these services here: javaEESvc and osb-config-jar. You can deploy OSB config jar to domain1, javaEESvc jar to domain2.

The steps below illustrate the test scenarios:

1.      use soap UI or other client tool to call the proxy service with ws name token header (with user name, password)
2.      OSB/OWSM authenticates the user. In this case, it uses Weblogic server’s default security realm. So you need to have a proper user defined with the Weblogic server. Proxy is invoked if the user authentication is successful.
3.      OWSM create a SAML assertion based on the authentication of step #2 above.
4.      OSB attaches the SAML assertion (user name only, no password) to the outbound business service call to the JavaEE service
5.      Weblogic/OWSM authenticates the web service call with SAML header. In this case, the authentication only verifies the user name exists in the security realm. The password doesn’t matter. If the authentication is successful, the JavaEE service is invoked.
6.      JavaEE service response is routed back to SoapUI (via business service and proxy service)

WSDL
<definitions xmlns="http://schemas.xmlsoap.org/wsdl/"
  xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:tns="http://tempuri.org/"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  name="samlPocService" targetNamespace="http://tempuri.org/">
  <types>
    <xs:schema xmlns:tns="http://tempuri.org/" xmlns:xs="http://www.w3.org/2001/XMLSchema"
      version="1.0" targetNamespace="http://tempuri.org/">
      <xs:element name="sayHello" type="tns:sayHello" />
      <xs:element name="sayHelloResponse" type="tns:sayHelloResponse" />
      <xs:complexType name="sayHello">
        <xs:sequence>
          <xs:element name="arg0" type="xs:string" minOccurs="0" />
        </xs:sequence>
      </xs:complexType>
      <xs:complexType name="sayHelloResponse">
        <xs:sequence>
          <xs:element name="return" type="xs:string" minOccurs="0" />
        </xs:sequence>
      </xs:complexType>
    </xs:schema>
  </types>
  <message name="sayHello">
    <part name="parameters" element="tns:sayHello" />
  </message>
  <message name="sayHelloResponse">
    <part name="parameters" element="tns:sayHelloResponse" />
  </message>
  <portType name="samlPoc">
    <operation name="sayHello">
      <input message="tns:sayHello" />
      <output message="tns:sayHelloResponse" />
    </operation>
  </portType>
  <binding name="samlPocPortBinding" type="tns:samlPoc">
    <soap:binding style="document"
      transport="http://schemas.xmlsoap.org/soap/http" />
    <operation name="sayHello">
      <soap:operation soapAction="" />
      <input>
        <soap:body use="literal" />
      </input>
      <output>
        <soap:body use="literal" />
      </output>
    </operation>
  </binding>
  <service name="samlPocService">
    <port name="samlPocPort" binding="tns:samlPocPortBinding">
      <soap:address location="http://localhost:7011/javaeesvc/samlPocPort" />
    </port>
  </service>
</definitions>

JavaEE service:
It’s a no thrill simple hello world java function:
package javaeesvc;
import javax.jws.WebService;
@WebService(targetNamespace = "http://tempuri.org/")
public class samlPoc {
    public samlPoc() { super(); }   
    public String sayHello(String s) {
        System.out.println("######Hello " + s + ", this is saml poc");
        return "######Hello " + s + ", this is saml poc";
    }
}

JavaEE service is deployed to domain 2:

Biz Service Configuration, no policy attached yet:

Proxy service:

Then attach WS name token policy to the proxy

Experiment 1SAML Policy without Encryption
Testing “oracle/wss10_saml20_token_client_policy
In this experiment, we’ll demonstrate the SAML policy without encryption (plain SAML policies). Without encryption, there will be no additional server configurations involved.  You can merely attach the corresponding SAML policies on both the JavaEE service and OSB business service, that’s it.
In the next section we’ll see when message protection (encryption) is involved, it becomes a different ball game. You need to configure quite a few things before SAML policies can work.
Let’s do the simple one first.

Attach SAML policy to JavaEE service:

Under “Web Service Endpoints” tab, click on endpoint name “samlPocPort”:

Click on “Attach/Detach” icon:

Select “wss10_saml20_token_service_policy” then press “Attach”
Attach SAML policy to OSB business service:


SOAP UI client test:

What has happened?
Part 1: Soap UI calls Proxy Service with WSS name token
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:tem="http://tempuri.org/">
  <soapenv:Header>
    <wsse:Security soapenv:mustUnderstand="1"
      xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <wsse:UsernameToken wsu:Id="UsernameToken-2"
        xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
        <wsse:Username>weblogic</wsse:Username>
        <wsse:Password
          Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">welcome1
</wsse:Password>
      </wsse:UsernameToken>
    </wsse:Security>
  </soapenv:Header>
  <soapenv:Body>
    <tem:sayHello>
      <arg0>soupui</arg0>
    </tem:sayHello>
  </soapenv:Body>
</soapenv:Envelope>

Part 2: Business Service Invokes JavaEE service with SAML token

Internally, OWSM first authenticates user “weblogic” with password, then it generate SAML token (assertion). OSB business service attaches the SAML assertion in the header. Business service calls JavaEE service with the SAML assertion.

<soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <wsse:Security soapenv:mustUnderstand="1"
    xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
    <saml:Assertion MajorVersion="1" MinorVersion="1"
      AssertionID="SAML-xnTew5qgIPuu2kkZGcVqhQ22" IssueInstant="2013-07-29T22:26:43Z"
      Issuer="www.oracle.com" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
      <saml:Conditions NotBefore="2013-07-29T22:26:43Z"
        NotOnOrAfter="2013-07-29T22:31:43Z" />
      <saml:AuthenticationStatement
        AuthenticationInstant="2013-07-29T22:26:43Z" AuthenticationMethod="urn:oasis:names:tc:SAML:1.0:am:password">
        <saml:Subject>
          <saml:NameIdentifier
            Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified">weblogic</saml:NameIdentifier>
          <saml:SubjectConfirmation>
            <saml:ConfirmationMethod>urn:oasis:names:tc:SAML:1.0:cm:sender-vouches
            </saml:ConfirmationMethod>
          </saml:SubjectConfirmation>
        </saml:Subject>
      </saml:AuthenticationStatement>
    </saml:Assertion>
  </wsse:Security>
</soap:Header>
<soapenv:Body>
  <urn:sayHello xmlns:urn="urn:examples:helloservice">
    <firstName xsi:type="xs:string" .>test</firstName>
  </urn:sayHello>
</soapenv:Body>
</soapenv:Envelope>

In case anyone wonders how I captured the SAML payload, I used TCPMon tool to trap the business outbound call. With OSB 11.1.1.6 version, you can also see similar payload using the OSB business service test console.

The main take away in this experiment is that OSB business service calls JavaEE with SAML assertion and user name, but no password is provided. JavaEE service is configured with a plain SAML token policy that blindly trusts anyone with a well formatted SAML token, as long as the user name exists in “Domain 2”. The web service request will be accepted.

Well, if you find this behavior questionable, then you are correct. Oracle doesn’t recommend using this “plain” dumb SAML policy. In next section, we’ll test with a SAML policy that requires “message protection”. i.e. payload will be encrypted with key (certificate), so the service requests (from the client) can be trusted with assurance by the web service (server).

SAML policy with message protection (encryption)
We’ll use wss11_saml20_token_with_message_protection_client_policy for our experiment.
This SAML policy requires the payload be encrypted (message protection). As far as OWSM policy configurations are concerned, there is not much difference between this policy and the plain SAML policy we discussed earlier. However, there are major configuration steps involved to setup the domains for the encryption/decryption to work.

Setting up certificate trusts between the two domains

If you never worked with OWSM key stores, it can be a daunting task to go through the steps in this section for the first time. You can reference this previous post to get familiar with the OWSM key stores.

By default, the key store is here {domain_home}/config/fmwconfig/default-keystore.jks. The actual key store location is specified in this file {domain_home}/config/fmwconfig/jps-config.xml

In order for message encryption/decryption to work, the two domains need to exchange their keys. “Domain 2” will import the public key of “Domain 1”. So when “domain 1” sends an encrypted message (encrypted with Domain 1’s private key), “domain 2” will be able to decrypt the message with “domain 1’s” public key. Vice versa, “domain 1” needs to import the public key of “domain 1”.

Here are the keytool commands to generate keys for each domain, and to import the public keys to each other’s key store.
 
  #### generate default keypair for domain1, valid for 10 years
  keytool -genkeypair -alias d1key  -keyalg RSA -keypass d1pass -keystore domain1-keystore.jks  -storepass welcome1 -validity 3650
  #### generate default keys for domain2, valid for 10 years
  keytool -genkeypair -alias d2key  -keyalg RSA -keypass d2pass -keystore domain2-keystore.jks  -storepass welcome2 -validity 3650

  #### list key stores
  keytool -list -v -storepass welcome1 -keystore domain1-keystore.jks
  keytool -list -v -storepass welcome2 -keystore domain2-keystore.jks

  #### export domain1 public key
  keytool -exportcert -alias d1key -storepass welcome1 -keystore domain1-keystore.jks -file domain1-pubkey.cer
  #### export domain1 public key
  keytool -exportcert -alias d2key -storepass welcome2 -keystore domain2-keystore.jks -file domain2-pubkey.cer

  #### import domain2 public key into domain 1 as "d2impkey"
  keytool -importcert -alias d2impkey -storepass welcome1 -keystore domain1-keystore.jks  -file domain2-pubkey.cer
  #### import domain1 public key into domain 2 as "d1impkey"
  keytool -importcert -alias d1impkey -storepass welcome2 -keystore domain2-keystore.jks  -file domain1-pubkey.cer

  ### if you need to remove a key
  keytool -delete  -alias yourkey  -keystore keystore.jks -storepass welcome1

The screen below shows how to configure the key store for domain 1. Pay close attention to each field in this screenshot and notice how the field values correspond to the values we used in the key commands above. You will need to bounce the servers if you change key store files. Remember the default is default-keystore.jks, which is specified in jps-config.xml.

This screen shows how to configure the key store for domain 1.

Once the OWSM key stores are configured for both domains. You will detach the plain SAML policies from javaEESvc on domain 2 and the business service on domain 1. Then reattach wss10_saml20_token_with_message_protection_service_policy to JavaEESvc, and wss10_saml20_token_with_message_protection_client_policy to the OSB business service.

You also need to configure the security property on the business service as shown below.The recipient alias is the domain 2 public key. We imported into domain 1 with the alias "d2impkey". This key will be actually used to decrypt the response message from domain 2, which is encrypted with domain's private key.

This screen shows how to configure the business service “security” tab:

You will need to configure domain 2 OWSM key store similarly.

Test in OSB console

If everything is configured correctly, you can test the business service from the OSB console (works for OSB 11.1.1.6, may not work for earlier version)
The test console shows data is encrypted:

Test with SoapUI
The SoapUI test will show the same thing either SAML policy is encrypted or not encrypted.
Final notes:
I used OSB and JavaEE service for the experiments. You can also use SOA composite to test the SAML policies. It is recommended to use JDeveloper to attach policies to SOA composite, but you can also use "em" console to the same.