Unix file permission when file adatper generates the output file

When use file adapter to write a file on Linux, what is the permission of the output file?

The common assumption is it is determined by the umask of the user who owns the “Weblogic” java process. If a user runs startWeblogic.sh from the command line, then that user is the owner of Weblogic processes (many places use “oracle” or “weblogic” as the user to start webogic processes).

For the discussion, let’s assume the owner of “weblogic” logic process is “webadmin”. We need to generate an output file with “664” permission. However, we keep getting 644.

We double checked “webadmin” user has 002 umask. We also tried to play with the umask settings in startWeblogic.sh script to no avail.

So it’s gloves off time, I added java embed to change the output file permission by calling “setWrtiable(true, false)”, and that, opened another can of worms.

To my surprise, I stumbled on another unexplainable file adapter behavior. After the file adapter finished writing the file (with 644 permission), if I use java embed to open the file, and do any operations on the file (even just read it), then it TRUNCATES the output file. I thought it has to do with the transaction, so I tried to call check_point() before I open the file in java. It made no difference.

In the end, I just skipped the file adapter, use java embedding to generate the output file, then set the permission. Here is the snippet:

        try {                   

                String content = (String)getVariableData("csvOutput");   // grab file content                       

                String fileName = (String)getVariableData("fileName"); 

                fileName = "/output/"+fileName;

                FileWriter fstream = new FileWriter(fileName);

                BufferedWriter out = new BufferedWriter(fstream);

                out.write(content+"\r\n");

                out.close();

                                               

                java.io.File f = new java.io.File(fileName);

                f.setWritable(true, false);  // set permissions

                f.setReadable(true, false);

        }      catch(Exception e)      {

                System.out.println("*****file exception:"+e.getMessage());

        }

 There is an Oracle Note - 1430075.1 that covers the same issue. Oracle recommendation is to use java embedding, then do

   System.exec(“chmod…”);

I didn’t try that. Not sure how it works. Hope it doesn’t truncate the output file.

Use JCA file adapter to Parse CSV file with master and detailed records

JCA file adapter can parse single record type CSV file easily. It also has limited support to parse mixed record type CSV files. However, it relies on the record data starts with a fixed values.

In my case, I have a CSV file with master-detailed records that look like this:

Master c1, master c2, master c3
 a, b, c
Det c1, det 2, det 3, det 4, det 5
1, 2, 3,4,5
6,7,8,9,0

Since my detailed records do not start with a fixed value (detail row 1 starts with 1, row 2 starts with 6), I cannot use JCA wizard to parse this file directly. Here is how I managed to do it. My solution is to create the two file adapters and parse the same data file twice. First parse the master record, then parse the detailed records.

For master record:

1.      Create a copy of the sample data file, and remove the detailed records

2.      Remove the spaces in the master header row

3.      Generate XSD with JCA adapter native file wizard, select uniform file, use 1^st line as header

4.      After XSD (let’s call it header.xsd) is generated, make these changes in the XSD:

·        nxsd:headerLinesTerminatedBy="${eol}" – make sure it’s like this.

·        nxsd:headerLines="1" – this is a misnomer, it simply means how many lines to skip

·        nxsd:hasHeader="false"     -- sounds contradictory to the line above. But the actual data file has spaces in the header title, this will make the parser skip header line

·        nxsd:dataLines="1"             –  so it only reads 1 line of data

Follow similar steps for detailed-records; remove the master head and records from sample data file, after XSD (call it body.xsd) is generated, make the following changes:

·        nxsd:hasHeader="false" – so it won’t parse header records, because spaces in header cause problems

·        nxsd:headerLines="3" – this will skip 3 lines in the data file

·        nxsd:headerLinesTerminatedBy="${eol}" – make sure the headers line is terminated properly

In the BPEL process, create two JCA file adapters.

The first adapter is called to load (parse) the header record into BPEL, choose header.xsd to when create the adapter. Remember, header.xsd skips the first line of the data file, and parse the 1^st line, since we set dataLines=”1”, it will skip rest of the detailed records.

Also make sure modify the .jca file, so after header record is loaded, do not delete the data file:
  <property name="DeleteFile" value="false"/>

Create the 2^nd file adapter and select body.xsd. Based on the XSD, the 2^nd adapter will skip the first 3 lines (master record header, data and detailed record header.

With this two-pass approach,  you can load both header and detailed records into BPEL.

File and FTP adatper

Working with Oracle File and FTP adapter is definitely like black magic! I have SOA 11.1.1.6.0. I need to use FTP adapter to copy a file from a remote server. Both FTP and File adapters are designed to read a file, not to copy a file. If you want to copy a file and not process it, you have to jump through the hoops.

I sifted through 3 versions  of "Oracle® Fusion Middleware, User's Guide for Technology Adapters, 11g Release" (11.1.1.6.0, 11.1.1.6.2 and 11.1.1.6.3). Part of the document is inaccurate to say the least.I even tried to look up the adapter java classes (Oracle_OSB1\lib\external\adapters\ftpAdapter.jar) looking for clues. In the end, I resorted to trial and error, and with the help of network sniffing to sort out the mess.

The main challenge is to follow the instruction in section "4.5.11.5 Moving a File from One Remote Directory to Another Remote Directory on the Same FTP Server". Here are the instructions with my comments:

1. Create an empty BPEL process 

I created a synchronous BPEL, selected all default.

2. Drag and drop FTP Adapter from the Component Palette to the External

References swim lane. The Adapter Configuration Wizard Welcome page is displayed.

3. Click Next. The Service Name page is displayed.

4. Enter a service name in the Service Name field.

5. Click Next. The Adapter Interface page is displayed.- 

Default "Define from operation and schema (specified later) is checked.

6. Click Next. The FTP Server Connection page is displayed.

7. Enter the JNDI name for the FTP server, and click Next. The Operation page is

displayed. – 

JDev default might be "eis/ftp/FtpAdapter", however, SOA server default may be eis/Ftp/FtpAdapter.Look closely, you may need to change the 2nd part from "ftp" to "Ftp" (upper case "F").

8. Select Synchronous Get File, enter FTPMove in the Operation Name field, and

then click Next. The File Directories page is displayed. – 

do not overlook that "FTPMove". This is part of the black magic, it has to be precisely spelled like that, or you will be doomed. Why can't there be a drop down selection of "FTPMove"? If you have to enter specific text, what's the point of using an IDE.

9. Enter a dummy physical path for the directory for incoming files, and then click Next. The File name page is displayed.

Note: The dummy directory is not used. You must manually change the directory in a later step.

This is misleading. You can put real path in here, and the adapter uses it if you don’t overwrite it dynamically in BPEL code. Same applies to the other parameters.

10. Enter a dummy file name, and then click Next. The File Name page is displayed.

11. Click Next. The Messages page is displayed.
12. Select Native format translation is not required (Schema is opaque), and then
click Next. The Finish page is displayed.
13. Click Finish. The outbound Oracle File Adapter is now configured.
14. Drag the small triangle in the BPEL process in the Components area to the drop
zone that appears as a green triangle in FTPMove in the External References area.
The BPEL component is connected to the Oracle FTP Adapter outbound service.

15. Click File, Save All.

16. Create an invoke activity for the FTPMove service that you just created.
The next step is to modify the generated WSDL file for FTPMove service and
configure it with the new interaction specification for the move operation.
17. Open the FTPMove_ftp.jca file and modify the interaction-spec, as shown
in the following example.
You must configure the JCA file with the source and target directory and file details. You can either hardcode the source and target directory and file details in the JCA file or use header variables to populate them. In this example, header variables are used.

<adapter-config name="FTPMove" adapter="Ftp Adapter"

xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

<connection-factory location="eis/Ftp/FtpAdapter" adapterRef=""/>

<endpoint-interaction portType="FTPMove_ptt" operation="FTPMove">

<interaction-spec

className="oracle.tip.adapter.ftp.outbound.FTPIoInteractionSpec">

<property name="SourcePhysicalDirectory" value="foo1"/>

<property name="SourceFileName" value="bar1"/>

<property name="TargetPhysicalDirectory" value="foo2"/>

<property name="TargetFileName" value="bar2"/>

<property name="Type" value="MOVE"/>

</interaction-spec>

</endpoint-interaction>

</adapter-config>

18. Map the actual directory and file names to the source and target file parameters by performing the following procedure:

a. Create 4 string variables with appropriate names. You must populate these variables with the source and target directory details. The BPEL source view shows you this:

<variable name="sourceDirectory" type="xsd:string"/>

<variable name="sourceFileName" type="xsd:string"/>

<variable name="targetDirectory" type="xsd:string"/>

<variable name="targetFileName" type="xsd:string"/>

b. Create an assign activity to assign values to sourceDirectory,

sourceFileName, targetDirectory, and targetFileName variables.

The assign operation appears in the BPEL source view as in the following example:

 I use BPEL2.0, my assignment looks like:

   <assign name="Assign1">

      <copy>

        <from>'/test/input'</from>

        <to>$sourceDirectory</to>

      </copy>

      <copy>

        <from>'input.txt'</from>

        <to>$sourceFileName</to>

      </copy>

      <copy>

        <from>'/test/output</from>

        <to>$targetDirectory</to>

      </copy>

      <copy>

        <from>'output.txt'</from>

        <to>$targetFileName</to>

      </copy>

</assign>

c. Pass these parameters as headers to the invoke operation. The values in these

variables override the parameters in the JCA file.

The code snippet has apparent errors, here is my code in BPEL2.0 format:

    <invoke name="Invoke" partnerLink="mvFile" portType="ns1:FTPMove_ptt"

            operation="FTPMove" inputVariable="Invoke_FTPMove_InputVariable"

            outputVariable="Invoke_FTPMove_OutputVariable"

            bpelx:invokeAsDetail="no">

      <bpelx:toProperties>

        <bpelx:toProperty name="jca.file.SourceDirectory"

                          variable="sourceDirectory"/>

        <bpelx:toProperty name="jca.file.SourceFileName"

                          variable="sourceFileName"/>

        <bpelx:toProperty name="jca.file.TargetDirectory"

                          variable="targetDirectory"/>

        <bpelx:toProperty name="jca.file.TargetFileName"

                          variable="targetFileName"/>

      </bpelx:toProperties>

    </invoke>

A few notes here, do NOT use the UI and select the “properties”, these properties do not show up in the list, AND the properties show up in the list do NOT work L (for example, jca.ftp.FileName would look like a perfect choice, but it is misleading, it does not work!).

Additionally, there are “To” and “From” properties I can’t find any documents about these choices.  I finally figured out (network sniffing) that “to” appears to be what used to be called “input” properties. I would assume “from” would be the same as old “output” properties.

19. Finally, add an initial receive or pick activity.

You have completed moving or renaming a file from a remote directory to another remote directory on the same FTP server.

If you created a default BPEL, you may already have your receive activity. Run your test, that should take move “/test/input/input.txt” to “/test/output/output.txt” on the same FTP server.

If you want to move the file from remote “/test/input/input.txt” to local “/test/output/output.txt”, you need to add this property to the jac file:

<property name="TargetIsRemote" value="false"/>

SOA deployment plan with JCA properties

Configuration/Deployment plan has a section for JCA, somehow i can't get this to work (11.1.1.4)
 <reference name="JcaCopyFile">
      <!-- ################### can't get this section to work, so add <wsdlAndSchema name="JcaCopyFile_file.jca"> at the bottom
         <binding type="jca">
          <property name="TargetPhysicalDirectory">
            <replace>/test</replace>
          </property>
          <property name="TargetDirectory">
            <replace>/test</replace>
          </property>
         </binding>
         -->
      </reference>
it works with this:

  <wsdlAndSchema name="JcaCopyFile_file.jca" xmlns:jca="http://platform.integration.oracle/blocks/adapter/fw/metadata">
    <jca:property name="TargetPhysicalDirectory">
      <replace>/test</replace>
    </jca:property>
  </wsdlAndSchema>