Friday, September 21, 2012

File and FTP adatper


Working with Oracle File and FTP adapter is definitely like black magic! I have SOA 11.1.1.6.0. I need to use FTP adapter to copy a file from a remote server. Both FTP and File adapters are designed to read a file, not to copy a file. If you want to copy a file and not process it, you have to jump through the hoops.

I sifted through 3 versions  of "Oracle® Fusion Middleware, User's Guide for Technology Adapters, 11g Release" (11.1.1.6.0, 11.1.1.6.2 and 11.1.1.6.3). Part of the document is inaccurate to say the least.I even tried to look up the adapter java classes (Oracle_OSB1\lib\external\adapters\ftpAdapter.jar) looking for clues. In the end, I resorted to trial and error, and with the help of network sniffing to sort out the mess.

The main challenge is to follow the instruction in section "4.5.11.5 Moving a File from One Remote Directory to Another Remote Directory on the Same FTP Server". Here are the instructions with my comments:

1. Create an empty BPEL process 
I created a synchronous BPEL, selected all default.
2. Drag and drop FTP Adapter from the Component Palette to the External
References swim lane. The Adapter Configuration Wizard Welcome page is displayed.
3. Click Next. The Service Name page is displayed.
4. Enter a service name in the Service Name field.
5. Click Next. The Adapter Interface page is displayed.- 
Default "Define from operation and schema (specified later) is checked.
6. Click Next. The FTP Server Connection page is displayed.
7. Enter the JNDI name for the FTP server, and click Next. The Operation page is
displayed. – 
JDev default might be "eis/ftp/FtpAdapter", however, SOA server default may be eis/Ftp/FtpAdapter.Look closely, you may need to change the 2nd part from "ftp" to "Ftp" (upper case "F").
8. Select Synchronous Get File, enter FTPMove in the Operation Name field, and
then click Next. The File Directories page is displayed. – 
do not overlook that "FTPMove". This is part of the black magic, it has to be precisely spelled like that, or you will be doomed. Why can't there be a drop down selection of "FTPMove"? If you have to enter specific text, what's the point of using an IDE.
9. Enter a dummy physical path for the directory for incoming files, and then click Next. The File name page is displayed.
Note: The dummy directory is not used. You must manually change the directory in a later step.
This is misleading. You can put real path in here, and the adapter uses it if you don’t overwrite it dynamically in BPEL code. Same applies to the other parameters.

10. Enter a dummy file name, and then click Next. The File Name page is displayed.
11. Click Next. The Messages page is displayed.
12. Select Native format translation is not required (Schema is opaque), and then
click Next. The Finish page is displayed.
13. Click Finish. The outbound Oracle File Adapter is now configured.
14. Drag the small triangle in the BPEL process in the Components area to the drop
zone that appears as a green triangle in FTPMove in the External References area.
The BPEL component is connected to the Oracle FTP Adapter outbound service.
15. Click File, Save All.
16. Create an invoke activity for the FTPMove service that you just created.
The next step is to modify the generated WSDL file for FTPMove service and
configure it with the new interaction specification for the move operation.
17. Open the FTPMove_ftp.jca file and modify the interaction-spec, as shown
in the following example.
You must configure the JCA file with the source and target directory and file details. You can either hardcode the source and target directory and file details in the JCA file or use header variables to populate them. In this example, header variables are used.
<adapter-config name="FTPMove" adapter="Ftp Adapter"
xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
<connection-factory location="eis/Ftp/FtpAdapter" adapterRef=""/>
<endpoint-interaction portType="FTPMove_ptt" operation="FTPMove">
<interaction-spec
className="oracle.tip.adapter.ftp.outbound.FTPIoInteractionSpec">
<property name="SourcePhysicalDirectory" value="foo1"/>
<property name="SourceFileName" value="bar1"/>
<property name="TargetPhysicalDirectory" value="foo2"/>
<property name="TargetFileName" value="bar2"/>
<property name="Type" value="MOVE"/>
</interaction-spec>
</endpoint-interaction>
</adapter-config>

18. Map the actual directory and file names to the source and target file parameters by performing the following procedure:
a. Create 4 string variables with appropriate names. You must populate these variables with the source and target directory details. The BPEL source view shows you this:
<variable name="sourceDirectory" type="xsd:string"/>
<variable name="sourceFileName" type="xsd:string"/>
<variable name="targetDirectory" type="xsd:string"/>
<variable name="targetFileName" type="xsd:string"/>

b. Create an assign activity to assign values to sourceDirectory,
sourceFileName, targetDirectory, and targetFileName variables.
The assign operation appears in the BPEL source view as in the following example:
 I use BPEL2.0, my assignment looks like:
   <assign name="Assign1">
      <copy>
        <from>'/test/input'</from>
        <to>$sourceDirectory</to>
      </copy>
      <copy>
        <from>'input.txt'</from>
        <to>$sourceFileName</to>
      </copy>
      <copy>
        <from>'/test/output</from>
        <to>$targetDirectory</to>
      </copy>
      <copy>
        <from>'output.txt'</from>
        <to>$targetFileName</to>
      </copy>
</assign>
c. Pass these parameters as headers to the invoke operation. The values in these
variables override the parameters in the JCA file.

The code snippet has apparent errors, here is my code in BPEL2.0 format:
    <invoke name="Invoke" partnerLink="mvFile" portType="ns1:FTPMove_ptt"
            operation="FTPMove" inputVariable="Invoke_FTPMove_InputVariable"
            outputVariable="Invoke_FTPMove_OutputVariable"
            bpelx:invokeAsDetail="no">
      <bpelx:toProperties>
        <bpelx:toProperty name="jca.file.SourceDirectory"
                          variable="sourceDirectory"/>
        <bpelx:toProperty name="jca.file.SourceFileName"
                          variable="sourceFileName"/>
        <bpelx:toProperty name="jca.file.TargetDirectory"
                          variable="targetDirectory"/>
        <bpelx:toProperty name="jca.file.TargetFileName"
                          variable="targetFileName"/>
      </bpelx:toProperties>
    </invoke>

A few notes here, do NOT use the UI and select the “properties”, these properties do not show up in the list, AND the properties show up in the list do NOT work L (for example, jca.ftp.FileName would look like a perfect choice, but it is misleading, it does not work!).
Additionally, there are “To” and “From” properties I can’t find any documents about these choices.  I finally figured out (network sniffing) that “to” appears to be what used to be called “input” properties. I would assume “from” would be the same as old “output” properties.


19. Finally, add an initial receive or pick activity.
You have completed moving or renaming a file from a remote directory to another remote directory on the same FTP server.
If you created a default BPEL, you may already have your receive activity. Run your test, that should take move “/test/input/input.txt” to “/test/output/output.txt” on the same FTP server.

If you want to move the file from remote “/test/input/input.txt” to local “/test/output/output.txt”, you need to add this property to the jac file:

<property name="TargetIsRemote" value="false"/>

6 comments:

  1. Hello,

    Is it possible to implement FTPMOVE operation using OSB?

    Mihir

    ReplyDelete
  2. Hello,

    Thanks for the input. We have a requirement where we need to read files from a remote server (the file directory and file name would be provided at run time). Can you help me with the same?

    Thanks Nids.

    ReplyDelete
  3. Nice post. Like it , thanks very much. One question. What if we want to move all the files in a directory. What will be the TargetFileName?

    ReplyDelete
  4. Can you please help to delete the files from target FTP location.
    we have a requirement where we need to check the file existence on target FTP location, if file exists then we need to delete and replace with new file. Is this possible..

    ReplyDelete