SOA Suite 11g Native Format Builder Complex Format Example
- by bob.webster
This rather long posting details the steps required to process a grouping of fixed length records using Format Builder. If it’s 10 pm and you’re feeling beat you might want to leave this until tomorrow. But if it’s 10 pm and you need to get a Format Builder Complex template done, read on… The goal is to process individual orders from a file using the 11g File Adapter and Format Builder Sample Data =========== 001Square Widget 0245.98 102Triagular Widget 1120.00 403Circular Widget 0099.45 ORD8898302/01/2011 301Hexagon Widget 1150.98 ORD6735502/01/2011 The records are fixed length records representing a number of logical Order records. Each order record consists of a number of item records starting with a 3 digit number, followed by a single Summary Record which starts with the constant ORD. How can this file be processed so that the first poll returns the first order? 001Square Widget 0245.98 102Triagular Widget 1120.00 403Circular Widget 0099.45 ORD8898302/01/2011 And the second poll returns the second order? 301Hexagon Widget 1150.98 ORD6735502/01/2011 Note: if you need more than one order per poll, that’s also possible, see the “Multiple Messages” field in the “File Adapter Step 6 of 9” snapshot further down. To follow along with this example you will need - Studio Edition Version 11.1.1.4.0 with the - SOA Extension for JDeveloper 11.1.1.4.0 installed Both can be downloaded from here: http://www.oracle.com/technetwork/middleware/soasuite/downloads/index.html You will not need a running WebLogic Server domain to complete the steps and Format Builder tests in this article. Start with a SOA Composite containing a File Adapter The Format Builder is part of the File Adapter so start by creating a new SOA Project and Composite. Here is a quick summary for those not familiar with these steps - Start JDeveloper - From the Main Menu choose File->New - In the New Gallery window that opens Expand the “General” category and Select the Applications node. Then choose SOA Application from the Items section on the right. Finally press the OK button. - In Step 1 of the “Create SOA Application wizard” that appears enter an Application Name and an Directory of your choice, then press the Next button. - In Step 2 of the “Create SOA Application wizard”, press the Next button leaving all entries as defaulted. - In Step 3 of the “Create SOA Application wizard”, Enter a composite name of your choice and Press the Finish Button These steps result in a new Application and SOA Project. The SOA Project contains a composite.xml file which is opened and shown below. For our example we have not defined a Mediator or a BPEL process to minimize the steps, but one or the other would eventually be needed to use the File Adapter we are about to create. Drag and drop the File Adapter icon from the Component Pallette onto either the LEFT side of the diagram under “Exposed Services” or the right side under “External References”. (See the Green Circle in the image below). Placing the adapter on the left side would indicate the file being processed is inbound to the composite, if the adapter is placed on the right side then the data is outbound to a file. Note that the same Format Builder definition can be used in both directions. For example we could use the format with a File Adapter on the left side of the composite to parse fixed data into XML, modify the data in our Composite or BPEL process and then use the same Format Builder definition with a File adapter on the right side of the composite to write the data back out in the same fixed data format When the File Adapter is dropped on the Composite the File Adapter Wizard Appears. Skip Past the first page, Step 1 of 9 by pressing the Next button. In Step 2 enter a service name of your choice as shown below, then press Next When the Native Format Builder appears, skip the welcome page by pressing next. Also press the Next button to accept the settings on Step 3 of 9 On Step 4, select Read File and press the Next button as shown below. On Step 5 enter a directory that will contain a file with the input data, then Press the Next button as shown below. In step 6, enter *.txt or another file format to select input files from the input directory mentioned in step 5. ALSO check the “Files contain Multiple Messages” checkbox and set the “Publish Messages in Batches of” field to 1. The value can be set higher to increase the number of logical order group records returned on each poll of the file adapter. In other words, it determines the number of Orders that will be sent to each instance of a Mediator or Composite processing using the File Adapter. Skip Step 7 by pressing the Next button In Step 8 press the Gear Icon on the right side to load the Native Format Builder. Native Format Builder appears Before diving into the format, here is an overview of the process. Approach - Bottom up Assuming an Order is a grouping of item records and a summary record…. - Define a separate Complex Type for each Record Type found in the group. (One for itemRecord and one for summaryRecord) - Define a Complex Type to contain the Group of Record types defined above (LogicalOrderRecord) - Define a top level element to represent an order. (order) The order element will be of type LogicalOrderRecord Defining the Format In Step 1 select “Create new” and “Complex Type” and “Next” In Step two browse to and select a file containing the test data shown at the start of this article. A link is provided at the end of this article to download a file containing the test data. Press the Next button In Step 3 Complex types must be define for each type of input record. Select the Root-Element and Click on the Add Complex Type icon This creates a new empty complex type definition shown below. The fastest way to create the definition is to highlight the first line of the Sample File data and drag the line onto the <new_complex_type> Format Builder introspects the data and provides a grid to define additional fields. Change the “Complex Type Name” to “itemRecord” Then click on the ruler to indicate the position of fixed columns. Drag the red triangle icons to the exact columns if necessary. Double click on an existing red triangle to remove an unwanted entry. In the case below fields are define in columns 0-3, 4-28, 29-eol When the field definitions are correct, press the “Generate Fields” button. Field entries named C1, C2 and C3 will be created as shown below. Click on the field names and rename them from C1->itemNum, C2->itemDesc and C3->itemCost When all the fields are correctly defined press OK to save the complex type. Next, the process is repeated to define a Complex Type for the SummaryRecord. Select the Root-Element in the schema tree and press the new complex type icon Then highlight and drag the Summary Record from the sample data onto the <new_complex_type> Change the complex type name to “summaryRecord” Mark the fixed fields for Order Number and Order Date. Press the Generate Fields button and rename C1 and C2 to itemNum and orderDate respectively. The last complex type to be defined is a type to hold the group of items and the summary record. Select the Root-Element in the schema tree and click the new complex type icon Select the “<new_complex_type>” entry and click the pencil icon On the Complex Type Details page change the name and type of each input field. Change line 1 to be named item and set the Type to “itemRecord” Change line 2 to be named summary and set the Type to “summaryRecord” We also need to indicate that itemRecords repeat in the input file. Click the pencil icon at the right side of the item line. On the Edit Details page change the “Max Occurs” entry from 1 to UNBOUNDED. We also need to indicate how to identify an itemRecord. Since each item record has “.” in column 32 we can use this fact to differentiate an item record from a summary record. Change the “Look Ahead” field to value 32 and enter a period in the “Look For” field Press the OK button to save entry. Finally, its time to create a top level element to represent an order. Select the “Root-Element” in the schema tree and press the New element icon Click on the <new_element> and press the pencil icon. Set the Element Name to “order” and change the Data Type to “logicalOrderRecord” Press the OK button to save the element definition. The final definition should match the screenshot below. Press the Next Button to view the definition source. Press the Test Button to test the definition Press the Green Triangle Icon to run the test. And we are presented with an unwelcome error. The error states that the processor ran out of data while working through the definition. The processor was unable to differentiate between itemRecords and summaryRecords and therefore treated the entire file as a list of itemRecords. At end of file, the “summary” portion of the logicalOrderRecord remained unprocessed but mandatory. This root cause of this error is the loss of our “lookAhead” definition used to identify itemRecords. This appears to be a bug in the Native Format Builder 11.1.1.4.0 Luckily, a simple workaround exists. Press the Cancel button and return to the “Step 4 of 4” Window. Manually add nxsd:lookAhead="32" nxsd:lookFor="." attributes after the maxOccurs attribute of the item element. as shown in the highlighted text below. When the lookAhead and lookFor attributes have been added Press the Test button and on the Test page press the Green Triangle. The test is now successful, the first order in the file is returned by the File Adapter. Below is a complete listing of the Result XML from the right column of the screen above Try running it The downloaded input test file and completed schema file can be used for testing without following all the Native Format Builder steps in this example. Use the following link to download a file containing the sample data. Download Sample Input Data This is the best approach rather than cutting and pasting the input data at the top of the article. Since the data is fixed length it’s very important to watch out for trailing spaces in the data and to ensure an eol character at the end of every line. The download file is correctly formatted. The final schema definition can be downloaded at the following link Download Completed Schema Definition - Save the inputData.txt file to a known location like the xsd folder in your project. - Save the inputData_6.xsd file to the xsd folder in your project. - At step 1 in the Native Format Builder wizard (as shown above) check the “Edit existing” radio button, then browse and select the inputData_6.xsd file - At step 2 of the Format Builder configuration Wizard (as shown above) supply the path and filename for the inputData.txt file. - You can then proceed to the test page and run a test. - Remember the wizard bug will drop the lookAhead and lookFor attributes, you will need to manually add nxsd:lookAhead="32" nxsd:lookFor="." after the maxOccurs attribute of the item element in the LogicalOrderRecord Complex Type. (as shown above) Good Luck with your Format Project
