Conversion of BizTalk Projects to Use the New WCF-SAP Adaptor
- by Geordie
We are in the process of upgrading our BizTalk Environment from BizTalk 2006 R2 to BizTalk 2010. The SAP adaptor in BizTalk 2010 is an all new and more powerful WCF-SAP adaptor. When my colleagues tested out the new adaptor they discovered that the format of the data extracted from SAP was not identical to the old adaptor. This is not a big deal if the structure of the messages from SAP is simple. In this case we were receiving the delivery and invoice iDocs. Both these structures are complex especially the delivery document. Over the past few years I have tweaked the delivery mapping to remove bugs from original mapping. The idea of redoing these maps did not appeal and due to the current work load was not even an option. I opted for a rather crude alternative of pulling in the iDoc in the new typed format and then adding a static map at the start of the orchestration to convert the data to the old schema.
Note WCF-SAP data formats (on the binding tab of the configuration dialog box is the ‘RecieiveIdocFormat’ field):
Typed: Returns a XML document with the hierarchy represented in XML and all fields being represented by XML tags.
RFC: Returns an XML document with the hierarchy represented in XML but the iDoc lines in flat file format.
String: This returns the iDoc in a format that is closest to the original flat file format but is still wrapped with some top level XML tags. The files also contained some strange characters at the end of each line.
I started with the invoice document and it was quite straight forward to add the mapping but this is where my problems started. The orchestrations for these documents are dynamic and so require the identity of the partner to be able to correctly configure the orchestration. The partner identity is in the EDI_DC40 segment of the iDoc. In the old project the RECPRN node of the segment was promoted. The code to set a variable to the partner ID was now failing. After lot of head scratching I discovered the problem was due to the addition of Namespaces to the fields in the EDI_DC40 segment. To overcome this I needed to use an xPath query with a Namespace Manager. This had to be done in custom code.
I now tried to repeat the process with the delivery document. Unfortunately when we tried to get sample typed data from SAP an exception was thrown.
The adapter "WCF-SAP" raised an error message. Details "Microsoft.ServiceModel.Channels.Common.XmlReaderGenerationException: The segment or group definition E2EDKA1001 was not found in the IDoc metadata. The UniqueId of the IDoc type is: IDOCTYP/3/DESADV01/ZASNEXT1/640. For Receive operations, the SAP adapter does not support unreleased segments.
Our guess is that when the WCF-SAP adaptor tries to down load the data it retrieves a data schema from SAP. For some reason the schema does not match the data. This may be due to the version of SAP we are running or due to a customization. Either way resolving this problem did not look easy.
When doing some research on this problem I found an article showing me how to get the data from SAP using the WCF-SAP adaptor without any XML tags.
http://blogs.msdn.com/b/adapters/archive/2007/10/05/receiving-idocs-getting-the-raw-idoc-data.aspx
Reproduction of Mustansir blog:
Since the WCF based SAP Adapter is ... well, WCF based, all data flowing in and out of the adapter is encapsulated within a SOAP message. Which means there are those pesky xml tags all over the place. If you want to receive an Idoc from SAP, you can receive it in "Typed" format (in which case each column in each segment of the idoc appears within its own xml tag), or you can receive it in "String" format (in which case there are just 2 xml tags at the top, the raw xml data in string/flat file format, and the 2 closing xml tags).
In "String" format, an incoming idoc (for ORDERS05, containing 5 data records) would look like:
<ReceiveIdoc ><idocData>EDI_DC40 8000000000001064985620
E2EDK01005 800000000000106498500000100000001
E2EDK14 8000000000001064985000002000000020111000
E2EDK14 8000000000001064985000003000000020081000
E2EDK14 80000000000010649850000040000000200710
E2EDK14 80000000000010649850000050000000200600</idocData></ReceiveIdoc>
(I have trimmed part of the control record so that it fits cleanly here on one line).
Now, you're only interested in the IDOC data, and don't care much for the XML tags. It isn't that difficult to write your own pipeline component, or even some logic in the orchestration to remove the tags, right? Well, you don't need to write any extra code at all - the WCF Adapter can help you here!
During the configuration of your one-way Receive Location using WCF-Custom, navigate to the Messages tab. Under the section "Inbound BizTalk Messge Body", select the "Path" radio button, and:
(a) Enter the body path expression as:
/*[local-name()='ReceiveIdoc']/*[local-name()='idocData']
(b) Choose "String" for the Node Encoding.
What we've done is, used an XPATH to pull out the value of the "idocData" node from the XML. Your Receive Location will now emit text containing only the idoc data.
You can at this point, for example, put the Flat File Pipeline component to convert the flat text into a different xml format based on some other schema you already have, and receive your version of the xml formatted message in your orchestration.
This was potentially a much easier solution than adding the static maps to the orchestrations and overcame the issue with ‘Typed’ delivery documents. Not quite so fast…
Note: When I followed Mustansir’s blog the characters at the end of each line disappeared.
After configuring the adaptor and passing the iDoc data into the original flat file receive pipelines I was receiving exceptions.
There was a failure executing the receive pipeline: "PAPINETPipelines.DeliveryFlatFileReceive, CustomerIntegration2.PAPINET.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=4ca3635fbf092bbb" Source: "Pipeline " Receive Port: "recSAP_Delivery" URI: "D:\CustomerIntegration2\SAP\Delivery\*.xml" Reason: An error occurred when parsing the incoming document: "Unexpected data found while looking for:
'Z2EDPZ7'
The current definition being parsed is E2EDP07GRP. The stream offset where the error occured is 8859. The line number where the error occured is 23. The column where the error occured is 0.".
Although the new flat file looked the same as the old one there was a differences. In the original file all lines in the document were exactly 1064 character long. In the new file all lines were truncated to the last alphanumeric character.
The final piece of the puzzle was to add a custom pipeline component to pad all the lines to 1064 characters. This component was added to the decode node of the custom delivery and invoice flat file disassembler pipelines.
Execute method of the custom pipeline component:
public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
//Convert Stream to a string
Stream s = null;
IBaseMessagePart bodyPart = inmsg.BodyPart;
// NOTE inmsg.BodyPart.Data is implemented only as a setter in the http adapter API and a
//getter and setter for the file adapter. Use GetOriginalDataStream to get data instead.
if (bodyPart != null)
s = bodyPart.GetOriginalDataStream();
string newMsg = string.Empty;
string strLine;
try
{
StreamReader sr = new StreamReader(s);
strLine = sr.ReadLine();
while (strLine != null)
{
//Execute padding code
if (strLine != null)
strLine = strLine.PadRight(1064, ' ') + "\r\n";
newMsg += strLine;
strLine = sr.ReadLine();
}
sr.Close();
}
catch (IOException ex)
{
throw new Exception("Error occured trying to pad the message to 1064 charactors");
}
//Convert back to stream and set to Data property
inmsg.BodyPart.Data = new MemoryStream(Encoding.UTF8.GetBytes(newMsg)); ;
//reset the position of the stream to zero
inmsg.BodyPart.Data.Position = 0;
return inmsg;
}
