I’m currently migrating a BizTalk Server infrastructure from BizTalk Server 2013 R2 to BizTalk Server 2016, this is a simple task to do in terms of BizTalk Server Applications, or BizTalk Server solutions. However, migrating or in fact, creating a new environment always brings challenges in terms of connectivity and/or proper configurations. SAP.Middleware.Connector.RfcCommunicationException: LOCATION CPIC (TCP/IP) on local host BTSSERVERNAME with Unicode, ERROR service ‘?’ unknown error is one of this cases!

We indeed did more than a simple migration, we toked this opportunity to improve our solutions and/or BizTalk Server environment to update our applications to the new the new available features. In terms of WCF-SAP Adapter, we are no longer using traditional RFC and we will be starting using SAP .NET Connector (NCo). Despite WCF-SAP adapter will continue to support both the RFC SDK and the SAP .NET Connector, SAP has announced deprecation of its classic RFC SDK (no longer be supported after March 31, 2016) and because of that, it shouldn’t be used anymore.

While trying to connect our BizTalk Server Receive Location to SAP in order to listen to incoming messages, we initial got this error message:

The Messaging Engine failed to add a receive location “IN_WCF_SAP_LISTENER_BTS” with URL “sap://CLIENT=400;LANG=EN;@a/SERVERNAME/00?ListenerGwServ=sapgw00&ListenerGwHost=LISTENERHOST&ListenerProgramId=BTS_LIST&RfcSdkTrace=False&AbapDebug=False” to the adapter “WCF-Custom”. Reason: “SAP.Middleware.Connector.RfcCommunicationException:

LOCATION CPIC (TCP/IP) on local host BTSSERVERNAME with Unicode

ERROR service ‘?’ unknown

TIME Fri Nov 23 13:47:22 2018

RELEASE 721

COMPONENT NI (network interface)

VERSION 40

RC -3

DETAIL NiErrSet

COUNTER 3

at SAP.Middleware.Connector.RfcServer.DoStart(Boolean logErrors)

at SAP.Middleware.Connector.RfcServer.Start()

at Microsoft.Adapters.SAP.NCo.RfcServerConnection.Open(RfcClientConnection conn)

at Microsoft.Adapters.SAP.SAPInboundContract.InitializeRfcServerConnectionNCo(String connectionArguments)

at Microsoft.Adapters.SAP.SAPInboundContract.StartListener(String[] listenerActions, TimeSpan timeout)

at Microsoft.ServiceModel.Channels.Common.Channels.AdapterChannelListener`1.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint.Enable()

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint..ctor(BizTalkEndpointContext endpointContext, IBTTransportProxy transportProxy, ControlledTermination control)

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiver`2.AddReceiveEndpoint(String url, IPropertyBag adapterConfig, IPropertyBag bizTalkConfig)”.

The receive location “IN_WCF_SAP_LISTENER_BTS_ DV” with URL “sap://CLIENT=400;LANG=EN;@a/SERVERNAME/00?ListenerGwServ=sapgw00&ListenerGwHost=LISTENERHOST&ListenerProgramId=BTS_LIST&RfcSdkTrace=False&AbapDebug=False” is shutting down. Details:”The Messaging Engine failed while notifying an adapter of its configuration. “.

Cause

The first thing you need to know is that this error is not related to the fact that we are now using the SAP .NET Connector (NCo). This error will still occur if you change back to the classic RFC.

The basic steps if of course to see if we have connectivity and proper access to SAP, and because we were in DEV environment a good way to test this is to Add WCF-SAP Adapter Metadata to a BizTalk Project inside Visual Studio

• In your Visual Studio BizTalk project, in Solution Explorer, right-click your project, click Add, and then click Add Generated Items.

You will have the same look and fill as creating a Receive Port on the Administrative Console with the advantage of being able to navigate the under the SAP object structure. This way you can test connectivity and access to the proper resources.

In our case, everything worked well. We were able to connect and generate the schemas.

However, all of this is not enough. For the WCF-SAP adapter to work properly in runtime you also need to configure, somewhere in the BizTalk Server machine the TCP port where the SAP adapter will be looking for these connections, because this property is not exposed through the BizTalk Server WCF-SAP Adapter GUI port (send or receive) configuration.

In our case, the .NET Connector need to know the port of the message server. This is also very common to happen in load balancing situations.

Solution

In order to specify this TCP port, you need to modify the ‘services’ file in each BizTalk Server machine, normally in:

• C:Windowssystem32driversetcservices

To include the following entry:

MSHOST 1234/tcp # SAP ENV System Message Server Port

Where:

• MSHOST is your Gateway server, in our case sapgw00
• And 1234 is your message server port, in our case: 3300

Just for curiosity, BizTalk isn’t the only that will use these values and this ‘services’ file. SAPGUI will also use these same values when connecting to SAP systems.

After we did this configuration, we were able to connect and start receiving messages from our SAP system.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post BizTalk Server WCF-SAP Adapter: SAP.Middleware.Connector.RfcCommunicationException: LOCATION CPIC (TCP/IP) on local host BTSSERVERNAME with Unicode, ERROR service ‘?’ unknown appeared first on BizTalkGurus.

In this blog post, we will explain how different stakeholders within your organization can be involved in operating the BizTalk environment and managing BizTalk interfaces, by smartly, but safely deploy BizTalk360.

The BizTalk Administrators – the primary users of BizTalk360

Primarily, BizTalk360 will mainly be used by BizTalk administrators. They will use the product in their day to day operations and will probably be the ones who firstly receive notifications of threshold violations and daily health check reports.

These administrators are responsible for the day to day operations of the BizTalk group(s) and will be the main group of users of BizTalk360. Therefore, they need all the authorizations so they will have Super User accounts in BizTalk360. They will also be responsible for creating and maintaining User Access Policies.

After installation of the product, they will set up alarms for the BizTalk platform and the BizTalk applications it contains and receive notifications which are generated by the alarms.

The way they configure monitoring of their BizTalk environments with BizTalk360, might evolve like this:

• Basic Threshold/Health Check monitoring with email notification
  • BizTalk Platform – Monitor platform components like Host Instances, SQL Server jobs, NT Services, BizTalk Health Monitor
  • BizTalk Applications – Monitor application artifacts like Receive Locations, Orchestrations, Send Ports, Instance states
• Advanced Environment Monitoring
  • Endpoints – Monitor web services, queues, file shares, FTP sites, Azure services
  • Data monitoring – Monitor interface processing, automate resuming and terminating of service instances
  • Enterprise notification channels – Receive notifications via HP Operations Manager, Slack, ServiceNow, Microsoft Teams, Webhook, PowerShell

In case of any issues, the (team of) BizTalk administrators will fix these issues themselves, or they contact other stakeholders to discuss how the issues need to be fixed.

Improve your business processes by extending the reach of BizTalk360

The power of BizTalk360 lies in the fact that it provides a rich user interface with many different dashboards for many different purposes, both from a technical and a functional perspective. In contrast with the tools which come out-of-the-box with BizTalk Server, all the capabilities in BizTalk360 are protected by User Access Policies and Auditing. This enables you to give any person exactly that set of permissions that they need to be able to do their job.

When the BizTalk administrators feel comfortable with the product and with BizTalk administration in general, or when the organization requires it, BizTalk360 can be deployed to other parts of the IT department or business departments.

Involve stakeholders by sending notifications and providing access to BizTalk360

By sending alerts to stakeholders directly, you improve information management as your BizTalk Administrators don’t need to send separate emails or contact these stakeholders based on issues that have occurred.
Although in many cases, email will be the primary means of sending notifications from BizTalk360, the product also contains the following Notification channels:

• Microsoft Teams
• ServiceNow
• Slack
• PowerShell
• Webhook

These channels can be configured on each BizTalk360 alarm and enable you to receive the notifications where it is most convenient for you. You can also use a simple to use SDK to develop your own custom Notification channel. Read more about Notification channels in our Documentation Portal.

But you can go even further, BizTalk360 allows you to give stakeholders secure and limited access to BizTalk360. This kind of access can reach from read-only access to particular parts of the BizTalk platform to the capability to act on certain issues.

Deployment of BizTalk360 to your organization can be done in the pace you and your organization feel comfortable with. By sending notifications to the stakeholders and provide access to BizTalk360, you can keep the stakeholders informed of any issues, improve the availability of your business processes and meanwhile spreading the workload between all the stakeholders.

Think of the following scenarios which could be achieved:

• Sending alerts to the help desk – This enables the help desk engineers to analyze any issues at hand and take countermeasures
• Informing your administrators directly of any issues – Think of your System Administrators or DBA’s who receive alerts about server or database issues
• Automated creation of support tickets in your ticketing system – This takes away the need to have an administrator to do it manually
• Inform business users of issues – Think of batches which are not processed (non-events) or faulty processing of their messages
• Provide access to dashboards/portals – Give your stakeholders access to portals like the BAM portal, ESB portal, Business Rules Composer, (Data) Monitoring/Analytics dashboard, etc. etc.

Identifying stakeholders

When it comes to deploying BizTalk360 throughout an organization, you can identify the following roles:

• IT Support personnel
• SQL Server DBA’s, System Administrators and BizTalk developers
• Business Users

Let’s have a look at the roles mentioned above and their potential responsibilities with regards to BizTalk Server/BizTalk360.

IT Support personnel

To have a good eye watching on your BizTalk environment, you could involve the IT Support staff of your organization. Depending on your organization, they might be available 24/7. So it makes sense to send notifications of unexpected behavior happening on the platform level, as they might be able to act before BizTalk Administrators can. Besides sending them notifications, you could give the IT Support staff read-only access to BizTalk, so they can explore any issues and maybe help you fix them, while you might be at home, having weekend.

SQL Server DBA’s, System Administrators and BizTalk developers

Since operating BizTalk involves more than just the BizTalk Server product itself, but also components like SQL Server databases and all kind of Windows Server components, you might consider involving System Administrators and SQL Server DBA’s in BizTalk operations by sending them notifications which might be relevant for them.

A few examples are:

• SQL Server Administrators can receive notifications in case the BizTalk related SQL Server jobs fail
• System Administrators can receive notifications when the BizTalk servers are running out of disk space
• BizTalk Developers can receive notifications in case issues arise with upgraded or newly deployed BizTalk applications

As a next step, you might give these administrators and developers access to BizTalk360. This ranges from providing read-only access to particular features to giving them full operational access to the parts of their interest.

If you give any of the stakeholders access to BizTalk360, it is also helpful to use the BizTalk360 Knowledge Base. When you properly maintain the Knowledge Base, your support people will have the help they need at their fingertips and be able to solve known issues quickly.
The Knowledge Base associates Knowledge Base articles to Service Instances, EventLog entries, ESB Exceptions, and Throttling data. Read more about its capabilities in our Documentation portal.

You might consider providing System Administrators, SQL Server DBAs and BizTalk developers with the following authorizations:

• System Administrators
  • Advanced Event Viewer
  • BizTalk Health Monitor
  • Topology
  • Host/Host Instances
  • BizTalk/SQL Servers
  • Manage BizTalk/SQL NT Services
  • Tracking Manager
  • Adapters
  • Backup/DR Visualizer
• SQL Server DBA’s
  • Secure SQL Queries
  • Advanced Event Viewer
  • BizTalk Health Monitor
  • Message Boxes
  • SQL Servers
  • Manage SQL NT Services
  • SQL Server Instances
  • Manage SQL Jobs
  • Backup/DR Visualizer
• BizTalk developers
  • MessageBox queries (with/without access to content/context)
  • Tracking Queries
  • Advanced Event Viewer
  • Tracking Manager
  • Secure SQL Queries
  • BAM portal
  • ESB Exception portal
  • EDI Reports
  • Messaging Patterns

Business Users

This category of users might exist both inside as outside your organization. Depending on that, it will differ how they are involved in managing the interfaces. Normally, they will not take part in managing the BizTalk platform itself.

Internal business users can be informed about the processing, by providing them with notifications of disruptions in the processing of their interfaces, i.e., inform them of suspended instances, transmission failures, and failing process monitoring.

When you want to give business users access to BizTalk360, you can think of the following features:

– Specific BizTalk applications
– Message Box (Queries)
– Graphical Flow (Tracking)
– Business Rules Composer
– EDI Reports, parties and agreements
– ESB Portal
– Business Activity Monitoring
– Messages Content/Context
– Secure SQL Queries

In case external business users are involved in certain interfaces, you might send them the same notifications as internal business users. As the external business users will be outside your organization, you normally will not give them access to the BizTalk360 User Interface.

Conclusion

We often see, that BizTalk Server is considered as a black box and deep BizTalk knowledge is needed to be able to find out what’s all happening inside that box. By using BizTalk360, we make it easy to gain that insight, even with little BizTalk expertise. Furthermore, by deploying BizTalk360 outside the BizTalk administrators team, you can give your middleware a face and achieve much more transparency about all the processing taking place in your BizTalk environment.

By using BizTalk360 outside the admin team, it is easier to notify other stakeholders by sending them notifications directly from BizTalk360. Even further, besides sending notifications to these stakeholders within (or outside) your organization, you can give people (limited) access to BizTalk360. This way they can view for themselves how all the processing is taking place or check the wellbeing of the environment, without the need of contacting the BizTalk Administrators team.

All in all, it must be clear, that by extending the use of BizTalk360 outside the admin team, you will have a better ROI of the product. If you would like to know more about how BizTalk360 can help your organization to manage your BizTalk Server middleware platform, feel free to contact us.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt

The post Deploying BizTalk360 within your organization appeared first on BizTalkGurus.

Following my last post and continuing on the pain points of BizTalk Server infrastructure tasks migration, I end up founding another interesting error: WCF-Custom Reason: System.ArgumentException: An item with the same key has already been added.

Again, we indeed did more than a simple migration, we take this opportunity to improve our solutions and/or BizTalk Server environment to update our applications to the new the new available features. In terms of WCF-SAP Adapter, we are no longer using traditional RFC and we will be starting using SAP .NET Connector (NCo). Despite WCF-SAP adapter will continue to support both the RFC SDK and the SAP .NET Connector, SAP has announced deprecation of its classic RFC SDK (no longer be supported after March 31, 2016) and because of that, it should be used anymore.

While trying to connect our BizTalk Server Receive Location to SAP in order to listen to incoming messages, we initially got this error message:

The Messaging Engine failed to add a receive location “WcfReceiveLocation_SAPBinding_IdocDELVRY05V3R700_Custom” with URL “sap://CLIENT=400;LANG=EN;@a/SERVERNAME/00?ListenerGwServ=sapgw00&ListenerGwHost=LISTENERHOST&ListenerProgramId=BTS_LIST&RfcSdkTrace=False&AbapDebug=False ” to the adapter “WCF-Custom”. Reason: “System.ArgumentException: An item with the same key has already been added.

at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)

at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)

at Microsoft.Adapters.SAP.NCo.RfcServerConnection.Open(RfcClientConnection conn)

at Microsoft.Adapters.SAP.SAPInboundContract.InitializeRfcServerConnectionNCo(String connectionArguments)

at Microsoft.Adapters.SAP.SAPInboundContract.StartListener(String[] listenerActions, TimeSpan timeout)

at Microsoft.ServiceModel.Channels.Common.Channels.AdapterChannelListener`1.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)

at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint.Enable()

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint..ctor(BizTalkEndpointContext endpointContext, IBTTransportProxy transportProxy, ControlledTermination control)

at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiver`2.AddReceiveEndpoint(String url, IPropertyBag adapterConfig, IPropertyBag bizTalkConfig)”.

Cause

What I found out was that this error was somehow related to the error reported in my previous post: BizTalk Server WCF-SAP Adapter: SAP.Middleware.Connector.RfcCommunicationException: LOCATION CPIC (TCP/IP) on local host BTSSERVERNAME with Unicode, ERROR service ‘?’ unknown.

The reason why I’m saying that is that each time I restarted the BizTalk Server Host Instances, the error that I got the first time I try to enable the port was this service ‘?’ unknown error. Afterward, whenever I tried to enable this port, I would receive this new error:

• System.ArgumentException: An item with the same key has already been added

Why this is happening for me is a mix of a small bug and consistency, let me explain better:

• Why consistency? This because in fact, you cannot have two SAP Receive Locations listening to the same SAP endpoint/SAP Listener Program Id.
  • Actually, this should be when this error would normally occur.
• Why a small bug? Because in fact there isn’t any Receive location enable for that specific SAP endpoint/SAP Listener Program Id. What is happening is that despite the error and the receive location became automatically disable it somehow stores in memory that SAP endpoint/SAP Listener Program Id

Solution

The solution is very simple and stupid:

• Restart the host instance that is running that Receive Location

After that, this problem goes away, but make sure you solve all the problems, or this may happen again.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post BizTalk Server WCF-SAP Adapter: System.ArgumentException: An item with the same key has already been added appeared first on BizTalkGurus.

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post Microsoft Integration Weekly Update: December 03, 2018 appeared first on BizTalkGurus.

I was thinking in calling these post series “Note to myself”, only not to be always looking for these solutions whenever these problems not related to BizTalk Server or other integration products or technologies occur. But I decided to call it “A fish out of water” that basically is what I feel sometimes. And to start this post series I choose an occurrent “issue” that happens a lot when I’m trying to do something whit in SQL Server Management Console: Saving changes is not permitted.

The full warning message would be:

Saving changes is not permitted. The changes that you have made require the following tables to be dropped and re-created. You have either made changes to a table that can’t be re-created or enabled the option Prevent saving changes that require the table to be re-created.

This behavior is very simple to solve and is well documented here: Error message when you try to save a table in SQL Server: “Saving changes is not permitted”. Once again, this is just for helping me not always to look at this “problem” again. My Blog is the first place I look, it is my personal notes.

Cause

This problem occurs when the Prevent saving changes that require the table re-creation option is enabled, and you make one or more of the following changes to the table:

• You change the Allow Nulls setting for a column.
• You reorder columns in the table.
• You change the column data type.
• You add a new column.

When you change a table so that you alter the metadata structure of the table, and then you save the table, the table must be re-created based on these changes. This may result in the loss of metadata and in a direct loss of data during the re-creation of the table. If you enable the Prevent saving changes that require the table re-creation option in the Designer section of the SQL Server Management Studio (SSMS) Options window, you receive the error message: Saving changes is not permitted. The changes that you have made require the following tables to be dropped and re-created.

Solution

To change the Prevent saving changes that require the table re-creation option, follow these steps:

• Open SQL Server Management Studio (SSMS).
• On the Tools menu, click Options.
• In the navigation pane of the Options window, click Designers.
• Uncheck (clear) the Prevent saving changes that require the table re-creation check box, and then click OK.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post A fish out of water: Saving changes is not permitted. The changes that you have made require the following tables to be dropped and re-created appeared first on BizTalkGurus.

Happy to introduce my first guest blog author: Peter Vervoorn from Virtual Green. Peter is a very experienced integration specialist. He has been working in ICT since 1995 and was a co-founder of Axon Olympus (now part of the Codit Group). Until his sabbatical in 2014 he headed the team of consultants at Axon Olympus as Technical Director. During his sabbatical in Thailand, he lost over 50 kilos and currently, he divides his time between Thailand and The Netherlands. He is a specialist with (Microsoft) Integration Tools and Technologies like BizTalk Server, Windows 10 IoT, Node-RED and is also involved in developing industrial automation prototypes.

Peter reach me with this funny, a bit unusual but quite interesting scenario: Calling multiple Disassemblers in a Receive pipeline, and I challenge him to be my first guest blog author on my blog. Challenge that he gladly accepted.

Situation

You receive a zip file, containing several files to extract. The extracted files should be disassembled too. (Possibly because they are in flat file format, or you want to call the XmlDisassembler to set the message type.)

Problem

Although the disassemble stage in the receive pipeline can contain multiple components, only the first component (matching the message) will be executed.

Solution

Create a new disassembler component, which will handle calling the sequential disassembler components.

The implementation of the Disassemble method is very easy, just call the initial component in the sequence.

The GetNext method is where it becomes a bit more interesting. Here we would have to extract all the messages from the first stage and feed them to the second stage. Note that it is not possible to create a single instance for the second stage and keep feeding it messages; each message requires its own instance of the component.

To do this, we have to create a new instance of the second stage component. Then we set the required properties (e.g., with values from the property bag.) Then we call the Disassemble method on the component. Next, we call GetNext to retrieve all messages and queue them.

The remainder of the implementation is for dequeening the messages.

Code

In the sample code below, all the usual methods (Load, Save, Validate, etc.) are not shown. Only the two relevant methods are shown. Also, all the plumbing has been removed for brevity/clarity.

[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]
[System.Runtime.InteropServices.Guid("YOUR-GUID-HERE")]
public class MultiDisassembler : IBaseComponent, IPersistPropertyBag, IComponentUI, IDisassemblerComponent
{
    private ExtractorComp extractPC = new ExtractorComp();
    private Queue<IBaseMessage> messages = null;

    public void Disassemble(IPipelineContext pContext, IBaseMessage pInMsg)
    { extractPC.Disassemble(pContext, pInMsg); }

    public IBaseMessage GetNext(IPipelineContext pContext)
    {
        if (messages == null)
        {
            messages = new Queue<IBaseMessage>();
            IBaseMessage msgS1 = null;
            while ((msgS1 = extractPC.GetNext(pContext)) != null)
            {
                XmlDasmComp  xmlDasmPC = NewXmlDasmWithPropertiesSet();
                xmlDasmPC.Disassemble(pContext, msgS1);
                IBaseMessage msgS2 = null;
                while ((msgS2 = xmlDasmPC.GetNext(pContext)) != null)
                { messages.Enqueue(msgS2); }
            }
        }

        if (messages.Count > 0)
        { return messages.Dequeue(); }
        return null;
    }
    //...
    //Missing Code
}

Tips

• You can add more stages, as required;
• If necessary, you can inspect a message and choose to either enqueue it directly or use a different component to disassemble that message.
  (An example would be a zip file containing a mix of XML files and nested zip files)
• In the above sample, all messages are queued when the GetNext method is called the first time. Another possibility is getting the next message in a just-in-time fashion. This requires a bit of additional work as you have to do additional housekeeping and handle possible empty messages.
• Often you see a disassembler also implementing IComponent. This is not required if only the DisassemblingParser attribute is set.

THIS SAMPLE CODE  IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

BizTalk Server Recipe: Calling multiple Disassemblers in a Receive pipeline (2 KB)
Microsoft | TechNet Gallery

BizTalk Pipeline Components Extensions Utility Pack

The pipeline component is available on BizTalk Pipeline Components Extensions Utility Pack project, that is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will provide an extension of BizTalk out-of-the-box pipeline capabilities.

The project is available on BizTalk Server Open Source Community repository on GitHub (https://github.com/BizTalkCommunity), and everyone can contribute with new pipeline components that can be extended or improve the existing BizTalk Server capabilities.

You can download BizTalk Pipeline Components Extensions Utility Pack from GitHub here:
BizTalk Pipeline Components Extensions Utility Pack
GitHub

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post Guest blog by Peter Vervoorn – Recipe: Calling multiple Disassemblers in a Receive pipeline appeared first on BizTalkGurus.

This week I installed on a BizTalk Server 2016 environment Host Integration Server 2016 (HIS), and to my surprise, I was receiving this quite unusual and annoying error: String was not recognized as a valid DateTime.

Each time I try to open the HIS Configuration Console.

Nevertheless, this error did not occur the first time I ran the configuration console, in other words, after I install the HIS, the first time I executed the Configuration console everything worked ok. Even, if I close and open again, everything was still working fine. The problem only occurred once I actually configure something.

This error only occurred If:

• I configure let’s say all the HIS features;
• Close the HIS Configuration console
• And then try to open the Configuration console again

Cause

As far as I know, there is no other reason, this is a bug!

Solution

The solution is very simple:

• Install Host Integration Server Cumulative Update 1

But in fact, there is already Host Integration Server Cumulative Update 2 available so I will recommend you installing this one instead:

After installing HIS CU2, everything worked as expected.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post Host Integration Server: String was not recognized as a valid DateTime while open HIS Configuration Console appeared first on BizTalkGurus.

There is something special when you find a solution or possible solution to an annoying error message (for not using another type of language) and this is one of these cases: There is a problem with this Windows Installer package!

This error occurred each time my team was trying to install a BizTalk Application on a non-developer environment: Test or QA. And the full error description was:

There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor.

And “Contact your support personnel or package vendor”? What a ****, It is me! I am the support personnel and package vendor/creator… you still write these kinds of error messages?

Cause

Unfortunately, this error may be caused by several reasons, but some places may tell you that this problem can be related to:

• The package is actually corrupted, you should export again and import it – not really like to a happen or solve it
• The fact of the destination location does not exist – that is not true, because the installation will create the directories;
• Invalid drive letter – again it is not true because the MSI will not go further on the wizard if that happens

To correctly diagnose the problem and find the cause of the problem you should consult the Event Viewer of the BizTalk Server in which you are trying to install the MSI. Normally you will find more details about the cause of the error.

The most common will be:

• The Installation Path that you are providing in the wizard along with the name of the assemblies will have more than 256 characters.

By default, the installation folder will be:

• C:Program Files (x86)Generated by BizTalkname of the BizTalk Application

That is:

• 46 characters for the default folder “C:Program Files (x86)Generated by BizTalk”
• Adding, more or less, 15 characters for the BizTalk Application name folder

Will give you a total of 195 characters to be used in the assembly’s names. Which normally is enough.

Solution

Once again, in this case, the solution is very simple:

• ·Change the installation folder to a small path like:
  • “C:BizTalkAppsAppName”

By providing a small path, my team was able to successfully install the MSI.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post BizTalk Server MSI installation error: There is a problem with this Windows Installer package appeared first on BizTalkGurus.

We are super excited to welcome our new release version of BizTalk360, v8.9.5. Here is a blog which explains the new feature introduction of Send Port Groups Monitoring in BizTalk360 v8.9.5.

One of the most powerful and overlooked features in BizTalk Server is Send Port Groups, helps to route a single message to more than one destination. A Send Port Group is a collection of send ports that BizTalk Server can use to send the same message to multiple destinations in one configuration. Now, you can monitor Send Port Groups in BizTalk360.

Customer Feedback

As you already know, Monitoring is the core functionality of BizTalk360. With this functionality, you can monitor all the BizTalk artefacts except the Send Port Groups in the earlier versions of BizTalk360.

In BizTalk360, we aim at improving the product and adding new features based on customer feedback and business scenarios. The Feedback portal is one such platform for the customers to provide their suggestions, which can be voted upon by other customers if they feel that these ideas fit in their business requirements as well. It’s based on the priority of voting; the features and enhancements get picked up for development. One such feedback was Send Port Groups monitoring on BizTalk applications.

Hence, we have chosen Send Port Groups monitoring for one of the features in this release.

Send Port Groups Monitoring in BizTalk360

Send Port Groups is a “key” feature of BizTalk Server’s “Publish and Subscribe” architecture, where a publisher can have more than one subscriber.

User Scenario: Let’s say you need to map an inbound XML document to three different outbound formats and send each outbound document to a different system. All that’s required is a Send Port Group and a simple Filter Expression. The Send Port Group handles the Filter Expression with each Send Port using its own map! If any of the three transmissions fail, you can resubmit the failed message without resending the other two documents. The user may have one or more Send Port Groups. It can be hard to monitor your Send Port Groups manually. Now with the support of BizTalk360, you can monitor the Send Port Groups easily! In BizTalk360, we have included the Send Port Groups for monitoring for each BizTalk Application. The user can monitor the Send Port Groups by mapping with the alarm as below.

The Send Port Group can go down for some situation; you can monitor the artifact in such situation by configuring the expected state in BizTalk360. When the current state is not equal to expected state, the user will get notified through an alert, as shown below.

The same error and status will be displayed in the Monitoring dashboard and in the Errors and warning section as seen below.

Auto Healing functionality for the Send Port Groups

By any chance, when the Send Port Group artefact goes down for some specific reason, you need not to log in to the BizTalk server to make it up again. BizTalk360 Auto Correct feature will get it back to the expected state automatically with the Auto correct option. For this, the user needs to enable Auto Correct for the configured Send Port Groups.

If in case the system is not able to Auto Correct Send Port Groups to its expected state i.e., when all the attempts to auto correct have failed, then the user will receive a down alert. For each attempt, the user will get notified with the detailed status information of auto correction operation that has happened on a Send Port Group. With this information, the user can get a clear picture of what is happening with the state of the Send Port Group.

User Scenario:  In BizTalk, the user cannot Start or Stop the Send Port Group from “unenlisted’ state when all the associated Send Ports are in “Unenlisted” state.In such scenario, there is a chance that the auto-healing may get failed to heal the Send Port Group from the “unenlisted” state to expected state and the auto correct attempts will get failed since the associated Send Ports gets “Started” or “Stopped” state.

Once all the maximum attempts failed, then the user will get intimated through down alert as “Maximum auto corrects attempts have been exhausted”.

Intelligent auditing

 BizTalk360 has very good intelligent auditing capabilities that will help administrators to find out “Who did what in the environment”. When the auto-healing is successful, it gets tracked at the Send Port Group level under “Application Activities” in Governance and Auditing section.

Live Activity Recorder

BizTalk360 has excellent Live activity tracking feature which helps you to govern user actions efficiently without compromising on regular activities. You can view the Live recording of all the user activities through the “Live Feed” on each successful auto healing of Send Port Groups. 

This feature will be available in our upcoming release of BizTalk360 v8.9.5. Stay tuned to monitor and maintain the Send Port Groups easily through auto healing functionality.

The post Introducing Send Port Groups for Monitoring in BizTalk360 appeared first on BizTalkGurus.

There are simple BizTalk Server installations and then there are some quite annoying BizTalk Server topologies and complex configurations which tend to give rise to the appearance of diverse types of errors: SQLNCLI11 ole db provider not found in the system. It’s one of the most recent I’ve found.

Let me contextualize the appearance of this error.

To simplify I have 5 machines in my environment:

• 2 SQL Server’s in a cluster and SSO Master Secret Server in a cluster
• 2 BizTalk Server’s
• And 1 IBM WebSphere MQ Server and with only BizTalk Server 2016 MQSeries Agent installed

I successfully installed:

• BizTalk Server 2016 Cumulative Update 5 (CU5) in the SSO Master Secret Server’s
• And BizTalk Server Feature Pack 3 with CU5 on BizTalk Server’s

Nevertheless, while I was trying to install BTS2016 CU5 on the MQ Server I got the following error:

Ole db providers : SQLOLEDB, MSDataShape, ADsDSOObject, MSDASQL, MSDASQL Enumerator, SQLOLEDB Enumerator, MSDAOSP

SQLNCLI11 ole db provider not found in the system.

TLS1.2 support requires SQL Server 2012 Native client 11.0 must be installed on all BizTalk machines. Install SQL Server Native client 11.0 before applying the update.

To download and install Microsoft SQL Server 2012 Native Client 11.0, see this Microsoft Download Center webpage. https://www.microsoft.com/en-us/download/details.aspx?id=50402&751be11f-ede8-5a0c-058c-2ee190a24fa6=True

Aborting installation of this update.

Please try after installing SQL Server 2012 Native client 11.0.

Cause

Unfortunately, to install BizTalk Server CU5 and I guess previous ones there is this default requirement that you need to have: SQL Server 2012 Native client 11.0 installed in the server.

In fact, for the BizTalk Server 2016 MQSeries Agent to work properly this component is not required. This is a validation requirement of the CU5.

Solution

The solution is quite simple:

After you install the SQL client you will be able to successfully apply the BizTalk Server 2016 CU5.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

The post BizTalk Server 2016 CU5 Installation error: SQLNCLI11 ole db provider not found in the system appeared first on BizTalkGurus.

When you are working with API’s and Logic Apps and there is lots of rows of data involved you will sometimes come up with the following problems:

1. An API often pages the data once you go beyond a certain number of records
2. When you want to insert lots of rows with a Logic App into SQL you will usually have a loop which iterates over a dataset and does inserts
   1. This takes a long time to execute
   2. There is a cost implication to your implementation when you pay for each action

I recently had a scenario in this space and used quite a cool approach to solve the problem which I wanted to share.

Scenario

The scenario I had started in Shopify. When I add products & collections to my online store in Shopify I wanted to have a daily extract from Shopify to synchronise these new product/collections to my Azure SQL database which I use for reporting with Power BI.

To achieve this I would have a Logic App with a nightly trigger which would take the following actions:

• Clean the table of which product is in which collection
• Extract all products in collections via the Shopify API
• Insert them all into the SQL table

The end result is I have a table which has all of the products in each collection listed for my analysis.

At a high level the scenario looks like the below diagram:

Implementation

As I mentioned above the problem is two folded here, when we query Shopify there many be thousands of products so we need to use a paging approach to query their API, secondly I want to insert into SQL in batches to try to minimise the number of action calls on SQL to improve performance and reduce cost.

Lets look at how I did this.

Paging API calls to Shopify

When it comes to the Shopify API you are able to execute a GET operation against the collection and it will return the products within it. If you have lots of products you can get them in pages. I chose to get 250 per time and you need to pass a page index to the API as a query parameter. The below picture shows you what a call to Shopify would look like with the paging parameters set.

Once I can make this single call I can then use a loop around the call to Shopify, but before I do this I need to know how many pages there are. I can do this by executing a GET against the collections API with the count extension on the url. This will return me the number of products in collections. You can see this below.

From the response I can parse the count and then I would set a variable which is the number of pages which I will work out with a calculation of dividing the number of products by the number of products I will get per page. I will also add 1 to this so I get 1 more page than the count incase the division is not a whole number. The calculation is shown below.

add(div(body(‘Parse_JSON_-_Count’)?[‘count’], 250),1)

Now I know the number of pages I can implement the loop where I will increment the page index each time until we have matched the number of pages. Within the loop we will get the next page of data from the API as shown in the picture below.

SQL Json Insert

It would be possible to just call the insert action for SQL in the logic app but if there are say 10000 products then the loop will do 10000 iterations which will take quite a while to run and also there is a cost associated with that. I wanted to look at options for inserting the data in batches. If I could insert the entire page returned from the API as a batch then with my 250 records at a time I could reduce the 10000 iterations down to 40. That should be a lot less time and a much lower cost.

To do this I developed a stored procedure where I passed the entire JSON string from the API response to the stored procedure as an NVARCHAR(max) parameter. In the stored procedure I was fortunate that the format of the json in this case was very table/row like making it easy to do this insert. I used SQL’s OPENJSON feature and was able to insert the entire page of data from the API in a simple insert statement as you can see in the SQL below.

Summary

Once it was all put together I was able to run my Logic App to refresh my SQL database each night and the process took 10 seconds to copy across 2500 records. This took 10 iterations of the loop.

That’s a nice and easy to support and run Logic App which does a nice job in this case.

The post Inserting lots of rows into SQL with a Logic App and a Stored Procedure appeared first on BizTalkGurus.

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

How get started with iPaaS design & development in Azure?

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post Microsoft Integration Weekly Update: December 10, 2018 appeared first on BizTalkGurus.

First of all, I’d like to apologise to all grandmothers out there… I mean you no disrespect. It’s just meant to be a catchy title, really. I know grandmothers who are smarter than most of us.

A couple of months ago I had the privilege of speaking at the API Days event in Melbourne. My topic was on Building Event-Driven Integration Architectures, and within that talk I felt a need to compare events to messages, as Clement Vasters did so eloquently in his presentation at INTEGRATE 2018. In a slight divergence within that talk I highlighted three common messaging patterns using a pizza based analogy. Given the time constraint that segment was compressed into less than a minute, but I thought it might be valuable enough to put in a blog post.

Photo courtesy of mypizzachoice.com

1) Synchronous Messaging

So before you can order a pizza, you need to know a couple of things. First of all, whether the pizza shop is open, and then of course what pizzas they have on offer. You really can’t do anything else without this knowledge, and those facts should be readily available – either by browsing a website, or by picking up the phone and dialling the shop. Essentially you make a request for information and that information is delivered to you straight away.

That’s what we expect with synchronous messaging – a request and response within the same channel, session and connection. And we shouldn’t have time to go get a coffee before the answer comes back. From an application perspective, that is very simple to implement as the service provider doesn’t have to initiate or establish a connection to the client; the client does all of that and service simply responds. However care must be take to ensure the response is swift, lest you risk incurring a timeout exception. Then you create ambiguity for the client who doesn’t really know whether the request was processed or not (especially troublesome if it were a transactional command that requires idempotency).

2) Asynchronous Messaging

So now you know the store is open and what pizzas to choose from. Great. So you settle on that wonderful ham & pineapple pizza on a traditional crust with a garnish of basil, and you place your order. Now if the shopkeeper hands you a pizza straight away, you’re probably not terribly inclined to accept it. Clearly you expect your pizza to be cooked fresh to order, not just pulled ready-made off a shelf. More likely you’ll be given an order number or a ticket and told your pizza will be ready in 20 minutes or so.

Now comes the interesting part – the delivery. Typically you will have two choices. You can either ask them to deliver the pizza to you in your home. This frees you up to do other things while you wait, and you don’t have to worry about chasing after your purchase. They bring it to you. But there’s a slight catch: you have to give them a valid address.

In the asynchronous messaging world, we would call this address a “callback” endpoint. In this scenario, the service provider has the burden of delivering the response to the client when it’s ready. This also means catering for a scenario where the callback endpoint it invalid or unavailable. Handling this could be as blunt as dropping the response and forgetting about it, or as robust as storing it and sending an out-of-band message through some alternate route to the client to come pick it up. Either way, in most cases this is an easier solution for the client than for the service provider.

But what if you don’t want to give out your address? In this case, you might say to the pizza maker, “I’ll come in and pick it up.”  So they say fine, it’ll be ready in 20 minutes. Only you get there in 10 minutes and ask if it’s ready; they say not yet. You wait a few more minutes and ask again, and get the same response. Eventually it is ready and they hand you your nice fresh piping hot pizza.

This is an example of a polling pattern. The only burden on the service provider is to produce the response and then store it somewhere temporarily. The client  has the job of continually asking if it is ready, and needs to cater for a series of negative responses before finally retrieving the result it is after. You might see this as a less favourable approach for the client – but sometimes this is driven by constraints on the client side, such as difficulties opening up a firewall rule to allow incoming traffic.

3) Publish & Subscribe

Now let’s say that the pizza was so good (even if a little on the pricey side) that you decide to compliment the manager. He asks if you’d like to be notified when there are special discounts or when new pizzas are introduced. You say “Sure!” and sign up on his mailing list.

Now the beauty of this arrangement is that it costs the manager no extra effort to have you join his mailing list. He still produces the same newsletter and publishes it through his mailing agent. The number of subscribers on the list can grow or shrink, it makes no difference. And the manager doesn’t even have to be aware of who is on that list or how many (although he/she may care if the list becomes very very short!)  You as the subscriber have the flexibility to opt in or opt out.

It is precisely this flexibility and scalability that makes this pattern so attractive in the messaging world. An application can easily be extended by creating new subscribers to a message, and this is unlikely to have an impact on the existing processes that consume the same message. It is also the most decoupled solution, as the publisher and subscriber need not know anything about each other in terms of protocols, language, endpoints, etc. (except of course for the publishing endpoint which it typically distinct and isolated from either the publisher or consumer systems). This is the whole concept behind a message bus, and is the fundamental principle behind many integration and eventing platforms such as BizTalk Server and Azure Event Grid.

The challenge comes when the publisher needs to make a change, as it can be difficult sometimes to determine the impact on the subscribers, particularly when the details of those subscribers are sketchy or unknown. Most platforms come with tooling that helps with this, but if you’re designing complex applications where many different services are glued together using publish / subscribe, you will need some very good documentation and some maintenance skills to look after it.

So I hope this analogy is useful – not just for explaining to your grandmothers, but to anyone who needs to grasp the concept of messaging patterns. And now… I think I’m going to go order a pizza.

The post How to Explain Messaging Patterns to your Grandmother appeared first on BizTalkGurus.

When using the IBM MQ Client library for .NET it is possible to directly make use of “Subscriptions” (1) on IBM MQ, without a topic object (2), by specifying the “TopicString” while sending a message.

The BizTalk MQSC adaptor cannot directly specify a “TopicString” in the adaptor. Also, it is not possible to directly connect to a “Topic” (2) object. It only supports sending to a “Queue” object.

The Solution:

It is however, perfectly possible to create a Queue Alias that has a Topic as BaseObject. This Topic can be linked to one or more Subscriptions based on the TopicString.

[Queue Alias, with base object Topic] => [Topic, with TopicString /xyz/] => [Subscriptions, based on TopicString /xyz/]

The post BizTalk MQSC Adapter – Send to IBM MQ Topic/Subscription appeared first on BizTalkGurus.

Welcome back to another entry on my blog post series about “BizTalk Server Tips and Tricks” for developers, administrators or business users and I couldn’t resist on speaking about a topic that normally divides BizTalk developers and BizTalk administrators: Tracking Data!

Problem

Normally Developers have in their environments Full tracking Enabled, why? Because it’s easier to debug, troubleshoot, analyze or validate and see if everything is running well or simply, what is happening with their new applications.

The important question is: Do Developers remember to disable tracking before they put the resources in production?

No! And actually… they don’t care about that! Is not their task to do it or control it. And if you ask them, you should always have them enabled! Sometimes, to be fair, they don’t know the right configurations that should be applied to production.

This can be an annoying and time-consuming operation. It will be the same as asking developers to change their way of being, and for them to remember each time they export an application to disable the Tracking data properties can be a big challenge… or even impossible!

Solution (or possible solutions)

My advice is, if you are a BizTalk Administrator, let them be happy thinking they are annoying you and take back the control of your environment by yourself.

These tasks can be easily automated and configured by easily creating or using PowerShell.

You should disable all Tracking or enable just the important settings at the application level. You may lose 1 day developing these scripts, but then you do not need to worry anymore about it.

As an example, with this script: BizTalk DevOps: How to Disable Tracking Settings in BizTalk Server Environment, you can easily disable all tracking settings for all the artefacts (orchestrations, schemas, send ports, receive ports, pipelines) in your BizTalk Server Environment

# Disable tracking settings in orchestrations     
$Application.orchestrations |  
%{ $_.Tracking = [Microsoft.BizTalk.ExplorerOM.OrchestrationTrackingTypes]::None } 
 
# Disable tracking settings in Send ports        
$disablePortsTracking = New-Object Microsoft.BizTalk.ExplorerOM.TrackingTypes 
$Application.SendPorts |  
%{ $_.Tracking = $disablePortsTracking } 
 
# Disable tracking settings in Receive ports 
$Application.ReceivePorts |  
%{ $_.Tracking = $disablePortsTracking } 
 
# Disable tracking settings in pipelines         
$Application.Pipelines |  
%{ $_.Tracking = [Microsoft.BizTalk.ExplorerOM.PipelineTrackingTypes]::None } 
 
# Disable tracking settings in Schemas 
$Application.schemas |  
    ?{ $_ -ne $null } | 
    ?{ $_.type -eq "document" } | 
    %{ $_.AlwaysTrackAllProperties = $false }

This can easily be edited by you to disable only one application or you can configure the right tracking setting that you want for your applications and environment.

If you are working with BizTalk Server 2016…

In previous versions of BizTalk Server, tracking settings were automatically imported with the rest of the application bindings. However, if you are working with BizTalk Server 2016, you have a new feature that allows you to have a better control while importing your BizTalk Applications: Import Tracking Settings.

If you are importing an MSI file, on the “Application Settings” tab, you will have a checkbox “Import Tracking Settings” that allows you to say: I don’t want to import the tracking from DEV or another environment in which the MSI was generated from.



If you are importing a Binding file you will also have this same option:



Of course, if you want to properly define the correct or minimum tracking settings of your application, you need to do it manually or, once again, using a PowerShell script to accomplish that.

Quick, simple and practical.

Stay tuned for new tips and tricks!

Author: Sandro Pereira

Sandro Pereira is an Azure MVP and works as an Integration consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. View all posts by Sandro Pereira

The post BizTalk Server Tips and Tricks: Take control of your environment: Tracking Data appeared first on BizTalkGurus.

TL;DR – There is no decent .NET support for PCF in the IBM client, but we can use IKVM.NET to convert JARs to DLLs so we can still use .NET instead of JAVA to use PCF.

Intro

Programmable Command Formats (PCFs) define command and reply messages that can be used to create objects (Queues, Topics, Channels, Subscriptions,…) on IBM Websphere MQ. For my current project we wanted to build a custom REST API to automate object creation based on our custom needs. The IBM MQ REST API was not a possible alternative at that moment in time.

The problem

The .NET PCF namespaces are not supported/documented by IBM and do not provide the possibility to inquired the existing subscriptions on a queue manager. All other tasks we wanted to automate are possible in .NET. Using JAVA seemed to be the only alternative if we wanted to build this custom REST API with all features.

Being able to use the .NET platform was a requirement at that time, because the whole build and deployment pipeline was focused on .NET.

IKVM.NET

After some searching I stumbled upon IKVM.NET:

“IKVM.NET is a JVM for the Microsoft .NET Framework and Mono. It can both dynamically run Java classes and can be used to convert Java jars into .NET assemblies. It also includes a port of the OpenJDK class libraries to .NET.“

Based on this description it sounded like it could offer a possible solution!

Using IKVM.NET we should be able to convert the IBM JARs to .NET assemblies and use the supported and documented IBM Java Packages from a .NET application.

From JAR to DLL

Now I will shortly explain how we were able to put it all together. Using IKVM.NET is not that easy when you use it for the first time. The whole process consists basically out of 3 steps:

1. Download (and Install) the IBM MQ redistributable client (in order to extract the JAR files)
2. Convert JARs to DLLs with IKVM.NET
3. Copy DLLs and Reference in .NET project
   • The IBM Converted JARs and the IKVM.NET Runtime dlls

Convert JAR to DLL

Download IKVM: https://sourceforge.net/projects/ikvm/
Extract the IKVM files (c:toolsIKVM)
I have the IBM client installed, so the JAR files will be on there default installation (C:Program FilesIBMMQjavalib)

Open up a Command Prompt:

1
2
3
4
5

set path=%path%;c:toolsIKVMbin
 
cd C:Program FilesIBMMQjavalib
 
ikvmc -target:library -sharedclassloader { com.ibm.mq.jar } { com.ibm.mq.jmqi.jar } { com.ibm.mq.headers.jar } { com.ibm.mq.pcf.jar }

You will find the output in the source directory of the JAR files:

Add References…

Now that we have our DLLs, we can add them to our .NET project. This seemed less easy then I thought, I spend a lot of time figuring out what dependencies I needed. In the end, this was my result:

1 = The IBM JARs converted to DLLs
2 = The IKVM.NET runtime DLLs

MqPcfAutomation Sample

To help you get started, I added my sample proof of concept solution to GitHub in the MqPcfAutomation repository.

In the sample a showcase the functionality described in the table at the beginning of this post.

Conclustion

In the end I am happy that I was able to build a solution for the problem. The question is if this approach is advised…
I don’t think IBM approves this approach, but it works for what we need it. We are using this solution now for more then 6 months without any issues. In the future we might be able to move to the IBM MQ REST API as more features will be added.

The post Automate IBM MQ object creation with PCF in .NET using IKVM.NET appeared first on BizTalkGurus.

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post Microsoft Integration Weekly Update: December 17, 2018 appeared first on BizTalkGurus.

A few days back we have received a question from one of our customer on the possibility to install BizTalk360 on Windows Azure Virtual machine. We have replied with suggestions, and he was impressed by the option what we have provided to install BizTalk360.

That’s the moment of inspiration for this blog. Here, we are going to look at the types of installers available and different ways to install BizTalk360.

I would recommend you to have a look at the prerequisites that should be met to install and work with BizTalk360.

Installing BizTalk360 – What are the options

One of the common questions that customers raise to us. There are various deployment choices available for BizTalk360.

This is one of the common questions we have observed recently on our customer engagements.

The deployments are purely based on the organization needs and based upon the BizTalk environment architecture. Here are the options BizTalk360 provides:

• Installing BizTalk360 On a BizTalk Server
• Installing BizTalk360 On a Stand-Alone (non-BizTalk) Server
• Installing Just the BizTalk360 Monitoring Service
• Installing Just the BizTalk360 Analytics Service
• Installing BizTalk360 on High Availability BizTalk Servers

How can you achieve the various deployment options?

If you are installing BizTalk360 for the first time, you can select the components that you wish to install on the server.

Option 1 – Installation on the BizTalk Server environment

You can install BizTalk360 in one of BizTalk Servers of the BizTalk Server environments (production, non-production etc.) and configure all the environments in one BizTalk360 interface.

Option 2 – Installation on a stand-alone server

This is another common scenario where customers don’t want to install BizTalk360 directly on their production environment.

BizTalk360 supports this scenario by allowing customers to install on a standalone server and access various environments. In this case, customers need to install BizTalk administration components (only) in the standalone server. Also, any third-party adapters must be installed on this server.

It is one of our recommendations to install BizTalk360 on a separate box for the customers who don’t want to install in their BizTalk Environment. The benefit is that once your BizTalk server goes down, your BizTalk360 server won’t go down along with it and be able to send notifications about your BizTalk server not being available.

Option 3 – Install Just the BizTalk360 Monitoring Service

The Monitoring Service plays a key role as it is responsible for fetching the status of artifacts from the BizTalk server to monitor them. Using this option, you can install the BizTalk360 Monitoring Service on a separate box and BizTalk360 database on a separate box.

This setup will be helpful in a scenario where the customer wants to have the BizTalk360 web services, user interface (IIS), BizTalk360 Monitoring service and the BizTalk360 Database all in separate boxes.

This setup can also be done when you want the BizTalk360 monitoring service to be in the high availability state. When monitoring service is installed in two servers in the high availability mode, when one service goes down, the other service will be up, where the monitoring of your BizTalk environment by BizTalk360 will be continued without any interruption.

Option 4 – Install Just the BizTalk360 Analytics Service

As like installing BizTalk360 monitoring service on a separate box you can install the BizTalk360 Analytics Service on a separate box as well.

Option 5 – Installing BizTalk360 on High Availability BizTalk Servers

You can install BizTalk360 on BizTalk high availability servers. At the same time, you can make BizTalk360 as a highly available setup.

Installing BizTalk360 on the single server environment is a straightforward task when the BizTalk server and database are existing in the same machine. You can simply run the BizTalk360 MSI from the Administrator Command prompt.

But in most of the cases, we have seen our customers use BizTalk360 in a High Availability mode to make it available all the time.

Different ways to install BizTalk360

To fulfil all the customer needs, there is a number of ways to install BizTalk360 and we provide different types of installer options:

1. Default installation
2. Silent Installation
3. Azure Marketplace installer
4. Azure easy installer

Default installation

The Default installation is the regular way of installing BizTalk360.

Once you have created and configured a machine with all the prerequisites met, download the latest version of BizTalk360 (.msi) and install it on the machine.

In which case will this be helpful?

When you have a physical/virtual server within an organization which may have or have not internet connectivity.

You can achieve all the various deployment choices,

• Installing BizTalk360 on a BizTalk Server
• Installing BizTalk360 on a Stand-Alone (non-BizTalk) Server
• Just the BizTalk360 Monitoring Service
• Just the BizTalk360 Analytics Service
• BizTalk360 on High Availability BizTalk Servers

You can download the latest version installer of BizTalk360 from http://www.biztalk360.com/free-trial/ . You must enter the registration information in the form before downloading the BizTalk360 MSI.

Silent Installation

This is one of the interesting cases which came to us. One of our customers was trying to install BizTalk360. Although he was from Admin team, he was not a member of the BizTalk groups.

He proposed a request that is there any option to install BizTalk360 without the installer user interface or screen. We do have Silent installation support.

So, we assisted the customer with the command which they need to run the installer without the installer GUI. But with the silent installation, you can’t achieve all the various deployment choices as seen with the regular installation (Refer the section “Default Installation -> In which case it will be helpful”).

Azure easy installer

The move to the cloud eventually means that customers using BizTalk360 for managing and monitoring their BizTalk server environments will need their BizTalk360 setup to run on their Azure setup. Without BizTalk360 on their Azure setup, the only way for customers to manage their BizTalk environment (running on Azure) is by physically logging into the server through RDP connection.

Also, there is a challenge with the number of RDP connections (no. of users) who can access the server at a time. Therefore, the presence of BizTalk360 becomes an important factor – be it in an on-premise setup or on a remote server (Azure).

In which case will this be helpful?

 Any server which has BizTalk Server installed, with the help of BizTalk360 Azure Easy Installer, you can easily install BizTalk360 by executing a single PowerShell command on your BizTalk Server Machine.

Why did we introduce this?

As the technology and usability evolve, we would like to encourage customers who are already using BizTalk server in Azure, to move their BizTalk360 environment to the cloud as well.

The customers using BizTalk360 for managing and monitoring their BizTalk server environments in Azure will need their BizTalk360 setup to run on their Azure setup as well.

How easy to use is it?

With the BizTalk360 Azure Easy Installer, you can easily install BizTalk360 by executing a single PowerShell command on your BizTalk Machine. The installer will take care right from the scratch of creating a service account for BizTalk360, configuring the BizTalk server (optional), enabling HTTP feature, granting SQL permissions and IIS.

Azure Marketplace installer

This is helpful for customers having a Microsoft Azure account and provisioned BizTalk server machines in the cloud with a more complex setup such as a domain, Active Directory, and one or more BizTalk environments in the network.

With BizTalk360 in Azure, you get a complete deployment that can target your BizTalk environment running on-premise via VPN or ExpressRoute or on Azure IaaS Platform.

In which case it will be helpful?

With the help of BizTalk360’s Azure Marketplace installer, you can simply spin up and add a new BizTalk360 machine into your existing BizTalk environment. We support BizTalk Server 2013R2 and 2016.

Why did we introduce this?

To ease the installation process, where the customer has the BizTalk server setup in domain Azure architecture, this tool will be very helpful. In any architecture, to add a new application software will be a difficult job because so many changes need to be done right from the providing permission to other prerequisites.

Likewise, to install BizTalk360, set of prerequisites must be performed before installing. To make this job easier for the users, our Azure marketplace installer comes into the picture.

How easy to use is it?

While adding a new machine for BizTalk360, select new Virtual machine in the Marketplace and select the image according to your existing environment. As of now, we are providing support for BizTalk server 2013 R2 and BizTalk 2016.

It is just a simple process; once after the creation of the new BizTalk360 Virtual machine, the Azure Marketplace installer will be opened. Just providing the credentials, it will be added to the domain (the machine will be automatically restarted as a part of this process), and BizTalk360 installation will continue.

Upon successful completion, BizTalk360 will be installed and you will notice a browser window firing open with the BizTalk360 screen. We want to convey that using the BizTalk360’s Azure Marketplace installer, and the Azure easy installer is free of costs.

Conclusion

Keeping the user perspective in mind, we have built various deployment choices for BizTalk360 customers which would ease the installation and provide a user-friendly experience.

Author: Sivaramakrishnan Arumugam

Sivaramakrishnan is our Support Engineer with quite a few certifications under his belt. He has been instrumental in handling the customer support area. He believes Travelling makes happy of anyone. View all posts by Sivaramakrishnan Arumugam

The post Different ways to install BizTalk360 appeared first on BizTalkGurus.

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post Microsoft Integration Weekly Update: December 24, 2018 appeared first on BizTalkGurus.

One thing I have been doing consistently on the last day of each year since I started BizTalk360 back in 2011 is donating a small amount to GOSH as a way of saying thank you for one more successful year. 

In spite of where I’m in the world, I make a note on my personal calendar to do this activity first thing on 31st December. 

A definition of success is different for different people, for me, it’s pretty simple, being healthy (people around me and myself),  moving forward a few steps ahead from where we have started and try and help others move forward as much as I can. As long as we are not standing still or moving backward and everyone around me is happy then it’s a successful year. 

This year we contributed $10k to GOSH bringing the total contribution to approximately $47,000 in the past 7 years. Here is the summary.

• 2012 – $1,625
• 2013 – $2,125
• 2014 – $5,000
• 2015 – $10,000
• 2016 – £5,000
• 2017 – $10,000
• 2018 – $10,000

About GOSH

GOSH is one of the world’s leading children’s hospitals, housing the widest range of specialists under one roof. GOSH opened its doors back in 1852 with just 10 beds, today they get around 600 new patients every day. It was the first hospital in the UK dedicated solely to the treatment of children.

GOSH was supported by some of the great individuals like Charles Dickens, Queen Victoria and Diana Princess of Wales. 

We feel proud to be associated with such a great organization with such a great cause.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

The post Thanks 2018 – time to give back appeared first on BizTalkGurus.