MSMQ
MSMQ
Microsoft Message Queuing (MSMQ) 3.0 and 4.0 are both supported as Transports. MSMQ ships with the Microsoft Operating Systems. Windows XP and Windows Server 2003 support MSMQ 3.0, whereas Vista, Windows 2008 and new operating systems support MSMQ 4.0.
The version of MSMQ supported on the operating system will determine flexibility of deployment in high availability (HA) and disaster recovery scenarios.
MSMQ should be used specifically if transactions, durable messaging or guaranteed and reliable delivery are required. This transport provides some unique benefits and patterns of deployment that the other transports do not offer, for example:
- If a Party is disconnected from the bus when a message is published to an MSMQ based Topic, it will still receive the message once it connects.
- If a Party is connected, but the Neuron ESB Service is down, the Party can still publish the message successfully. When the Neuron ESB Service is restored, the message will be relayed to its subscribers.
- Parties may directly send (route) messages to connected subscribers (Parties) if the Neuron ESB Service is down.
- All messages are sent to a single instance of a subscribing Party. If there are more than 1 instance of the same subscribing Party active, then message delivery will be load balancing between the active instances of the Party.
Using the MSMQ Transport has the following system requirements:
- MSMQ 3.0 or MSMQ 4.0 Server and associated queues installed locally on the Neuron server, or on a dedicated remote server.
- MSMQ Client API is installed on all machines running the Neuron ESB Party
Transactional considerations:
- IF using MSMQ 3.0 and using transactional messaging, then MSMQ server and associated queues MUST be located on the Neuron ESB server, unless Queue Mapping is used.
- IF using MSMQ 4.0 and using transactional messaging, the MSMQ server and associated queues can be located either on the Neuron ESB Server, or on independent MSMQ servers.
- MSDTC must be configured for remote network access to support distributed transactions. Please refer to Microsoft support article: https://support.microsoft.com/kb/817064
Firewall considerations:
- Ports must be open for MSMQ. Please refer to Microsoft support article : https://support.microsoft.com/kb/183293
- Port 135 must be open in both directions for MSDTC. If you are using a firewall that does not manage RPC connections, you will need to modify the registry on the computers involved to limit RPC secondary outbound ports to 5000-5020 (or whatever your preferred ports are) for MSDTC. Please refer to Microsoft support article: https://support.microsoft.com/kb/306843
- If the MSMQ Transactional Receive Service is used, then the port specified for the MSMQ Receive Service property for the active Zone must be opened. This can be found by selecting Deployment ->Divisions->Zones from the navigation bar located on the left hand side of the Neuron ESB Explorer. Select the Ports tab for the specific Zone.
All configurable properties for the MSMQ Transport are located on the Transport Property page of the Details Pane.
When a Topic is configured to use MSMQ as its Transport, there must be an underlying MSMQ Queue created for the Topic, as well as a unique Queue (both transactional and non transactional) created for each Party that references the Topic. The MSMQ Queues can be auto created (or scripts can be exported) through the Manage Queue s Management Window by selecting Deployment ->Manage->Queues from the navigation bar located on the left hand side of the Neuron ESB Explorer.
There are 4 unique modes that the MSMQ Transport can be configured that directly affect the scale and distribution of messages published to the Topic. In all modes, Transactions and Recoverable messaging can either be enabled or disabled.
| Direct | Queue Mapping |
| Yes | Yes |
| Yes | No |
| No | Yes |
| No | No |
No Direct/No Queue Mapping
This is the default configuration. This configuration is active when the “Direct” property is set to False and Queue Mapping is not explicitly defined. In this configuration all Topic and Party queues are located on the same server, either where the Neuron ESB service is running, or on a dedicated MSMQ Server, remote to the Neuron ESB Service.
This is a common configuration for Neuron ESB server deployments or development environments. This configuration is supported in both single server and farm scale out scenarios. Scale is affected as all queues must be local to a single MSMQ server.
In a typical distributed deployment of Neuron ESB Service and Parties, Parties perform a remote transactional write to publish, as well as a remote transactional read to subscribe. If using MSMQ 3.0, the Transactional Receive Service property can be set to True to enable support for remote transactional reads. This is ONLY required in MSMQ 3.0 environment and ONLY where transactional reads are required.
Direct
This configuration is active when the “Direct” property is set to True. This allows a publishing Party to submit a message directly to the subscribing Party’s corresponding queue. All routing is done locally on the publisher’s machine. If the Neuron ESB Service is down, the message will still be published successfully to the subscribing Party’s queue. Direct only mode performs substantially faster than No Direct/No Queue Mapping mode as there is no intermediate Topic queue thereby reducing I/O and the number of transactions required.
In this configuration all Party (Subscriber) queues are located on the same server, either where the Neuron ESB service is running, or on a dedicated MSMQ Server, remote to the Neuron ESB Service. This configuration is supported in both single server and farm scale out scenarios. Scale is affected as all queues must be local to a single MSMQ server.
Queue Mapping
Queue Mapping provides a more scalable queuing architecture than either the “Direct” or “No Direct/No Queue Mapping” mode. Queue Mapping specifically allows the individual queues that correspond to Parties to be distributed across N number of machines.
Queue Mapping should be considered if Memory or I/O becomes a bottleneck on the MSMQ server. Also, this should be considered if using transactions, as the subscriber queues could be relocated so that they are local to the subscribing the Parties, changing an otherwise remote transactional read into a local transactional read. This will result in improved performance if using transactions over the “No Direct/No Queue Mapping” mode.
Queue Mapping can be enabled and managed through the Queue tab of the Details Pane of the selected deployment group for the Topic. Select Deployment ->Environments->Deployment Groups from the navigation bar located on the left hand side of the Neuron ESB Explorer, and then select the active deployment group for the Topic.
To create a mapping, select the Topic, Party, and Machine and then enter the name of the Queue that will be associated with the Party. For a machine name to be displayed in the Machine drop down box, it must first be entered and configured in the Machines tab. Additionally, the Party name can be left blank, effectively mapping the Topic, rather than the Party.
If the machine name configured for the queue is not the same machine that the Neuron ESB Explorer is installed on, the queues will have to be created manually. Scripts to generate the queues can be created by clicking the “Create Script” toolbar button at the top of the Manage Queues Management Window.
Direct + Queue Mapping
This mode is a combination of Direct and Queue Mapping and in most scenarios will perform the best, as well as allow greater scalability through the a wider range of distributed queue options. This mode provides all the benefits of Direct as well as Queue Mapping. In this mode, the Topic level queue is bypassed, writing messages directly to the subscriber queues. However, subscriber queues can be mapped to various machines within the network.
MSMQ Transport Property Table
| Property Name | Required | Description | ||||||||
| Maximum Batch Size | Yes | Default is 100. Number of messages that will be submitted to the bus in the same transaction. | ||||||||
| Recoverable | Yes | Default is True. Determines whether or not all messages are written to disk, rather than memory. Synonymous with Durable. | ||||||||
| Transactional | Yes | Default is True. Determines whether or not Neuron ESB will read and write to underlying queues transactionally. If True, exactlyOnce is also set to True. | ||||||||
| Custom Dead Letter Queue | No | Optional. Only required if “Dead Letter Queue” property is set to “Custom”. Name of queue to use for dead letter functionality (poison messages) of MSMQ. | ||||||||
| Dead Letter Queue | Yes | Default is System. 3 options, either System, None or Custom. Used to store poison messages of MSMQ | ||||||||
| Direct | Yes | Default is False. If set to True, publishers will bypass Topic based queue and write directly to Subscriber queues. All routing is accomplished by Publisher. | ||||||||
| Max Retry Cycles | Yes | Default is 3. An integer that indicates the number of retry cycles used by the poison-message detection feature. A message becomes a poison message when it fails all delivery attempts of all cycles. | ||||||||
| Queue Transfer Protocol | Yes | Default is Native. Options are Native, Srmp and SrmpSecure. Srmp and SrmpSecure are used to enable SOAP Reliable Messaging Protocol. Srmp and SrmpSecure cannot be used with Public Queues. Srmp and SrmpSecure can only be used for publishing messages. All messages will always be received using Native. | ||||||||
| Receive Error Handling | Yes | Default is Fault. Options are Fault, Reject, Move and Drop. These values specify how poison and nondispatchable messages are handled.
| ||||||||
| Receive Retry Count | Required | |||||||||
| Retry Cycle Delay | Required | |||||||||
| Transactional Receive Service | Required | |||||||||
| Use Public Queues | Required |
Many of the properties and their range of configuration values can be found here: https://msdn.microsoft.com/en-us/library/ms731380.aspx