|
|
礼貌的椰子 · go 中解析JSON的三种姿势 - ...· 1 年前 · |
|
|
侠义非凡的哑铃 · 阿里内部爆款K8s+Docker+Jenki ...· 2 年前 · |
|
|
买醉的草稿本 · curl查看请求头-掘金· 2 年前 · |
|
|
八块腹肌的砖头 · DevExpress ...· 2 年前 · |
|
|
重感情的热水瓶 · 教程:调试 C++ 代码 - Visual ...· 2 年前 · |
This article describes the exploitation of the pluggable transport infrastructure provided as part of ASP.NET Web Services, which allows developers to deliver Soap messages built in the document/literal format to be delivered to an end point via mechanisms other than default HTTP transport.
In this article, a framework that supports the addition of alternative physical delivery mechanisms is presented, and two such implementations are included – MSMQ and Websphere MQ are chosen to illustrate a store-forward capability for a more 'guaranteed' Web Services request delivery. The framework that will be developed integrates the Web Services Enhancements (WSE) 1.0 for secure 2-way transport of messages over non-HTTP mechanisms, thereby giving the developer a choice of 'trusted' (clear message) or 'untrusted' (secure message) delivery to the chosen endpoint.
Soap was designed as a transport neutral protocol to be used in combination with a variety of transport protocols such as HTTP(S), SMTP, FTP etc. to deliver structured and typed information between two participants. A great deal of work has gone into making ASP.NET Web Services support Soap message generation in a way that promotes interoperability within heterogeneous environments (for example .NET consuming a BEA Weblogic Server hosted Web Service) but to date the standard access to Web Services within the Microsoft arena has been almost exclusively geared to HTTP.
There are of course strengths and weaknesses of the HTTP transport – a definite plus point is the maturity of the associated security infrastructure, which has risen on the back of the ubiquity of the web in general. Conversely, a weakness is seen to be the lack of any form of guaranteed and reliable (single message only) delivery within the protocol, which has spawned solutions such as HTTPR from IBM. Add to this the fact that the back-end Service has to physically be on line for the request to be received over HTTP, and there are some pretty compelling reasons to want to find alternative ways to "get the message across".
While ASP.NET does a lot to promote the HTTP-centric binding to Web Services, HTTP is not the only protocol available for transporting Soap. Indeed, a little talked-of area is the Web Services infrastructure in place within the .NET framework that allows message delivery mechanism and endpoint to be altered either statically (design time) or dynamically (run time). The real power of this infrastructure – named "pluggable protocols" by Microsoft – is that the transport method can be altered underneath the logical client request to direct Soap via any sensible medium the developer cares to implement.
What this case study will demonstrate is the ability to effectively switch transports underneath ASP.NET client between HTTP, MSMQ and Websphere MQ. It will demonstrate how trivial it is to alter the 'target' of a Soap Web Service request from a standard IIS hosted variant, to a listener waiting on an MSMQ or MQ queue. We'll demonstrate the exercising of API using standard types like strings, arrays, binary data and custom serializable types. We'll also show how this support can be achieved in an asynchronous fashion.
It is likely that the queuing style transports would find use within the enterprise i.e. within the trusted network of an organization – with their commensurate (typically reduced) security requirements. However, by allowing non-HTTP delivery options, we've potentially taken away a layer of proven security that we ought to investigate a replacement for. Fortunately, a great deal of work has been undertaken in developing some joint standards for a new generation of web services protocols covering (amongst others) security, message routing and message attachments. This push really centers round support for securing messages between multiple processing nodes involved in a business transaction in a transport-agnostic way – without having to rely on technologies like SSL which only work well point-to-point. This effort has culminated recently in the release of Microsoft's implementation of the aforementioned standards in the form of the Web Services Enhancements (WSE) 1.0 package. So, the case study will also show how the implementation of WS-Security within the WSE 1.0 can be employed to secure (in our case digitally sign and symmetrically encrypt) the messages produced by ASP.NET clients and Servers instead of using the native security mechanisms inherent in the two queuing products. We'll also look at the DIME support provided by the WSE as an alternative way to send binary data, which was developed for transmission of large binary data streams.
Hopefully by the end of the case study the following aspects will be clarified:
The original context for the work involved a Proof of Concept project designed to determine how easily a .NET client could access an existing Java-oriented backend using Web Services for interoperability, but via IBM Websphere MQ rather than HTTP. This article provides a .NET oriented back-end on the grounds that in doing so we can keep the skill sets covering the article constrained to .NET and can demonstrate secure communications more easily by virtue of having WSE at both ends. However, idiosyncrasies of interoperation with a Java service implementation will be identified at appropriate points throughout the article.
The built solution will require the following deployment environment:
1. Windows 2000 SP2 or Windows XP Pro with MSMQ and IIS installed
2. .NET Framework 1.0 (SP1 or above)
3. WSE 1.0 SP1 runtime
4. (For MQ support only ) Websphere MQ 5.3 Trial Release [2]
5. (For MQ support only) VB6 runtime [10]
In order to rebuild the solution binaries, you will require the following additional tools:
6. VS.NET (C#)
7. WSE 1.0 SP1 – full installation recommended
8. VS.NET WSE Settings Tool (optional) [8]
Essentially we aim to make progress in the following areas:
Interface contracts are described using WSDL (akin to IDL for COM). Once this WSDL is available at a URL or within a file, a Web Services proxy class can be generated either by directly using the standalone tool WSDL.EXE shipped with the SDK or implicitly when using the Visual Studio .NET 'Add Web Reference' facility:
Figure 1: The WSDL-2-Proxy process
The generated proxy file (usually called
reference.cs
) will live
in a subfolder of the project off a subfolder called
SoapHttpClientProtocol
. The generated class does a very good job of
hiding much of the complexity of the serialization and transport specifics
pertinent to the standard XML Web Services invocation. Within the resultant
generated proxy file, one may typically see a combination of synchronous and
asynchronous forms of methods - for example the following shows a basic
generated (synchronous) method:
[SoapDocumentMethodAttribute(...)] public string IsCollaboratorRegistered(string vstrEnvUUID) object[] results = this.Invoke("IsCollaboratorRegistered", new object[] { vstrEnvUUID }); return ((string)(results[0]));With the standard proxy in place, clients can the access the Web Service at the appropriate URL using code similar to the following:
// Get an instance of the proxy localhost.RegService objWSReg = localhost.RegService(); // Endpoint can be varied at runtime etc. objWSReg.Url = "http://192.168.32.1/MyService.asmx"; // Make the call string strRetXML = objWSReg.IsCollaboratorRegistered("12345678901234567890123456789012");Being able to vary the URL and to describe this URL in terms of a "protocol scheme" and an "end point" is the key aspect of the approach.
At runtime, the single line within the proxy containing the call
this.Invoke( … )hides a lot of work that is segmented into the following basic pieces:Transport selection / confirmation of support Soap serialization of API call Transport protocol driving We'll break this down further later in the article to show how the pluggable protocol infrastructure is combined with Soap serialization to build and deliver messages. In particular there are several methods that can be overridden within the proxy to tap into or even replace the message transport protocol process.
Defining Internet and other Network Transports for Web Services
While the de-facto delivery protocol for Soap messages is HTTP, there is an infrastructure in place within the .NET framework that allows message delivery mechanisms to be altered either statically (design time) or dynamically (run time).
The .NET Framework uses three specific classes to provide the information required to access Internet / Network resources through a request / response model:
1. the
Uriclass, which contains the URI of the Internet resource you are seeking (including the protocol scheme e.g. http://www.microsoft.com/webservices/stockcheck.asmx2. the
WebRequestclass, which encapsulates a request for access to a network resource,3. the
WebResponseclass, which provides a container for the incoming response from the network resource.Client applications create
WebRequestinstances by passing the URI of the network resource to theWebRequest.Create()method. This static method creates aWebRequestinstance for a specific protocol, such as HTTP. TheWebRequestinstance that is returned provides access to properties that control both the request to the server and access to the data stream that is sent when the request is made. TheGetResponse()method on theWebRequestinstance sends the request from the client application to the server identified in the URI.[NOTE: A Uniform Resource Identifier (URI) is a compact string of characters for identifying an abstract or physical resource as defined in RFC2396. The "scheme" is a standard part of the URI, which can be broken down into a number of distinct segments. All network identifiers are capable of mapping into one of the parts. The scheme is typically the protocol identifier e.g. http, ftp etc.]
Out of the box there is support for three specific protocol schemes - "http:", "https:" and "file:". We will create new versions of the WebRequest and WebResponse classes to provide support for a further two:
"msmq:" Support a request response model where the target is a local or remote MSMQ Queue "mq:" Support a request response model where the target is an MQ Queue Manager / Queue Name combination One can imagine that additional support could be added for other schemes such as "mailto:" or "ftp:" for SMTP and FTP respectively.
Note at this point that attempting to create a WebRequest for a Uri involving a scheme that is unsupported:
Uri uri = "mq://accountq1"; WebRequest newWebRequest = WebRequest.Create(uri);will yield an exception of the form
NotSupportedExceptionwhich indicates that the request scheme specified in requestUri is not registered. It is this issue we must address next in order to successfully introduce new schemes. It should be becoming clear now that our goal here is to allow a client to be able to identify a queue as the Uri of a request and be able to execute a method call against that Uri as they would against an http-based end point.Defining custom transports
The
WebRequestandWebResponseclasses form the basis of the "pluggable protocols" infrastructure. That means that in order to specify a new protocol like"xxx://", one should create at minimum derivations of each of these base classes, plus a class that implementsIWebRequestCreate. The following diagram depicts the basic model involved in production of new protocols:Figure 2: The Object model for pluggable protocols
The main operations and attributes have been included to show the key aspects of any implementation.
Common Aspects
Several of the attributes and operations that can be overridden by descendants of the
WebRequestandWebResponseclasses are fairly redundant when it comes to implementing non-internet-specific transports. As well as this, there is a large degree of commonality in the two queue implementations. This gives us the opportunity to abstract away those commonalities and redundancies into a single intermediate class. As an example, one could imagine that theMethodproperty that is used as part of every web type request (GET, POST etc.) would find no relevance in the new scheme implementations we are proposing.We define the following new derived types to extend the hierarchy:
Type Base Type Purpose MyBaseWebRequestWebRequestContains overrides common to MQ and MSMQ implementations MyBaseWebResponseWebResponseContains overrides common to MQ and MSMQ implementations MSMQWebRequestMyBaseWebRequestMSMQ implementation of WebRequest (sealed) MSMQWebResponseMyBaseWebResponseMSMQ implementation of WebResponse (sealed) MQWebRequestMyBaseWebRequestMQ implementation of WebRequest (sealed) MQWebResponseMyBaseWebResponseMQ implementation of WebResponse (sealed) Three criteria must be met in order for a protocol-specific class to be used as a pluggable protocol:
1. Preparing for scheme registration
The implementation must incorporate a version of the
IWebRequestCreateinterface e.g. for MSMQ:public class MSMQRequestCreator : IWebRequestCreate public WebRequest Create(Uri uri) // Create a new MSMQ request object return new MSMQWebRequest(uri);2. Scheme registration and Uri creation
Descendant classes of WebRequest that wish to implement new protocols must be registered with the WebRequest class in order to be allowed to manage the details of making the actual connections to non-standard network resources like Queues.
This is usually achieved in a similar fashion to the following:
// Register the "mq:" prefix here for WebRequest MQRequestCreator objCreator = new MQRequestCreator(); WebRequest.RegisterPrefix("mq", objCreator);This effectively allows the subsequent hooks up of the Create code for an instance of the new scheme, such that the code we mentioned above:
Uri uri = "mq://accountq1"; WebRequest newWebRequest = WebRequest.Create(uri);will then work since the new scheme is known to .NET Web Services infrastructure.
3. Implementation of key overridden members
The derived classes must override the abstract methods and properties of
WebRequestandWebResponseto provide the pluggable interface. These members are well documented for .NET Framework 1.0 - the relevant subset are shown here:
Member Purpose MethodUsually associated with a form of protocol method to use in this request e.g. PUT, GET etc. Not used here. HeadersGets or sets the collection of header name/value pairs associated with the request. Not used here. ContentLengthContains the number of bytes of data sent to the Internet resource by the WebRequestinstance.ContentTypeWhen overridden in a descendant class, gets or sets the content type of the request data being sent. In our case always text/xml. CredentialsGets or sets the network credentials used for authenticating the request with the Internet resource. Unused in our case. PreAuthenticateIndicates whether to preauthenticate the request. Unused in our case. ProxyGets or sets the network proxy to use to access this Internet resource. Unused in our case. GetRequestStreamTypically creates and returns a Streamfor writing data (Soap message) to the Internet resource. Implemented in our base class.BeginGetRequestStreamAsynchronous version of the above. Not supported in our case. EndGetRequestStreamAsynchronous version of the above. Not supported in our case. GetResponseReturns a response to a request. This override forms the work horse of our implementations of MSMQ and MQ as this is where the protocol-specific code is accessed. BeginGetResponseAsynchronous version of the above. Not supported in our case. EndGetResponseAsynchronous version of the above. Not supported in our case. <CODE>AbortCancels an asynchronous request started with the BeginGetResponsemethod. Not supported in our case.As implied above, we'll implement any overrides common to MQ and MSMQ versions in
MyBaseWebRequest/MyBaseWebResponse, and the specifics in the appropriate derived classes.Execution of a WebRequest-derived instance
Here we'll attempt to map the general client-side serialization and transportation process onto the discrete calls made by the .NET Web Services infrastructure during a synchronous web method call.
The diagram depicts the following sequence of events on the “request” execution path:
Figure 3: Interaction diagram for WebRequest
Key facets of operation include:
Overriding GetWebRequestin our generated Soap proxy (derived fromSoapHttpClientProtocol) allows us to hook in a registration (for schemes of msmq / mq) and creation of the appropriateWebRequestinstance. If the scheme is "msmq", this creation is achieved via theMSMQRequestCreatorinstance.The .NET Web Services infrastructure calls a private method, BeforeSerialize(), which triggers the getting and setting of various member data within the instance of our commonWebRequest-derived class, setting the method (irrelevant for us but we have to support the operation - returning aNotSupportedexception causes the .NET Web Services infrastructure to reject the whole request!) and getting the (empty) web headers collection.Next, the GetRequestStream()method is called on ourWebRequest-derived class where we simply create a stream (aMemoryStreamin this case) and return it to the infrastructure.At this point, the infrastructure begins the Soap serialization process and deposits the results into our stream. A very huge problem at this point is that in order to get the serialized stream pushed out onto the wire for transmission, the standard processing involves closing the stream! Given we've yet to reach the point where we want to use the stream to push at a queue this is at first glance a bit of a showstopper!! So, let's conveniently ignore it for now :-). The section entitled "Tapping into the serialized Soap stream" explains the issues and possible solutions. Once the stream has been prepared, the infrastructure then calls our overridden GetResponse()in theMSMQWebRequestclass instance. Here the specifics of dealing with queues via the appropriate mechanism (System.Messagingfor MSMQ or the IBM-provided COM library for MQ) are employed. This will undoubtedly involve writing the Soap message and awaiting a response to it from the back end service. The specifics of MSMQ and MQ transports are covered in the next section. Yet again we conveniently glossed over access to the stream at this point...Whatever the specifics of the MSMQ / MQ work, the result is to either deliver a response object of the appropriate type (e.g. MSMQWebResponse), or raise an instance of aWebExceptionif a failure has occurred in the transport. These types of exception are discussed later in the article.If you do get the chance, I'd strongly recommend a little trawl through the debugger, setting breakpoints all over the code as it really is instructive as to the ordering of events in the overall process.
Implementing MSMQ and Websphere MQ transports
Assuming we can access the serialized Soap message (which we deal with below), we need to hook in a protocol handler, based on
WebRequest. This section highlights the key facets of the implementation and expands the information given in the Sequence diagram above.Miscellaneous common Request aspects
Several overridden properties are not required for operation and so may be allowed to throw aNotSupportedexception e.g.:public override IWebProxy Proxy /* override */ get { throw new NotSupportedException(); } /* override */ set { throw new NotSupportedException(); }Others, like
ContentLength,ContentTypeandTimeoutetc. are required and are set up correctly:public override int Timeout /* override */ get { return m_intTimeout; } /* override */ set { m_intTimeout = value; }These aspects, shared by MSMQ and MQ implementations, are abstracted into the
MyBaseWebRequestclass.Providing a Stream for serialization
MyBaseWebRequestcontains a protected data member (accessible in derived classes) calledm_RequestStream, which is set up when called by the client-side Web Services infrastructure:public override Stream GetRequestStream() if (m_RequestStream == null) m_RequestStream = new MemoryStream(); throw new InvalidOperationException("Request stream already retrieved"); return m_RequestStream;As we would not expect the stream to be provided twice in the course of a single Web Services method call, and we also do not anticipate ever sharing
WebRequestinstances amongst multiple proxies, we provide a little protection against this happening.NOTE: I hope this is obvious but just in case, we don't do the serialization ourselves. We simply provide a container (stream) for the Web Services infrastructure to do so.
Miscellaneous common Response aspects
In addition,
MyBaseWebResponseis utilized unchanged by both MSMQ and MQ implementations. This class contains information more akin to the Internet resource world (status code, status description etc), and in fact since all errors are thrown as exceptions up the stack, the only key methods within this class areSetDownloadStream(), which allows aWebRequestcaller to deposit a response Soap stream within the class instance, andGetResponseStream()which returns this response stream to the calling .NET Web Services response processing infrastructure. Assuming a successful call, once the stream has been returned standard Soap de-serialization takes place.Common facets of a Queue-oriented transport
There are several operational aspects we would like to incorporate into our queue management.
A "Request Timeout"
The standard generated proxy includes a timeout data member, which has a default value and can be set prior to invocation of the actual Web Service call. We'd like that timeout to be honored by the Queuing implementations and we'd like them to raise exceptions like the standard protocol handlers would in the event of a timeout.
Request-Response matching (at the protocol level)
In a multi user scenario, it is likely that many requests will be outstanding at any one time, and that responses to those requests will be received concurrently. In this situation there is a requirement to be able to match up the response to the associated request. There are principally two options available to support this feature:
1. Support some form of Request ID at the application level API (i.e. in WSDL) that the Service can act on.
2. Use any native support provided by the infrastructure to correlate the request to the response.
While the first option would give us the ability to support a transport-neutral way of correlating messages, it would be at the expense of the queue protocol handlers, which would need to effectively peek at all the messages in the queue to identify a response. Taking option 1 also perverts the purity of the Application API somewhat, in that each method call is required to support an extra parameter just to allow the consolidation to occur. Finally, both forms of queuing provide support for correlation by including a
CorrelationIDslot within their version of the Queue Message. So after much agonizing, it was decided that the protocol would handle the correlation of messages (this decision may need to be revisited if more transports are added, and typically if the decision were reversed, the outcome would be the addition of custom headers into the Soap envelope to support the correlation approach). This correlation handling needs to be performed within theGetResponse()method of each implementation.A "Response Uri"
Although the transportation infrastructure was developed explicitly to support non-standard protocols, it predominantly revolves around a target endpoint and assumes a single point of contact around the delivery of the request and response. Actually, when it comes to queuing we'd like to be able to give a client the ability to describe both a request Uri (endpoint to hit) and a response Uri, which the endpoint service could effectively call back on with the results of the call. What this means in our parlance is that we want be able to describe both a request queue and a response queue for the target of the request and target of the response respectively.
Figure 4: Object Model for MySoapClientProtocol
The best approach appears to be to derive from
SoapHttpClientProtocol, and introduce a strict interface (ISoapClientProtocol) to police the management of a new data member calledResponseUrl(so named in order to bring a symmetry to the request - response Url pair).The resulting class is called
MySoapHttpClientProtocol. Introduction of this class requires that each generated proxy derive from it in order to be able to access the new property. This will require a manual re-pointing of the associatedreference.csfile.A point worth bearing in mind is that any changes made to this proxy are lost during regeneration of the proxy using the VS.NET menu item "Update Web Reference" or when the Web Reference is removed. It may be worth backing up the
reference.csfile once the changes have been hand-made to the proxy so that they may be subsequently re-applied if, for example, the underlying Web Service API definition is required to be extended and hence the proxy needs re-generating.In terms of how the response queue information might be used, one could imagine an implementation within the handlers
GetResponse()method, which waited on the relevant response queue for a message corresponding to the request it had sent. This typically involves using Queue-specific API such asReceiveByCorrelationID()."Uri representation" of a Queue
As was suggested in the section entitled "Defining Internet and other Network Transports for Web Services" that the
Uriclass is the means by which we identify our endpoint service using URI syntax. The URI syntax is dependent upon the protocol scheme. In general, absolute URI are written as follows:
<scheme>:<scheme-specific-part>The URI syntax does not require that the scheme-specific-part have any general structure or set of semantics, which is common amongst all URI. However, a subset of URI do share a common syntax for representing hierarchical relationships within the namespace. This "generic URI" syntax consists of a sequence of four main components:
<scheme>://<authority><path>?<query>Well know examples include http formats like:
http://www.microsoft.com/webservices/testservice.asmx?wsdlIn order to make use of the pluggable protocol infrastructure we are required to map MSMQ and MQ style queue names into this syntax. As can be seen in the next section, this can be achieved with varying degrees of ease depending on which queuing implementation we are talking about.
Handling Errors
As part of the general .NET Web Services pluggable protocol infrastructure, version of the
WebRequestandWebResponseclasses throw both system exceptions likeArgumentExceptionand transport-specific exceptions (typically these take the form ofWebExceptions thrown by the specifc overriddenGetResponse()method). We'd like to map our queue-related failures into this exception structure as it nicely hides the differences between the worlds of MQ and MSMQ exception reporting. TheWebExceptionclass derives fromSystem.InvalidOperationExceptionand contains some extensions for supporting detailed error information:Figure 5: WebException Object Model
The two queuing protocols support this approach, using the Status property to reflect the following
WebExceptionStatusenumerated returns:
Status Description ConnectFailureThe remote service could not be contacted at the transport level. In our case this means that the opening of the request queue failed for a reason other than inability to resolve the queue name to a resource. NameResolutionFailureThe name service could not resolve the host name. In our case, this means that the request Queue (or Queue Manager in the case of MQ – see below) identified in the Uri was unable to be mapped to a resource. ProtocolErrorThe response received from the server was complete but the indicated an error at the protocol level. In our case, this means that a valid WebResponse-derived object was returned but that it will contain information pertaining to a failure within the relevant queued transport. This object will be made available as part of theWebException.ReceiveFailureA complete response was not received from the remote Server. In our case, this means that the reading of the response queue failed for a reason other than a timeout or inability to resolve the queue name to a resource. SendFailureA complete request could not be sent to the remote Server. In our case, this means that the placing of the message on the request queue failed after the queue had been successfully opened (connected). TimeoutNo response was received within the timeout period set for the request. In our case, this means that the request was placed successfully on the request queue by the relevant transport but no corresponding response was identified on the response queue before the timeout period was reached. These status codes are mapped from the specific MSMQ / MQ failure codes within the relevant protocol specific sections below.
MSMQ-specific implementation
All the work is done in the overridden
GetResponse()method.Accessing MSMQ functions
In order to make use of MSMQ, the
System.Messagingassembly must be included in the references for the project. For the uninitiated, this provides an easy-to-use object model for interaction with MSMQ, which covers connection to and administration of local and remote message queues, as well as the sending and receiving of messages via those queues.Figure 6: System.Messaging
Queue Name Format
There are several ways to identify an MSMQ queue – by Path, by Format Name or by Queue Label. The Format Name is the native, unique representation of the Queue. The Path is a friendly representation of the Queue, which must be resolved to a Format Name by the MSMQ Directory Services. Normally the format of a Queue Path specification as understood by MSMQ takes one of the following forms:
Queue Type Path Syntax Public queue MachineName\QueueNamePrivate queue MachineName\Private$\QueueName, .\Private$\QueueNameJournal queue MachineName\QueueName\Journal$Machine journal queue MachineName\Journal$Machine dead-letter queue MachineName\Deadletter$Machine transactional dead-letter queue MachineName\XactDeadletter$We need to be able to express this kind of Path level information in an Uri format to work with the pluggable protocol infrastructure. Unfortunately there does not appear to be a particularly nice fit with MSMQ Path Names in this case. The RFC 2396 entitled "Uniform Resource Identifiers (URI): Generic Syntax" suggests that several characters are disallowed from the URI specification and hence removed during parsing by implementations such as the .NET
Uriclass. One of these is the "\" character, which is a problem for us given the format of an MSMQ queue name. In fact, this ultimately forces us to describe our MSMQ queue resources using an alternative means – in our case using "/" to segment parts of the queue name hierarchy. This will be held at Uri level in a valid form:
msmq://./private$/QueueNameWhile this then works well with Uri, it requires two things:
The developer to remember to perform this alteration before calling the Web Service function using this transport The WebRequest implementation must convert this Uri-friendly format back to one capable of being used by MSMQ However, we'd like to describe the queue in the standard way – as it may well be being used elsewhere in the wider context of our application, may be read from a configuration file etc. So we need to provide some assistance. As we'd ideally like to not have to differentiate between caller code making requests against MSMQ and code doing so against MQ (or indeed HTTP etc.), we will add a new method called
FormatCustomUri()to theISoapClientProtocolinterface and theMySoapHttpClientProtocolimplementation class. This will allow callers of the proxy to continue to make use of MSMQ-friendly queue names, but wraps up a simple Uri-friendly alteration before the string is fed to the Uri class:// Get Queue name (from configuration file perhaps) string strMyQ = ".\\private$\\myq1"; // Create the Service proxy objHelloWorldService = new localhost.Service1(); // Set up the URI objHelloWorldService.Url = objHelloWorldService.FormatCustomUri("msmq://" + strMyQ); // Make call string[] arrRes = objHelloWorldService.HelloWorldArr("Simon");To be clear, the
FormatCustomUri()call acts as a pass-through for HTTP and MQ specifications – only affecting MSMQ queue name formats.MSMQ Processing
This section essentially describes the guts of the GetResponse() method.
First, we need to place the Soap message on the MSMQ queue described in the request Uri. As just explained, for MSMQ the Uri format of a queue name looks a little awkward and needs to be re- formatted to be MSMQ-friendly:
// Create a message System.Messaging.Message objMsg = new System.Messaging.Message(); // Simply hook in the stream as the Body of the message objMsg.BodyStream = m_RequestStream; // Here we need to interpret the MSMQ Uri a little string strQueueName = this.m_RequestUri.LocalPath.Replace("/", "\\"); if (strQueueName.ToUpper().IndexOf("PRIVATE$") > = 0) strQueueName = "." + strQueueName; // Open the Queue for writing too objQueue = GetQ(strQueueName); // Send the message under a single MSMQ internal transaction objQueue.Send(objMsg, MessageQueueTransactionType.Single); // Free resources on main Queue objQueue.Close();Making use of the MSMQ "Single" transaction type ensures a delivery of one (and only one) message onto the queue. For more sophisticated heterogeneous transactions support (e.g. covering the audit of the sending of this message to, say, a database) we would need to look at integrating this core work with .NET Enterprise Services to make use of COM+ style transactions. Note that, since we established we would use transport-specific correlation, the MSMQ message ID of the outbound message is captured and remembered at this point. We will rely on the Back End process to provide this ID as a "Correlation ID" (both MSMQ and MQ support this feature) to allow us to look for a matching response more easily:
// Get the ID of the outbound message to use as the Correlation ID // to look for on any inbound messages string strCorrelationID = objMsg.Id;Once the message is placed on the queue, we then wait for a response to the message on a queue defined by the response Uri with a Correlation ID match. We wait for the period defined by the caller who can influence the timeout when creating the proxy:
// Open the response MSMQ Queue and wait for a message // Wait for a response with the Correct Correlation ID TimeSpan tWaitResp = new TimeSpan(0, 0, m_intTimeout / 1000); objQueue = GetQ(strQueueNameResp); msg = objQueue.ReceiveByCorrelationId(strCorrelationID, tWaitResp); catch (Exception e)One of two things will happen as a result of waiting on the response Queue:
1. An error occurs (error accessing the outbound / inbound queue, timeout waiting for the message response from the Back End etc. In this case, the error received from the managed MSMQ provider is thrown direct to the caller. Referring to the general WebException status code table presented in the section named "Handling Errors", here we list the specific mapping of major MSMQ failure scenarios to corresponding WebException status codes:
WebException Status Triggering event ConnectFailure Failed to open the request queue in MSMQWebRequest::GetQ()NameResolutionFailure Failed to locate the request or response queue in MSMQWebRequest::GetQ()SendFailure Failed to put the message on the request queue in MSMQWebRequest::GetResponse()Timeout Failed to receive the corresponding message from the response queue before the timeout period was reached in MSMQWebRequest::GetResponse()ReceiveFailure Failed to receive the corresponding message from the response queue for a non-timeout reason in MSMQWebRequest::GetResponse()ProtocolError Any other failure within MSMQWebRequest/MSMQWebResponse. In this case, anMSMQWebResponseis created which details the exact failure.Note, in the latter case, an
MSMQWebResponseis created and passed as a parameter to the constructor of the WebException class – this also wraps up the actual exception which will become the 'InnerException' of theWebException:catch( … ) // Have we already processed this exception in some way? if ((e is System.Net.WebException) == false) // No - Create an MSMQWebResponse - this is a protocol specific error objMSMQWebResponse = new MSMQWebResponse( (into)QueueTransportErrors.UnexpectedFailure, e.Message, e.StackTrace); // And throw the exception throw new WebException( "MSMQ Pluggable Protocol failure.", WebExceptionStatus.ProtocolError, objMSMQWebResponse);2. A Soap message is received as a stream and the
MSMQWebResponseinstance is created and primed directly with the returned stream, in preparation for the Web Services client-side infrastructure to process it:// Create a return, and stream results into it... objMSMQWebResponse = new MSMQWebResponse(); objMSMQWebResponse.SetDownloadStream(msg.BodyStream); return objMSMQWebResponse;Websphere MQ-specific implementation
As with the MSMQ support, all the work is done in the overridden
GetResponse()method.Figure 7: IBM ActiveX classes for MQ
Accessing MQ functions
In order to make use of MQ, the only credible option available to us in Websphere MQ 5.3 is the COM wrapper that is provided with the MQ Client (MQ is optional. The demo application can run on any combination of HTTP, MSMQ and MQ transports. This is configurable within the demo). The COM wrapper must be included as a COM interop reference in our project. The additional notes section describes an alternative that improves the overall throughput when using MQ, for now the COM wrapper will suffice.
Note that the resulting interop assembly, called
Interop.MQAX200.dll, is not signed when included this way and therefore any assemblies using it will not be capable of being placed in the GAC unless a signed version of the Interop assembly can be created.This signing can be achieved by undertaking the following steps:
locate the COM DLL mqax200.dll, which can be found for a default installation within theC:\Program Files\IBM\WebSphere MQ\binfolderif a strong name is not already used within the development team, using sn –k <KEYFILENAME>to generate a file containing a strong name key pairrun the tlbimputility on the COM DLL file, specifying the appropriate keyfile as an option/keyfile<KEYFILENAME>and supplying a name for the output "signed" resultant assembly using/out<OUTFILENAME>Once this has been done, and the resultant assembly included as a reference to the project, the host project could then be signed and deployed to the GAC as required.
Queue Name Access and Format
The model for accessing an MQ queue is different to that for MSMQ. Principally, whereas MSMQ object hierarchy is fairly "flat" in nature, MQ employs a hierarchical structure that involves a 3-tier access consisting of:
Acquiring an MQ Session Connecting to a named MQ Queue Manager via the Session Requesting access to a named MQ Queue via the Queue Manager Further to this, the name of a queue is built from two parts:
<queue manager name>/<queue name>e.g.QM_yoda/queue32A major thing to note here is that, unlike MSMQ, entity names are case sensitive. Indeed there are several 'gotchas' when working with a combination of the Uri class (which if you remember lower- cases host names). When using the framework, a tell tale sign of a naming issue is a
WebExceptionwith a status code ofNameResolutionFailure, indicating either the Queue Manager or Queue could not be resolved (see the detail below). Some examples of queue access follow.If connecting to a queue called "fred" the default Queue Manager, one would use a Uri notation like:
objHelloWorldService.Url = "mq://fred"If connecting to a queue called "bill" on a Queue Manager called "QM_john" , the Uri for this would be:
objHelloWorldService.Url = "mq://QM_john/bill"Hopefully this gives a flavor of MQ queue nomenclature. In order to help unbundled a Uri into a Queue Manager / Queue name combination, a utility method called
ResolveManagerAndQueueName()is provided, and the transport code uses this like so:// What is the Queue Manager and Queue Name we need to direct the message at? ResolveManagerAndQueueName( this.m_RequestUri, out strQueueManagerName, out strQueueName);MQ Processing
Apart from the obvious semantic differences described in the last section, the main differences between this implementation and the MSMQ version are in the syntax when sending messages and waiting on a response Queue. Also, whereas MSMQ provides good support for direct use of streams when sending and receiving messages, there is no such support for MQ (at least when using the ActiveX wrapper) and we must rely predominantly on sending and receiving byte arrays instead.
So, since MQ does not support the automatic use of streams as the body of a message, we need to convert the stream to an array of bytes, which can then be used to prime the outbound message:
// Access the Soap serialised stream as an array of bytes byte[] bytBody = new Byte[m_RequestStream.Length]; m_RequestStream.Read(bytBody, 0, bytBody.Length);Given we have the Soap body, and have resolved the request and response Queue Manager / Queue Name combinations (shown above), the first thing to do is to get an MQ Session. This is required before any queuing work may be attempted:
// Access a Session objMQSession = new MQAX200.MQSessionClass();Next, we attempt a connect to the Queue Manager, and ask it for a Queue object given the name:
// Get the named Queue Manager objQueueManager = GetQManager(objMQSession, strQueueManagerName, true); // Get the named Queue from the Manager objQueue = GetQ(objQueueManager, strQueueName, true);Once this has succeeded, we construct a message setting up message options, and write the body of the message:
// Set up the message to send objMsg = (MQAX200.MQMessage)objMQSession.AccessMessage(); objPutMsgOpts = (MQAX200.MQPutMessageOptions)objMQSession.AccessPutMessageOptions(); // We need to send a byte array to cover off binary attachments etc.... objMsg.Write(bytBody);Putting the message on the queue uses the (default) message options we just set up :
// Finally, put the message to the queue objQueue.Put(objMsg, objPutMsgOpts);[NOTE: If conversing with MQ running on AS/400 which operates in EBCDIC, the message would explicitly need to indicate it was in ASCII format prior to sending - objMsg.Format = "MQSTR " ;]
Note that, since we established we would use transport-specific correlation, the MSMQ message ID of the outbound message is captured and remembered at this point. We will rely on the Back End process to provide this ID as a "Correlation ID" (both MSMQ and MQ support this feature) to allow us to look for a matching response more easily:
// Get the ID of the outbound message to use as the Correlation ID // to look for on any inbound messages string strCorrelationID = msg.MessageId;Once the message is placed on the queue, we then wait for a response to the message on a queue defined by the response Uri with a Correlation ID match. We wait for the period defined by the caller who can influence the timeout when creating the proxy:
// Open the response MQ Queue and wait for a message objQueueResponse = GetQ(objQueueManager, strQueueNameResp, false); objMsgResp = (MQAX200.MQMessage)objMQSession.AccessMessage(); objMsgResp.CorrelationId = strCorrelationID; objGetMsgOpts = (MQAX200.MQGetMessageOptions) objMQSession.AccessGetMessageOptions(); objGetMsgOpts.Options = (int)MQAX200.MQ.MQGMO_SYNCPOINT + (int)MQAX200.MQ.MQGMO_WAIT; objGetMsgOpts.WaitInterval = this.Timeout; objQueueResponse.Get( objMsgResp, objGetMsgOpts, System.Reflection.Missing.Value); // Use the appropriate reader strMsgRecv = objMsgResp.MessageData.ToString(); catch (Exception e)One of two things will happen as a result of waiting on the response Queue:
1. An error occurs (error accessing the outbound / inbound queue, timeout waiting for the message response from the Back End etc. In this case, the error received from the managed MSMQ provider is thrown direct to the caller. Referring to the general WebException status code table presented in the section named "Handling Errors", here we list the specific mapping of major MSMQ failure scenarios to WebException status codes:
WebException Status Triggering event ConnectFailure Failed to open the request queue manager in MQWebRequest::GetQManager()or failed to open the request queue inMQWebRequest::GetQ()NameResolutionFailure Failed to locate the request or response queue in MQWebRequest::GetQ()SendFailure Failed to put the message on the request queue in MQWebRequest::GetResponse()Timeout Failed to receive the corresponding message from the response queue before the timeout period was reached in MQWebRequest::GetResponse()ReceiveFailure Failed to receive the corresponding message from the response queue for a non-timeout reason in MQWebRequest::GetResponse()ProtocolError Any other failure within MQWebRequest/MQWebResponse. In this case, anMQWebResponseis created which details the exact failure.2. A Soap message is received as a byte array, converted to a stream, and the
MQWebResponseinstance is created and primed directly with the converted stream, in preparation for the Web Services client-side infrastructure to process it:// Convert the result into a byte array byte[] buf = new System.Text.UTF8Encoding().GetBytes(strMsgRecv); MemoryStream stResponse = new MemoryStream(); stResponse.Write(buf, 0 , buf.Length); // Create a return, and stream results into it... objMQWebResponse = new MQWebResponse(); objMQWebResponse.SetDownloadStream(stResponse); return objMQWebResponse;Tapping into the serialized Soap stream
As explained above, when implementing a
WebRequest–derived class, we need to provide a stream for the Soap serialization to make use of. Then before we can send this Soap "request stream" to any network resource via one of our new transports, we must first get access to this serialized stream.The
WebRequest–derived class we introduced above from which we shall derive our MSMQ and MQ variants contains a data member calledm_RequestStreamof base typeStream, specifically to carry the outbound Soap stream. The infrastructure will call our overridden methodGetRequestStream()at the appropriate time to create and provide this stream during the outbound delivery phase of a Web Service call. Our initial attempt will use a member variable of typeMemoryStreamHowever, it appears (from a degree of single stepping through the code flow) that there is no direct access in
WebRequestderivations to the request stream until ourGetResponse()override is called (at which point the request stream has already been disposed of – see theLengthproperty ofm_RequestStreamas an illustration of this):Figure 8: Access to the Soap request stream
Observation of the UML Sequence diagram presented earlier in the Case Study will confirm that the infrastructure has already closed the stream at the point we need it. We need to come up with an alternative approach to gain access to the stream before it is disposed of. It turns out that the Microsoft infrastructure has to
Close()the stream you provide so that the data is pushed out onto the wire - this is the standard programming model forWebRequests.The following two options are available to us:
Tapping the stream just after serialization by creating and using a SoapExtensionand an associated attribute. We could hook theSoapMessageStage.AfterSerializeprocessing stage to store the serialized Soap stream. This will add extra classes to the mix, cause more rework of the (re-)generated proxy and may cause us issues as we develop our framework onwards to include WSE.Provide a custom stream, and override Close()to modify the streams behavior - effectively delaying the actual close of the stream until we've had chance to access it. This is the preferred approach as it fits more neatly with the framework.Figure 9: MySoapStream
So, we provide a class called
MySoapStream, suppressing the actual closure of the stream and providing an alternative calledInternalClose(), which actually closes the underlying Stream.public override void Close() // DON'T CLOSE! BUT REWIND! m_Stream.Position = 0; internal void InternalClose() m_Stream.Close();We also change the implementation of the GetRequestStream() method to use this new type of stream:
public override Stream GetRequestStream() if (m_RequestStream == null) m_RequestStream = new MySoapStream(new MemoryStream(), true, true, true); throw new InvalidOperationException("Request stream already retrieved."); return m_RequestStream;Hence, revisiting the MSMQ WebRequest handler as an example, the
GetResponse()override alters slightly (MSMQ version shown but both applicable) looks like:public override WebResponse GetResponse() // Create a message using the Stream objMsg.BodyStream = m_RequestStream; objMsg.Recoverable = true; // Open the Queue for writing objQueue = GetQ(strQueueName); // Send the message under a single MSMQ internal transaction objQueue.Send(objMsg, MessageQueueTransactionType.Single); // And close Soap Stream m_RequestStream.InternalClose(); // Now wait on a responseIn this case, the stream may be used directly as the MSMQ message body using the
BodyStreamproperty of the latter. While there is no equivalent direct-stream linkage in MQ, only one more step is required to read the stream into an array of bytes, which can then be written directly to the MQ message.Once the message has been sent, we can close the stream ourselves using our
InternalClose()method.Hooking the new protocols into the generated proxy classes
Now we have two new transport protocol handlers, let's make them useable from the proxy class. The resultant proxy now looks a little different to its original "freshly-generated" form:
public class Service1 : MySoapClientProtocol // The individual methods of the API are unchanged from their generated form [System.Web.Services.Protocols.SoapDocumentMethodAttribute(...)] public string[] HelloWorldArr(string name) object[] results = this.Invoke("HelloWorldArr", new object[] {name}); return ((string[])(results[0])); // The individual methods of the API are unchanged from their generated form // Overridden methods for directing Soap messages protected override WebRequest GetWebRequest(Uri uri) // Delegate to a common generator for new protocols return ProxyCommon.GetWebRequest(uri, this); protected override WebResponse GetWebResponse(WebRequest webReq) // Delegate to a common generator for new protocols return ProxyCommon.GetWebResponse(webReq);The proxy now derives from
MySoapClientProtocol, including overrides forGetWebRequest()andGetWebResponse()calls. This constitutes the only changes required. Note that these changes will be lost and will need reapplying if the web service proxy is subsequently regenerated for any reason.Using the new proxy from a client
Now the hard work is out of the way, clients of the proxy can make use of it in the following way:
localhost.Service1 objHelloWorldService = null; string strRequestUrl = "msmq://.\\private$\\myreq"; string strResponseUrl = "msmq://.\\private$\\myresp"; // Create the Service proxy objHelloWorldService = new localhost.Service1(); // Set up the request Queue via the Uri objHelloWorldService.Url = objHelloWorldService.FormatCustomUri(strRequestUrl); // Set up an alternative response Queue other than "<QUEUENAME>_resp" objHelloWorldService.ResponseUrl = objHelloWorldService.FormatCustomUri(strResponseUrl); // And the timeout objHelloWorldService.Timeout = vintTimeoutInSecs * 1000; // Run the method and print out the return Console.WriteLine(objHelloWorldService.HelloWorld("Simon")); catch(Exception e)Support for asynchronous message delivery
There are numerous articles describing how to take advantage of asynchronous support in Web Services (see foot of the article for details), so we'll not dwell on this subject too much. Suffice to say there are three basic options when implementing asynchronous calling:
Polling for Completion Using WaitHandles Using Callbacks Whichever method is chosen they all rely on access to a token returned to them once an asynchronous call has begun. The generated proxy classes have synchronous and asynchronous versions of each method call – the asynchronous ones include a pair of methods of the form:
public System.IAsyncResult BeginHelloWorldArr( string name, System.AsyncCallback callback, object asyncState) return this.BeginInvoke("HelloWorldArr", new object[] {name}, callback, asyncState); public string[] EndHelloWorldArr(System.IAsyncResult asyncResult) object[] results = this.EndInvoke(asyncResult); return ((string[])(results[0]));The
BeginXXXmethod call will return immediately with a token of typeIAsyncResult, which may be used to poll or wait on request completion.While not particularly complicated, asynchronous request support is nicely encapsulated in an assembly like
AsyncUIHelperas provided in the MSDN article [3]. Use of this library means that our queue transports don't need to implement the asynchronous features for themselves (well for now, anyway).The key aspect of the support is a class called
Asynchronizer:private Asynchronizer m_ssiReturnImage = null;The caller of a method would include the use of the Asynchronizer as follows, in our example first hooking up the delegate-based callback, and then invoking the standard
ReturnImage()call:// Create the Service proxy objHelloWorldService = new localhost.Service1(); // Set up the URI objHelloWorldService.Url = objHelloWorldService.FormatCustomUri(vstrURI); // And the timeout objHelloWorldService.Timeout = vintTimeoutInSecs * 1000; // Create a new Async token m_ssiReturnImage = new Asynchronizer( new AsyncCallback(this.ReturnImageCallback), objHelloWorldService); // Begin the method call IAsyncResult ar = m_ssiReturnImage.BeginInvoke( new ReturnImageEventHandler(objHelloWorldService.ReturnImage), null);Any asynchronous responses would be directed to an event handler based on a delegate, which is defined with the return type of the standard synchronous version of the method call we are "asynchronizing":
protected delegate byte[] ReturnImageEventHandler();In this case, the responses to the call (namely the image data in this case) are accessible through a data member in the asynchronous token:
protected void ReturnImageCallback(IAsyncResult ar) localhost.Service1 objHelloWorldService = (localhost.Service1)ar.AsyncState; AsynchronizerResult asr = (AsynchronizerResult) ar; byte[] bytImage = (byte[])asr.SynchronizeInvoke.EndInvoke(ar);Faking it – the "Back-end Service"
So far, we've majored on what it takes to get the client side talking in the right way to the request and response queues. This is pretty meaningless unless we have some form of service to pick off messages and process them. In real life solutions, one would expect the use of Web Services to be targeted at heterogeneous environments where the client and server run on different platform and Web Services is a nice way to mesh the two worlds.
However, due to a desire to keep the skill set required to keep up with the article to a manageable amount, we'll use .NET application at the back to consume the messages and produce (dummy) responses in order to allow the clients message reception to be tested. It should be noted that the service exists purely to act as a counterpoint to the client framework we have developed, and so very little work has gone into making it robust or scalable – but there's enough there to demonstrate the client capability.
In the non-WSE arena we are currently describing, the Service really can be quite basic. Its job is to perform the following steps:
Read a configuration file containing the MSMQ / MQ Queues to wait on Loop the queues awaiting messages for a short period of time For any messages positively identified, a response Soap stream is manually generated and placed on the response queue for picking up by the client The logic is contained within one class,
BEService.This first reads the supplied app.config file for all application level settings (that is those within the
<appsettings>node):// Read configuration settings NameValueCollection colSettings = ConfigurationSettings.AppSettings;Each queue found in the configuration file is added to the relevant pot:
string[] arrVals = colSettings.GetValues(strKeyName); if (arrVals[0].ToLower().IndexOf("msmq://") == 0) arrMSMQQueuesToMonitor[intNumMSMQQueues++] = arrVals[0].Replace("msmq://", string.Empty); if (arrVals[0].ToLower().IndexOf("mq://") == 0) arrMQQueuesToMonitor[intNumMQQueues++] = arrVals[0].Replace("mq://", string.Empty);The MSMQ and MQ queues are distilled from the collection, as is a polling delay (the back end service does not currently employ a thread pool). The following simple loop is then entered:
// Wait on some queues - some MSMQ, and some MQ... for (;;) // Do MSMQ queues first foreach (string strName in arrMSMQQueuesToMonitor) WaitOnMSMQQ(strName, intPollDelay); // Now MQ queues foreach (string strName in arrMQQueuesToMonitor) WaitOnMQQ(strName, intPollDelay);WaitOnMSMQ implementation
This method is straightforward. First we wrap all work in a try-catch block in order to trap all exceptions and filter out "healthy" ones such as a timeout waiting on request message:
… // Do the work here catch(Exception e) // Ignore timeouts as this simply means no queue message was there if (e.Message.IndexOf("Timeout for the requested operation has expired") < 0) TSLog("Exception caught in WSAltRouteBEFake[MSMQ]: " + e.Message + e.StackTrace);What we'll do first is wait a while:
// We'll need to wait a little time TimeSpan tWaitResp = new TimeSpan(0, 0, 0, 0, vintPollTimerDelay); // Get hold of the request queue MessageQueue objQueue = GetQ(vstrQueueName); // Wait for a message Message objRequestMsg = objQueue.Receive( tWaitResp, MessageQueueTransactionType.Single); // Close the Queue objQueue.Close();Assuming we got a message, i.e. no exception has been thrown, we open up the message and get the data content into a byte array, while picking off some salient attributes such as a potential Correlation ID and response queue name:
// Get the message content direct from the BodyStream string strResp = string.Empty; byte[] bufIn = new Byte[objRequestMsg.BodyStream.Length]; objRequestMsg.BodyStream.Position = 0; objRequestMsg.BodyStream.Read(bufIn, 0, (int)msg.BodyStream.Length); string strCorrelationId = objRequestMsg.Id; // Close stream objRequestMsg.BodyStream.Close(); // Get the response queue - we shall assume a default but // hope the client has identified a queue for us to call back on string strResponseQueue = vstrQueueName + "_resp"; if (objResponseMsg.ResponseQueue != null) strResponseQueue = objRequestMsg.ResponseQueue.QueueName;We then check to see if this is a known message and if so, build a response message for it. There is very little intelligence in the matching method or generation of the response message – it exists purely to serve the client response code.
// Fake out a response to the message strResp = BuildFakeResponseFromBytes("MSMQ" , bufIn, strResponseQueue);
If we identified a request and were able to generate a response for it, then send it back to the client via the response queue using the Correlation ID:
// If we have a valid response...use it if (strResp.Length > 0) // Get the response queue objQueue = GetQ(strResponseQueue); // Send a message back MemoryStream stResp = new MemoryStream(); stResp.Write(new UTF8Encoding().GetBytes(strResp), 0, strResp.Length); Message objMsg = new Message(); objMsg.BodyStream = stResp; objMsg.Recoverable = true; objMsg.CorrelationId = strCorrelationId; // Send the message under a single MSMQ internal transaction objQueue.Send(objMsg, MessageQueueTransactionType.Single); // Free resources on response Queue objQueue.Close();Here we are geared up to use the
BodyStreamproperty of the MSMQ message once again, so we write the contents of our response string into the stream. We send as a single MSMQ transaction once again, and then tidy the response queue.That's really all that is required to effect basic end-end Web Service processing over the medium of queues. We could stop at this point or exploit a very interesting new technology that will help us secure our messages independent of the transport over which we are running them.
How Web Services Enhancements (WSE) alters the picture for Client and Server
The WSE is Microsoft's initial implementation of some of the first 2nd generation GXA Web Services specifications - WS-Security, WS-Attachments (via DIME) and WS-Routing / WS-Referral. It was released in Dec 2002, after 4 months in beta test. The whole standardization push continues the initiative to improve the capability for interaction / interoperability within the web service stack and implementations of these particular standards will really form the baseline for entry-level trading partner applications based around the Internet.
While it is anticipated that Soap over a queued transport will find use within the trusted networks of a single organization, this does not negate the possibility that the data will require protection in transit between enterprise applications (e.g. sensitive payroll data).
So, the main aspect of the WSE that is interesting to us in this article stems from the implementation of the WS-Security specification; namely the ability to:
control the "valid lifetime" of a Soap message, generate a "username token" with an associated password which identifies a user for verification, sign a Soap payload with that token to allow for tamper checking, encrypt the payload of the Soap body using a symmetric key (known as a "shared secret" since the Back End Service will have access to the same key), secure the Soap requests and responses to make this an end-end-end solution. It should be noted that this functionality represents a subset of what the WSE is capable of providing. Interesting features we are not using here include the use of binary security tokens (which could be capable of housing a Kerberos ticket for example) and asymmetric encryption using X509 certificates.
In addition, as we are using WSE and WSE implements the WS-Attachments specification, we can look at extending our demonstration application to explore the ability to add binary attachments via DIME support. It will become evident that the DIME support does not follow the standard Soap processing model – expect to see it's implementation alter over the next 12 months.
WSE overview
The WSE is a data processing engine, which essentially applies (under instruction) a series of transformations to outbound Soap messages once they have been serialized by the .NET Web Services infrastructure, and conversely applies further transformations on incoming Soap messages prior to their de-serialization into objects / parameters. These transformations involve the inclusion of additional Soap headers and in some cases (like encryption) alterations to the body of a Soap message as well.
The API provided by the WSE is relatively low level which makes it great for getting in amongst and influencing the operation of both client and server side. A great article covering the inner workings of the WSE can be found in [4]. We'll draw out the salient points in the sections that follow to give us some context for the changes we shall make to our client and server.
The physical manifestation of the WSE 1.0 is a single GAC-installed assembly called
Microsoft.Web.Services.dll, which implements the majority of the three specifications named above.The WSE Filter Pipeline Architecture
The single most major change between the beta version of the software (called the WSDK, released Aug 2002) and the WSE 1.0 is that the functionality effecting the transformations on Soap messages has been repackaged into a series of filters. These filters are one of two varieties, Output filters concentrate on manipulation of an outbound Soap message, whereas Input filters operate on incoming Soap messages.
Figure 10: Filter Pipeline Architecture
Input Filter Output Filter Purpose TraceInputFilterTraceOutputFilterWrite messages to log files to help with debugging SecurityInputFilterSecurityOutputFilterAuthentication, signature and encryption support TimestampInputFilterTimestampOutputFilterTimestamp support ReferralInputFilterReferralOutputFilterDynamic updates to routing paths RoutingInputFilterRoutingOutputFilterMessage routing Each set of filters is organized into either an input filter or output filter pipeline, which out-of-the-box runs the filters in the following order (for Client-to-Server flow only):
Figure 11: Filter Pipeline Architecture in more detail (Client -> Server flow)
Amongst other things this ensures that the encryption and signing (Security) are applied to a Soap payload that all other modifications have been completed on and that securing happens just prior to sending of a request, whereas decryption and signature verification happen straight after reception of a request and before any other data sensitive operations can take place.
The main thing to remember here is that each pipeline operates on a Soap envelope to produce different content within that envelope.
The pipeline concept within the WSE is a very powerful one, as pipelines can be manipulated programmatically to remove or reorder stages, and new custom pipeline stages may be introduced based on the WSE-provided base classes
SoapOutputFilterand / orSoapInputFilter.The integration of the WSE pipeline execution of input and output filters with the standard Soap serialization and delivery mechanisms needs to be managed in two different ways:
For ASP.NET Web Service clients, this is achieved by deriving the generated proxy class from a new proxy base class called Microsoft.Web.Services.WebServicesClientProtocol.For ASP.NET Web Services, a new server-side Soap extension has been provided in the form of Microsoft.Web.Services.WebServicesExtension.The purpose of the new proxy base class and server side extension is to sit between the transport and Soap serialization / de- serialization modules, intercepting messages and building / analyzing Soap headers as appropriate. As a result it is highly possible that when using, say, signature verification within a Web Service, should this verification detect tampering and generate an appropriate Soap fault, not a single line of application business logic will ever have been called by the time the fault is generated. This nicely insulates application code from WSEs infrastructure code, which seems an attractive benefit.
The WSE SoapWebRequest and SoapWebResponse
The WSE itself employs the .NET Web Services pluggable protocol infrastructure we talked about earlier to provide two new communication classes called
SoapWebRequestandSoapWebResponse. As with our MSMQ and MQ versions, these are derived fromSystem.Net.WebRequestandSystem.Net.WebResponserespectively.The
SoapWebRequestclass parses the request stream containing the standard serialized Soap message into an instance of another WSE-provided class called theSoapEnvelope, which is derived fromSystem.Xml.XmlDocument. It then drives the standard output filter pipeline, which conditions the contents of the envelope as described in the section above.Our interest within the framework will revolve around integrating our queue-based classes with these new Soap web request / response classes, and in particular picking up the resultant Soap stream for use by our protocol handlers.
However, in the case of our Back End Service,
SoapWebRequestandSoapWebResponseare irrelevant to us as we are operating outside the client or ASP.NET server infrastructure and must "hand-craft" WSE processing ourselves.The WSE SoapContext for communication
We've talked about the operation of the WSE but so far not about how this processing is influenced by any application code we may write. The SoapContext class is the vehicle by which application code and WSE formatting code indirectly communicate. This happens in one of two ways, depending on the point in the process:
When building Soap headers during outbound processing, the application first primes attributes of the SoapContext, which become the instructions for the filters to operate against i.e. in this case it is theSoapContextattributes that shape the content of the outgoing Soap message.When analyzing headers during inbound processing, the WSE primes a SoapContextwith the "results" of the inbound filtering, prior to handing control to the server side application i.e. in this case it is the content of the incoming Soap message that shapes the setup of theSoapContext.The class contains several attributes of interest:
Figure 12: WSE Soap Context
The Envelope contains XML marking the "work in progress" as the filters are run. The Security attribute is a place to hang encryption KeyInfo and signature information off. Attachments hold DIME attachments – more of these later.
It will come as no surprise to know that both the client and server sides support the exposure of the
SoapContextvia theSoapWebRequestclass at the client and through the staticHttpSoapContext.RequestContextandHttpSoapContext.ResponseContextclasses within an ASP.NET the Web Service. This then is partly fine for us as our client should be capable of manipulating aSoapContextin the hope that the filters will be influenced accordingly. However, as per usual, the Back End service is not hosted in ASP.NET land and so we will have to hope there is another way to retrieve aSoapContextin this environment.The SecurityProvider architecture
As explained earlier, the WSE infrastructure is based on interception and it keeps the security checking distinctly separate from any application business logic. This raises an interesting question, since the sending side will be doing the generation of a "user token" incorporating a user name and an associated (hashed in our case) password, signing and symmetrically encrypting the Soap body etc. but it would follow that there needs to be knowledge of both the user name / password and a corresponding key for decryption at the target end, if the Soap payload is to be successfully processed.
The WSE provides hooks in it's inbound processing to facilitate this. Specifically it provides two interfaces,
IPasswordProviderandIDecryptionKeyProvider. The former would be implemented by application code to provide the password for a given user, typically from a database of users. The latter would be implemented to provide the decryption key. Both would be called during WSE inbound message processing if the Soap headers indicate the message has been signed and / or encrypted:Figure 13: WSE Security Provider architecture
The WSE allows the developer to write code for these 2 hooks and then make them known to the WSE- runtime via information in the relevant application configuration file. There are 3 possibilities as to what this file may be, depending on environment:
For ASP.NET Web Services, it is the web.config file For ASP.NET clients, it is the app.config file For standalone programs it is also the application configuration file The entries for both providers would appear something like:
<microsoft.web.services> <security> <passwordProvider type="WSCommon.WSESecurityProvider, WSAltRouteBEFake" /> <decryptionKeyProvider type="WSCommon.WSESecurityProvider, WSAltRouteBEFake" /> </security> </microsoft.web.services>Each entry indicates the name of a Type (class) and the assembly in which it should be found. Hence, for two-way security (i.e. Soap request and response messages), one would expect to find similar entries both at the Service end (used when receiving a Soap request) and the Client end (used when receiving a Soap response).
The WSE 1.0 Settings Tool for .NET
The only issue with the flexibility of hooking in these providers is that it's pretty easy to get wrong - or maybe that's just me! Anyway to that end (and not just for me) Microsoft have also produced an unsupported VS.NET add-in [8], which allows these settings to be added more easily. Once installed, a new context menu item appears when right clicking a node at project level in the Solution Explorer. This allows developers to enable WSE security then specify the provider information in a dialog box and have it injected into the relevant configuration file. In this case, we have indicated that our Back End Service will contain a single class called
WSCommon.WSESecurityProviderthat will handle both password and decryption key resolution.Figure 14: WSE Settings tool
![]()
Note that the authors suggest that the tool will only work with WSE 1.0, so may well not work with future updates of the WSE, but by then we should all be a lot more comfortable with configuration settings.
Updating the Client framework to use WSE
OK, enough of the theory. There are a specific set of changes required to move our framework to a point where it can handle both non-WSE and WSE requests.
Addition of a complimentary class to MySoapClientProtocol
Since WSE expects a proxy to derive from
Microsoft.Web.Services.WebServicesClientProtocolin order to invoke filter pipeline processing, we need to introduce a complimentary class to our framework calledMyWSESoapClientProtocol, which will also support aResponseUrlproperty:Figure 15: WSE access from the client
Changes to the generated proxy
The only alterations to the proxy code involve rebasing on
MyWSESoapClientProtocoland slight alteration of the overriddenGetWebRequest()method call and removal of the overriddenGetWebResponse()method call:public class Service1 : MyWSESoapClientProtocol // All API method calls remain unchanged // Overridden portions protected override WebRequest GetWebRequest(Uri uri) // Register our scheme if needs be ProxyCommon.RegisterRelevantScheme(uri, this); // And do the default thing using SoapWebRequest, which will integrate // nicely with our MSMQ / MQ WebRequest / WebResponse classes when it // needs to. return base.GetWebRequest(uri);This works due to the beauty of the
SoapWebRequest, which is able to seamlessly hook out to filter processing code of the WSE, and then deliver the resultant Soap envelope to our transport of choice further down the line for transmission. The rest of the framework code (ourMSMQWebRequest,MQWebRequestetc.) is not required to change at all.Changes to the calling code
Of course, as we discussed earlier, the application is required to setup the
SoapContextobject associated with the call to be made. This is achieved by adding a new private method to the client, which adds the basic information to theSoapContext. A caller will then initiate the call as follows:// Create the Service proxy objHelloWorldService = new localhostWSE.Service1(); // Set up the URI objHelloWorldService.Url = objHelloWorldService.FormatCustomUri(vstrURI); // Augment request context with WSE goodies via our common routine // This includes: // * Time-to-live of 60s // * User name and password credentials // * Encryption and Signing // To achieve this we shall be prodding attributes into an initially empty // SoapContext!! ProxyCommonWSE.SetupSOAPRequestContext(objHelloWorldService.RequestSoapContext); // Run the method and get the return string strResult = objHelloWorldService.HelloWorld("Simon"); // Verify that a SOAP request was received, with valid credentials // and a signature via our common routine – we will be doing this by // interrogating the generated SoapContext!! ProxyCommonWSE.ValidateCredentials(objHelloWorldService.ResponseSoapContext);To give an insight into how the SoapContext object is primed, here is a portion of the private method
SetupSOAPRequestContext():// Get a symmetric encryption key to encrypt the Soap body with EncryptionKey encKey = GetEncryptionKey(); // Create a User Token from a tick count and static User Name UsernameToken userToken = new UsernameToken( strUserName, CalculatePasswordForProxyUser(strUserName), PasswordOption.SendHashed); // Stick it in the Soap Context vobjSoapContext.Security.Tokens.Add(userToken); // Request a signature for the message (based on the User Token in our case) vobjSoapContext.Security.Elements.Add( new Microsoft.Web.Services.Security.Signature(userToken)); // Request the Soap body be encrypted (with a separate symmetric key in our // case) vobjSoapContext.Security.Elements.Add( new Microsoft.Web.Services.Security.EncryptedData(encKey)); // Set an expiry on this SOAP message too vobjSoapContext.Timestamp.Ttl = SOAP_MSG_EXPIRY_TIME; // 60sSalient points include the ability to send a 160-bit SHA-1 (non-reversible) hash of the password. The idea is that this hash will be tested against a computed version on the far side once the
SecurityProviderhook has been called on that side. Signing and encryption are requested using different keys, and the message is made to expire inSOAP_MSG_EXPIRY_TIME(60 seconds).Care should be taken when encrypting and / or signing a Soap message. Options exist to express the granularity at which the message should be secured from everything down to individual Soap message elements. Although we have not made use of the Routing features within the WSE here, the reader should be aware that care should be taken when securing a message that is expected to flow via intermediate nodes on the way to its ultimate destination. Primarily the reason for this is that the Soap processing model allows for removal of Soap headers by the intermediate stages. So for example, if the whole message (including headers) is signed, and then delivered from processing node A to node C via node B, there is a possibility that node B may be able to accidentally invalidate the message by removing a header which will cause a signature verification failure at the ultimate destination.
A special note should be made here of current weirdness observed when running a proxy derived from
WebServicesClientProtocol(via ourMyWSESoapClientProtocol) class in a scenario but where nothing is set up in theSoapContextobject prior to a call being made. In this case one might expect that the non-WSE-enabled receiving application should happily be able to consume the Soap message. This is not the case. Two issues have been observed to date, both centered on odd header behavior:You will receive a "The path does not contain an <action> element." exception when running any Web Services method call. This appears to happen only in custom transport cases – the standard HTTP transport seems to cover this off internally. Having explicitly set this up by priming the routing path with something like vobjSoapContext.Path.Action = string.Empty, although one might have expected nothing else to be added to a Soap message, and therefore that for a backend not running a WSE- conformant toolkit, messages would be consumed quite happily. This is not the case. As well as Routing information, Timestamp header information is added, and we've seen specific cases of Java back-end services react very oddly to this and this Routing information, mainly on the grounds of thesoap:mustUnderstand="1"attribute embedded in the Soap fragment representing the latter – this tells the receiving end to terminate processing if it can not make sense of the particular header with which this attribute is associated.This is a shame since it means that instead of being able to derive a proxy from a single class (
MyWSESoapClientProtocol) and use the presence or absence of calls to ourSetupSOAPRequestContext()method to control securing of Soap messages, we currently need to have two versions of the proxy, one derived fromMyWSESoapClientProtocoland the other derived fromMySoapClientProtocol. Further, we must know when to use each (largely) at design time. As I understand it, these issues are known but constitute further wrinkles that will be ironed out as the standards mature.The
soap:mustUnderstandflag can be influenced by the sender of the message so this gets round a portion of the issues.Updating the Back-end Service to use WSE for secure communications
Given the Service we have described earlier in the piece is merely an executable picking messages off a queue, it lives very definitely outside the cocoon that ASP.NET Web Services provides, and because of this we must use the WSEs raw pipeline facilities to drive the input filter pipeline when verifying signatures and decrypting content in order to get at the payload. Conversely, as we support two-way security, we must also drive the output filter pipeline for the encryption and signing services to kick in prior to returning the Soap response (basically all the work that is done literally for free in a traditional HTTP-hosted ASP.NET Web Service).
Preparing the SecurityProvider
The configuration definitions we discussed earlier in the section "The SecurityProvider architecture" need to be made. This directs the WSE to the right place when asking for the user password and symmetric key for decryption.
Invoking the inbound message pipeline for requests
Assuming we got a message, i.e. no exception has been thrown, we open up the message and get the data content into a byte array, while picking off some salient attributes such as a potential Correlation ID and response queue name:
// Get the message content direct from the BodyStream string strResp = string.Empty; byte[] bufIn = new Byte[objRequestMsg.BodyStream.Length]; objRequestMsg.BodyStream.Position = 0; objRequestMsg.BodyStream.Read(bufIn, 0, (int) objRequestMsg.BodyStream.Length); string strCorrelationId = objRequestMsg.Id; // Process through WSE - this may be a passthru operation // if WSE was not used to create the stream // Hmmm - we need to see if this is DIME based or not // To do this we shall simply see if we can read the stream // as a DIME stream - creating an Envelope in any case SoapEnvelope env = InputStreamToEnvelope(objRequestMsg.BodyStream); // Close stream objRequestMsg.BodyStream.Close(); // Create Pipeline - with default filter collection for input and output // (i.e. we are not overriding default behaviour in any way) Pipeline objWSEPipe = new Pipeline(); // Run the default input pipeline objWSEPipe.ProcessInputMessage(env); // Extract the resultant Envelope bufIn = new UTF8Encoding().GetBytes(env.OuterXml); // Get the response queue - we shall assume a default but // hope the client has passed us a queue to call back on string strResponseQueue = vstrQueueName + "_resp"; if (objResponseMsg.ResponseQueue != null) strResponseQueue = objRequestMsg.ResponseQueue.QueueName;The emboldened text shows the hydration of a
SoapEnvelopeobject from the inbound message stream. Once we have this envelope (more of how we actually get it later), we can create a WSE pipeline and run the default input filters against the envelope. We care not what the contents of the input pipeline are (although, since we know we are not using routing, we could actually programmatically remove the Routing and Referral filters from the input pipeline which would slightly improve the speed of the pipeline run).The
ProcessInputMessage()call is pretty much an atomic operation which will result in a clear text version of the Soap message being left in the env variable. As a side effect of the run, theContextproperty within the envelope object will have been populated with the interpretation of the inbound Soap message.One rather cool feature is that our
SecurityProvidersupport will be hooked into this process automatically during the pipeline run if either encryption or signature headers are found in the inbound Soap message.Once the pipeline process completes, our input buffer is then stuffed with the results of the pipeline run, ready to be processed by the rest of our application code.
Invoking the outbound message pipeline for responses
Once the request message has been matched and a response message generated, we need to decide whether to secure the latter. A good indication of whether WSE was involved in the formulation of the request message is given by the synthesized
env.Contextproperty, which is an instance of theSoapContextclass we discussed a few sections above:// Try and ascertain if the WSE was involved... if (vobjSoapContext != null) if (vobjSoapContext.Attachments.Count > 0 || vobjSoapContext.Security.Elements.Count > 0 || vobjSoapContext.Security.Tokens.Count > 0) strWSEIsInvolved = " {WSE}";Remember that we've decided to support 2-way secure messages. So the above logic is all that's needed to decide whether to secure the response message:
// Should we apply WSE on this message response before giving it // over to transportation? We only want to do this if we have a response // message to operate on AND the request was WSE'd if (strResp.Length > 0 && strWSEIsInvolved.Length // Yes....Process through WSE // Create an Envelope SoapEnvelope env = new SoapEnvelope(); // Get the basic Soap information into it env.LoadXml(strResp); // Augment request context with WSE goodies // This includes: // * Time-to-live of 60s // * User name and password credentials // * Encryption and Signing // Add the instance of EncryptedData to the SOAP response. WSCommon.SetupSOAPRequestContext(env.Context); // Create Pipeline - with default filter collection for input and output // (i.e. we are not overriding default behaviour in any way) Pipeline objWSEPipe = new Pipeline(); // Run the default output pipeline objWSEPipe.ProcessOutputMessage(env); // Extract the resultant Envelope strResp = env.OuterXml;What can be seen is that we are priming the
env.Contextproperty before running the output pipeline. We use a copy of the same routineSetupSOAPRequestContext()described in the section "Changes to the calling code" above, which the client side used.Updating the Back-end Service to use WSE for binary attachments
We can use the WSE to send and receive binary attachments that do not need to be encoded as
base64Binarywithin a Soap envelope. So, while we are in and around the reading of Soap input messages we shall look at for an indication of one or more DIME messages being passed to us.DIME is a specification that describes the ability to carry binary content outside a Soap envelope and to refer to it from within. This has pros and cons – the binary information will not be encoded and so would be smaller than the equivalent Web Services representation of an API parameter of, say, type
byte[]. However, being carried outside the Soap envelope gives rise to two major issues: it can't be expressed as a parameter or really described in WSDL and worse, there is no way to apply the same Soap processing model (including signing and encryption measures) to DIME content. We have to live with these restrictions for now but watch this space.[NOTE: Work is currently underway to consolidate the way DIME works with the desire to have a single Soap processing model. It is anticipated that binary attachments will be brought back within the Soap envelope where they could be subject to the same rules as other elements. We'd still expect them to retain their efficient non-Base64 encoded format through this change (somehow).]
Finally, it should be understood that the DIME processing is the one area that is implemented totally independently of the WSE filter processing (really because it has a processing model totally alien to the standard Soap model which is designed to operate on the contents of a Soap envelope), so we will need to work a little to prime the
SoapContext.Attachmentscollection property in the right way. This is achieved in theInputStreamToEnvelope()method, which we rather glossed over when presenting the section "Invoking the inbound message pipeline for requests":// NOTE – some code omitted to keep size down SoapEnvelope env = new SoapEnvelope(); // Create reader for DIME message DimeReader dr = new DimeReader(vstInputMessage); // Try to read record containing SOAP message. If this fails with a DIME // version error, the stream was most probably not a DIME one DimeRecord rec = dr.ReadRecord(); // Read Soap message from DIME record env.Load(rec.BodyStream); // Now add any Attachments we find in this stream to the Soap Context while (rec != null) // Read the next record rec = dr.ReadRecord(); // Did we? if (rec != null) // Here is our item - pull out the binary data representing it BinaryReader stream = new BinaryReader(rec.BodyStream); // And store in a new stream stNew.Write(bytAttachmentItem, 0, bytAttachmentItem.Length); // ... which we will attach to the Soap Context DimeAttachment objBin = new DimeAttachment( rec.Id, rec.Type, rec.TypeFormat, stNew); env.Context.Attachments.Add(objBin); catch(Exception) // If we get an exception here, assume this stream was not DIMEd // Now just load the whole stream into an array of bytes byte[] bufIn = new Byte[vstInputMessage.Length]; vstInputMessage.Read(bufIn, 0, (int)vstInputMessage.Length); // ...and from there into the Soap envelope env.LoadXml(new UTF8Encoding().GetString(bufIn)); return env;Once the above loading has been completed, the Back End code reacts to the contents of the
SoapContext.Attachmentscollection property as follows:// Look for the DIME attachment - there should only be one DimeAttachmentCollection colAttachments = vobjSoapContext.Attachments; // Any attachments found? if (colAttachments.Count > 0) // Get the first attachment only – hey, it's just a demo! // Here is our item - pull out the binary data representing it BinaryReader stream = new BinaryReader(colAttachments[0].Stream); byte[] bytAttachmentItem = new byte[stream.BaseStream.Length]; stream.Read(bytAttachmentItem, 0, bytAttachmentItem.Length); stream.BaseStream.Close(); stream.Close(); // Write the binary data to a file – we know it's a gif, but could // check the type as described in this DimeAttachment object. FileStream fs = new FileStream(vstrLocOfImages + "SendDIMEImage" + vstrScheme + strWSEIsInvolved + ".gif", FileMode.Create); fs.Write(bytAttachmentItem, 0 , bytAttachmentItem.Length); fs.Close();Summary of Client options – to WSE or not to WSE
This table attempts to clarify the requirements placed on the generated proxy and its callers for various scenarios using queue-based transports that we've described in detail above.
Scenario Requirement on generated proxy Requirement on caller of proxy No WSE Derive from MySoapClientProtocolImplement overridden GetWebRequest()andGetWebResponse()Optionally setup ResponseUrlproperty (default is Url + "_resp")WSE Derive from MyWSESoapClientProtocol Implement overridden GetWebRequest() only Optionally setup ResponseUrl property (default is Url + "_resp") Setup SoapContext property using SetupSOAPRequestContext() Setup configuration file to include SecurityProvider information Concurrency in the Queuing world
The sample application outlined below has been tested in a "multiple client – one server" mode. In this mode, clients place messages on a "single request" queue concurrently and wait for response on a different "response queue" concurrently. The idea is to tease out any multi-user issues around queue and message identification and access.
When first tested, the MQ transport performed faultlessly with up to 30 clients (at which point I lost a degree of interest (!)) but the MSMQ transport regularly threw exceptions of the form:
"Message that the cursor is currently pointing to has been removed from the queue"The reason for this appears to be that
ReceiveByCorrelationID()API is non transactional and hence in the time it takes a thread of activity to identify the message by the Correlation ID, the cursor marking the message may have become invalid. While not scientific in any way, observation has shown that with 20 processes each waiting for 100 correlated messages on the same MSMQ queue, the above exception is thrown on average 2.7 times per run (of 2000 message reads) – an error rate of 0.135%.If this happens, the receive must be retried again. Hence the loop in the code.
Framework Packaging
The classes we have been describing in the article that make up the framework are packaged into an assembly called
WSQTransports.dll:Figure 16: WSQTransports packaging
The leftmost types make up the support for extra facilities such as the
ResponseUrl. The other types make up our implementation of the queue-based pluggable protocols. The assembly is signed which makes it eligible for the GAC.The Sample Application
Due to the nature of the work being focused far more on a technology than a real world application of it, the demo is basic in that it shows the interaction of 2 console applications – Client and Server – exchanging messages using the framework we developed above.
The demo shows both passing "clear" and "WSE-secured" Soap messages based on various standard data type and DIME API combinations. While the demo is basic, it does illustrate the various concepts fairly well.
Figure 17: The demo running two clients and one back-end MQ / MSMQ service
The demo shows the Back End service processing incoming Soap messages on MSMQ and MQ queues and returning responses. The handling of non-trivial data types (real serialized custom objects) is also included in the demo. The screen shot actually shows the processing of complex types based on an arbitrary class called "Product". It shows arrays of these "Products" being serialized and passed between Server and Client. In the screenshot above, two clients can be seen running Soap calls over queues and via HTTP.
In order to assess the level of concurrency possible, start up one instance of the back end service, followed by as many instances of the test client as you would like to test. All instances of the client should complete their test runs without any exceptions being thrown. Since Correlation IDs are used to tie the request and response together, the client is guaranteed that it is given the correct response to it's request and there should be no issues with cursors moving underneath a message-get request.
Application introduction
The following diagram introduces the discrete portions of the sample application and depicts their fit within the client server model:
Figure 18: Article deliverables
There are two ASP.NET Web Services (one WSE-enabled) that support a basic API consisting of:
HelloWorld variants returning strings and arrays of strings Binary data operations – sending and receiving of images Complex data type serialization – a ProductService serving up Product instances [for the WSE version of the service] Sending of images using the DIME support within the WSE The IIS Virtual Directories for these Services are known as
WSAltRouteandWSAltRouteWSE.The dummy back-end (
WSAltRouteBEFake.exe) also lives in the Server space – it operates functionally a lot like the aforementioned ASP.NET Web Services but reacts to messages received on queues rather than via HTTP.The framework code that implements the pluggable queued protocols lives within a single Assembly (
WSQTransports.dll) so that it can be used within multiple projects.The framework is driven off a sample console application (
WSAltRouteTest.exe), which tests various flavors of API over various transports.The support for asynchronous operation (not shown) is provided by an assembly called
AsyncHelper.dll.The MSMQ and MQ Queues are configured using a tool called
WSQSetup.exe(not shown).Application Setup
The zip file contains several projects described in the section below. The key actions for deployment are:
Ensure the prerequisites you require (especially MQ and WSE) are installed Unzip the file preserving folder structure Locate the \BinfolderRun the WSQSetup.exeexecutable to create the relevant queues. Follow the on-screen prompts.Locate the \ClientsubfolderEdit the WSAltRouteTest.exe.configfile, review and make any changes. The list of entries is:
Entry Description NoTraceIf true, does not dump the stack trace when an exception occurs EnableHTTPCallsIf true, make the API calls over HTTP transport EnableMSMQCallsIf true, make the API calls over MSMQ transport EnableMQCallsIf true, make the API calls over MQ transport HTTPUriNonWSEEndpoint of the WSAltRouteWeb ServiceHTTPUriWSEEndpoint of the WSAltRouteWSEWeb ServiceMSMQRequestQEndpoint of the MSMQ request queue MSMQResponseQEndpoint of the MSMQ response queue MQRequestQEndpoint of the MQ request queue MQResponseQEndpoint of the MQ response queue LocationOfImagesPath for reading and writing images For example, if you do not want to run over HTTP, set
EnableHTTPCallsvalue to false.Re-locate the \BinfolderLocate the \ServersubfolderEdit the WSAltRouteBEFake.exe.configfile, review and make any changes. The list of entries is:
Entry Description NoTraceIf true, does not dump the stack trace when an exception occurs QueueToMonitor1Name of the 1st Queue to monitor for Soap messages QueueToMonitor2Name of the 2nd Queue to monitor for Soap messages QueueToMonitorXName of the Xth Queue to monitor for Soap messages PollDelayTimeReceive delay in ms. LocationOfImagesPath for reading and writing images For example, if you want to add a queue to monitor, add
QueueToMonitor3and set the associated value to the Uri name of the queue.If wanting to explore the HTTP transport: locate the \WebServicesfoldercopy the two subfolders ( WSAltRouteandWSAltRouteWSE) to your main web site folder (typicallyc:\inetpub\wwwroot).For the first folder, create a new IIS Virtual Directory called WSAltRouteoff the default web site (port 80), pointing to theWSAltRoutetarget folder.For the second folder, create a new IIS Virtual Directory called WSAltRouteWSEoff the default web site (port 80), pointing to theWSAltRouteWSEtarget folder.If wanting to explore the MSMQ transport: Check that the MSMQ request and response queues defined in the WSAltRouteTest.exe.configof the client application exist. Use the Computer Management feature of Windows 2000 or Windows XP to check this.If wanting to explore the MQ transport: Check that the MQ request and response queues defined in the WSAltRouteTest.exe.configof the client application exist. Use the Websphere MQ Explorer tool to check this.Re-locate the \BinfolderLocate the \ImagessubfolderCopy the two .giffiles,CD.gifandHelp.gifto the folder specified in both theWSAltRouteTest.exe.configof the client andWSAltRouteBEFake.exe.configof the server under theLocationOfImageskey name.
To run the demo, start up the back end process WSAltRouteBEFake.exefirst, observing the startup message appears ok. Then start up the client processWSAltRouteTest.exe. Activity should then be displayed in both consoles over the next few moments, as various Web Service API are exercised across the chosen transports. No exceptions should be seen.If exceptions are seen, only the message portions of the exception may be shown. If this appears to be the case, stack trace can be turned on by altering the value for NoTracein the relevant (typically client) application configuration file to false.Application Build
The Solution is made up of the following projects:
No sorry this is all ancient history now. The likelyhood of getting round to converting it is extremely low. Out of interest, what kind of failures in the conversion are you seeing? Is it just obsoleted API?
I am not aware of other libraries or whether the .NET Fr 3 (specifically the old Indigo stuff) would help you more.
Simon.
i need to control access to my web service to prevent any one from discover it.only who i garent the rights
Thanks
I'm wondering if it is possible to implement 'one request / multiple responses' scenario..
After some researching I thought that I could do it with async invocation, but ended up with a fact that EndXXX method can be invoked only once, otherwise a user could invoke as many times as his async callback would be called. Unfortunately, the flag and the class (respectively its constrcutor) where it is defined (WebClientAsyncResult) indicating that EndXXX was called is internal so I cannot subclass it..
Is there any other approach I could follow?
Thank you.
Ales Pour
i developing a project in .net technologies in this i use an webservices ,
in this project i want to use tally as an interfaces to my project some accounting process are done , so i send crrospoding data to tally when i call my web services it interact with tally ,then tally process the data and send result back to the webservices.This is my problem so any one know plz help me............
thank u for ur help
I'm guessing that the IIS server side is so wrapped up with an expectation that it's dealing with HTTP traffic over that there's no scope to "change the transport". By the time you get control (which you can do by adding HttpServer modules) it's kinda a done deal about the fact you got invoked by the IIS-ASP.NET pipeline.
I did note, however the following article:
http://www.codeproject.com/useritems/UsingAspRuntime.asp
Again think this is "too late" in the process as IIS is handing off to the ASP.NET runtime - but it's interesting non the less.
Rgds,
Simon.
thanks for your reply. I agree with your conclusion.
I wonder now... isn't it somehow possible to access IIS via COM? Or, in case of services written in .NET, aren't they accesible via COM?
Thanks.
Look here for how:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndcom/html/cis.asp
Bit nasty so I'm told.
W.r.t .NET services - depends what you mean by services. At the component level, a .NET type can be exposed as a COM object.
the TCP/IP version is now posted at:
http://www.codeproject.com/cs/webservices/WSAltRoute2.asp
Please feeback any comments as you see fit.
Regards, Simon.
Have you posted the new improved code anywhere, or are you willing to share?
I was about to start the same project it would sure be nice to use your stuff rather than starting from scratch.
thanks!
Web03 2.8:2023-05-13:1
Name of project Purpose WSAltRouteTestThe Client program, which exercises the new transports. WSAltRouteBEFakeThe dummy Server program, which responds to requests. WSQTransportsThe code supporting the pluggable transport framework and the queuing transport implementations. AsyncHelperAsynchronous support for Business Objects from MSDN article [3] WSAltRouteASP.NET Web Service configured without WSE WSAltRouteWSEASP.NET Web Service configured for WSE operation WSQSetupSetup program for the queues – run this after installing MSMQ (and optionally MQ) but before running the demo. WSSetupMQSupport for creating MQ Queues – requires VB6 runtime Loading the Solution file should be pretty much instantaneous. You may need to re-point the
.webinfofiles for the two HTTP-oriented Web Services (which assume Port 80 by default) but apart from that the projects should compile directly to the\Bin\Clientand\Bin\Serversub-folders (assuming the files within are not locked).Case Study Review
Here's a recap of the major aspects of the solution:
Basic support
At first glance, a generated Soap proxy looks tightly bound to the HTTP transport. This is not actually the case. ASP.NET Web Services includes an infrastructure specifically designed to allow support of alternative transport protocols based on System.Net.WebRequestandSystem.Net.WebResponse.In order to support new protocols, a scheme registration mechanism is provided as part of the infrastructure. Once new "network schemes" are registered, a client may make use of them when specifying an endpoint. Both MSMQ and Websphere MQ are easily capable of being wrapped with a transport protocol, and it is straightforward for the client to specify an end point based on a scheme. This has the effect of minimizing the code changes required for a client in order to make use of a new transport. Secure support
The WSE can easily be integrated seamlessly into ASP.NET clients, Servers and in a more raw form in arbitrary back end .NET services. In the case of the former, the SoapWebRequestdoes the majority of the work such as driving serialization of Soap messages, and then handing off to a specific transport for delivery.There are very few changes required to the code (both framework and client) to move from "clear text" mode to WSE-secured mode. Binary support
With the WSE in place, supporting binary attachments through the SoapWebRequestis a simple process.Further Work
Constructing this section is like shooting fish in a barrel! A selection of things to consider include:
There are places in the code where strings are concatenated rather than built using StringBuilder. Because I use this 'technique', does not mean I endorse it! A lot more work should be done to secure the various aspects of the solution. Two immediate ones springing to mind: The use of StrongNameIdentityPermissionAttributeto explicitly control who calls sensitive assemblies (only callers signed with a particular strong name key)The encryption keys should be stored outside the code, either in a (well secured) file or in the secure area of the Registry The demo has been built around a very RPC-style of message protocol, where responses to requests are expected. Investigation of 1-way Soap messages would be interesting - this most closely maps to a Queue style send operation without the need for a corresponding receive and would be a useful addition to the library. This framework has been used to interoperate with a Java back end using the Axis Toolkit 1.1 RC1 under J2SE 1.4.1. Unfortunately, throwing Java into the mix here would probably have resulted in double the copy! Depending on reaction to this article, a follow-up concentrating more on this interoperability aspect is possible. It would be interesting to add a compression capability like the SmartLib compression library [9] into the DIME section for large attachments. Other transports could be implemented within the framework – SMTP, FTP, SQL2000 etc. are good candidates. As an example, I've implemented TCP/IP recently within this framework which proved to be a trivial process. Related Links
[1] As indicated earlier, the
MQWebRequestclass uses the IBM MQSeries Automation Classes for ActiveX COM type library as a reference. There are more performant alternatives to this approach such as the managed ".NET Provider for MQSeries" from Neil Kolban at http://www.kolban.com/mq/DotNET/index.htm . This may be incorporated into a future version of Websphere MQ.[2] Websphere MQ 5.3 is available in trial form from IBM at: http://www-3.ibm.com/software/ts/mqseries/messaging/
[3] The Asynchronous support is provided by a slightly adapted version of the code supplied within the article "Creating Asynchronous Business Objects for Use in .NET Windows Forms Clients" at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwinforms/html/asyncui.asp
[4] Programming with Web Services Enhancements 1.0 for Microsoft .NET at: http://msdn.microsoft.com/webservices/building/wse/default.aspx?pull=/library/en- us/dnwebsrv/html/progwse.asp
[5] Inside WSE pipeline describes the pipeline model for WSE filters at: http://msdn.microsoft.com/webservices/building/wse/default.aspx?pull=/library/en- us/dnwebsrv/html/insidewsepipe.asp
[6] Programming with Web Services Enhancements 1.0 for Microsoft .NET at: http://msdn.microsoft.com/webservices/building/wse/default.aspx?pull=/library/en- us/dnwebsrv/html/progwse.asp
[7] Using Web Services Enhancements (WSE) for Username / Password Authentication at: http://www.eggheadcafe.com/articles/20021227.asp
[8] The WSE Settings Tool can be found at: http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/msdn- files/027/002/108/msdncompositedoc.xml
[9] The SmartLib compression library is a fully managed .NET compression library at: http://www.icsharpcode.net/OpenSource/SharpZipLib/
[10] The VB6 SP5 runtime is required if you are using WSQSetup.exe to set up the MQ queues for our demo application – this can be found at: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=BF9A24F9- B5C5-48F4-8EDD-CDF2D29A79D5.
History
Apr 8th, 2003 - Updated for WSE support. Jan 18th, 2003 - Initial release I need to send a secure soap message using MQ as in the transport layer and WSE 3.0 to add the security to the soap message.
I was wondering if there are any out of box libraries that would help with this or if there was an update to this project. Studio 2005 could not upgrade this project.
Thanks
clarkSign In· View Thread ![]()
Sign In· View Thread ![]()
Simon Gregory wrote:No sorry this is all ancient history now. The likelyhood of getting round to converting it is extremely low. Out of interest, what kind of failures in the conversion are you seeing? Is it just obsoleted API?
I think it may be the later. I am trying to port the WSSecurityCertificate project that comes as a sample with the WSE 3.0 library. I beleive I followed the model in your code sample exactly but I get the "not supported" error below.
System.NotSupportedException was unhandled
Message="WSE808: The following transport scheme is not supported: mq."
Source="Microsoft.Web.Services3"
StackTrace:
at Microsoft.Web.Services3.WebServicesClientAsyncResult..ctor(WebServicesClientProtocol client, SoapEnvelope envelope, AsyncCallback callback, Object state)
at Microsoft.Web.Services3.Xml.SoapEnvelopeWriter.Finish()
at Microsoft.Web.Services3.Xml.XmlWrappingWriter.Flush()
at System.Web.Services.Protocols.SoapHttpClientProtocol.Serialize(SoapClientMessage message)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at localhost.WSSecurityCertificateServiceWse.StockQuoteRequest(String[] symbols) in C:\Program Files\Microsoft WSE\v3.0\Samples\CS\QuickStart\Security\WSSecurityCertificate\Policy\WSSecurityCertificatePolicyClient\Web References\localhost\Reference.cs:line 101
at WSSecurityCertificatePolicyClient.WSSecurityCertificateClient.Main(String[] args) in C:\Program Files\Microsoft WSE\v3.0\Samples\CS\QuickStart\Security\WSSecurityCertificate\Policy\WSSecurityCertificatePolicyClient\WSSecurityCertificateClient.cs:line 69
at System.AppDomain.nExecuteAssembly(Assembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()Sign In· View Thread ![]()
Hello Simon,
your article is amazing , could you please tell me how to run the application ..as i'm new to .NET and i'm currently ina learning phase.
Thanks
Sign In· View Thread ![]()
Sign In· View Thread ![]()
Using direct access, with pathname like that:
Set obj = GetObject("queue:FormatName=DIRECT=OS:ServerName\PRIVATE$\articleqcom/new:qc.sm")
, the methods of QC component are called correct.
But what about a call using HTTP transport, a new enhancements of MSMQ 3.0 ?(windows xp and 2003)
Just changing the FormatPath with something like this :
Set obj = GetObject("queue:FormatName=DIRECT=HTTP://ServerName/msmq\PRIVATE$\articleqcom/new:QC.sm")
will give an error:
"Automation error.Invalid Syntax."
What is the problem ?
(Note, sending a simple message qeue using the HTTP is working very good)
Adrian BacaianuSign In· View Thread ![]()
Hello again.
I'm having a trouble with getting SOAPAction inside client transport, because it appears that when using custom transport, this information is not passed down to WebRequest subclass. I've found that I can create a SoapExtension, where SOAPAction is is available, but I do not know how to pass this info down to my custom WebRequest?
Thank you.
Best regards,
Ales PourSign In· View Thread ![]()
Sign In· View Thread ![]()
Amazing... I can reset the flag using reflection despite it is private (I expected some permission exception), and let the client application call EndXXX from async callback multiple times! Sign In· View Thread ![]()
Could you able to get multiple responses for single async request? I got the same situation, I appreciate if you provide me sample code if you achieved this scenario.
Thanks
Srinivas
SrinivasSign In· View Thread ![]()
The short answer is that I got it working by overriding async methods of WebRequest in my subclass plus implementing some stuff, such as simple implementation of IAsyncResult.
When an application code calls EndXXX method on a proxy, EndGetResponse method of WebRequest is called in the end. The param passed to it is instance of WebClientAsyncResult, whose "EndSendCalled" (boolean type) field has to be set to false to avoid "end method already called" exception (or something like it).
I also found that it is better to set its "Buffer" field (byte[] type, iirc) to null so that a new response buffer is allocated for each response, otherwise a garbage may stay in the buffer.
These field are internal, so reflection has to be used.
Hope this helps at least a bit,
Sign In· View Thread ![]()
Sign In· View Thread ![]()
New URL for the settings tool:
http://www.microsoft.com/downloads/details.aspx?FamilyId=E1924D29-E82D-4D9A-A945-3F074CE63C8B&displaylang=en
Also, fantastic job on the article!Sign In· View Thread ![]()
I wonder if there is a way to plug-in custom transport also on the server-side, into IIS? I guess I could write eg. a queue listener which would redirect message to http://localhost:80/..., but I wonder if IIS supports pluggable transport, somehow...
Thank you!
Regards,
Ales pourSign In· View Thread ![]()
Sign In· View Thread ![]()
Sign In· View Thread ![]()
Sign In· View Thread ![]()
I've looked at the project you mentioned again, browsed MSDN doc, and it seems that HttpRuntime is something I could use to achieve what I wanted - to bypass IIS's HTTP transport and yet invoke the service. I've found clear explaination in http://msdn.microsoft.com/msdnmag/issues/02/09/HTTPPipelines/ of what happens behind the scene.
It is in fact complete IIS bypassing because service/(virtual)directory is accessed directly via filesystem, but if it works, and it does as I tried the example you have pointed me to, I buy it. I understand that I'll miss funcionality like security, pooling etc provided by IIS for web service endpoints, but I do not need it now.
Thanks for your help!
Best regards,
Ales PourSign In· View Thread ![]()
... I was mistaken... even the security including WS-Security should work. Greate stuff is that HttpRuntime. Sign In· View Thread ![]()
Sign In· View Thread ![]()
IBM now has .NET classes for MQ at: http://www-3.ibm.com/software/integration/support/supportpacs/individual/ma7p.html (based on the work by Kolban, I believe.) It is listed as "category 2" or "as-is".
scolestockSign In· View Thread ![]()
I now have the above working without the need for a SoapExtension (which means less markup after a proxy is generated), supporting asynchronous calling, and more importantly with the option of using 2-way secure comms using WSE 1.0. Currently deciding whether to update this article here or post the new improved one on csharptoday.com.
Simon.Sign In· View Thread ![]()
Sign In· View Thread ![]()
|
|
买醉的草稿本 · curl查看请求头-掘金 2 年前 |