Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
IXMLHttpRequest.responseXml is empty, with no parse error, when responseText contains valid Xml

Ask Question
i am fetching some XML from a government web-site :
http://www.bankofcanada.ca/stats/assets/rates_rss/noon/en_all.xml
i am using the following, fairly simple code:
   szUrl: string;
   http: IXMLHTTPRequest;
begin
   szUrl := 'http://www.bankofcanada.ca/stats/assets/rates_rss/noon/en_all.xml';
   http := CoXMLHTTP60.Create;
   http.open('GET', szUrl, False, '', '');
   http.send(EmptyParam);
   Assert(http.Status = 200);
   Memo1.Lines.Add('HTTP/1.1 '+IntToStr(http.status)+' '+http.statusText);
   Memo1.Lines.Add(http.getAllResponseHeaders);
   Memo1.Lines.Add(http.responseText);
i won't show all the body that returns, but it does return valid xml in the responseText:
HTTP/1.1 200 OK
Cache-Control: max-age=5
Connection: keep-alive
Connection: Transfer-Encoding
Date: Fri, 30 Mar 2012 14:50:50 GMT
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Expires: Fri, 30 Mar 2012 14:50:55 GMT
Server: Apache/2.2.16 (Unix) PHP/5.3.3 mod_ssl/2.2.16 OpenSSL/1.0.0d mod_perl/2.0.4 Perl/v5.12.0
X-Powered-By: PHP/5.3.3
<?xml version="1.0" encoding="ISO-8859-1"?>
<rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns="http://purl.org/rss/1.0/"
    xmlns:cb="http://www.cbwiki.net/wiki/index.php/Specification_1.1"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:xsi="http://www.w3c.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3c.org/1999/02/22-rdf-syntax-ns#rdf.xsd">
    <channel rdf:about="http://www.bankofcanada.ca/stats/assets/rates_rss/noon/en_ALL.xml">
        <title xml:lang="en">Bank of Canada: Noon Foreign Exchange Rates</title>
        <link>http://www.bankofcanada.ca/rates/exchange/noon-rates-5-day/</link>
Okay, fine, there's valid xml in there. i know it's valid because...well just look at it. But i also know it's valid by parsing it:
   szXml: WideString;
   doc: DOMDocument60;
begin
   szXml := http.responseText;
   doc.loadXML(szXml);
   Assert(doc.parseError.errorCode = 0);
   Memo1.Lines.Add('============parsed xml');
   Memo1.Lines.Add(doc.xml);
The origianal IXmlHttpRequest contains a responseXml property. From MSDN:
Represents the parsed response entity body.
If the response entity body is not valid XML, this property returns DOMDocument that was parsed so that you can access the error. This property does not return IXMLDOMParseError itself, but it is accessible from DOMDocument.
In my case the responseXml property exists, as it should:
Assert(http.responseXml <> nil);
And there is no parse error of responseText:
doc := http.responseXml as DOMDocument60;
Assert(doc.parseError.errorCode = 0);
as there should be, since the xml is valid.
Except that when i look at the http.responseXml document object, it's empty:
   Memo1.Lines.Add('============responseXml');
   Memo1.Lines.Add(doc.xml);
Is is IXMLHttpRequest (and IXMLServerHttpRequest) returning an empty XML document, when:
there is xml
the xml is valid
there is no parse error
In long form:
    msxml2_tlb;
procedure TForm1.Button1Click(Sender: TObject);
    szUrl: string;
    http: IXMLHTTPRequest;
    doc: DOMDocument60;
begin
    szUrl := 'http://www.bankofcanada.ca/stats/assets/rates_rss/noon/en_all.xml';
    http := CoXMLHTTP60.Create; //or CoServerXmlHttpRequest.Create
    http.open('GET', szUrl, False, '', '');
    http.send(EmptyParam);
    Assert(http.Status = 200);
    doc := http.responseXml as DOMDocument60;
    Assert(doc.parseError.errorCode = 0);
    ShowMessage('"'+doc.xml+'"');
How do i make XmlHttpRequest (and more importantly ServerXMLHTTP60) behave as documented?
                Delphi version information is key in all questions involving the RTL and standard libraries. What version?
– Warren  P
                Mar 30, 2012 at 15:24
                Related: stackoverflow.com/questions/8925798/…   - note the Keepalive comment. Tried that? Also, what IE version and MS XML version, since those matter in these cases, too. I believe MS XML's HTTP methods use WinInet, which has some fun bugs, and it gets updated when you update IE.
– Warren  P
                Mar 30, 2012 at 15:41
                @WarrenP i tried the timeout; it doesn't change the result (nor should it, since i'm getting a valid response). ie9, msxml 6.0. Do you get the same behavior if you copy-paste the final simplified 8-line version?
– Ian Boyd
                Mar 30, 2012 at 15:55
                It's something to do with your use of DOMDocument60 instead of http.responseText, I think.
– Warren  P
                Mar 30, 2012 at 16:06
After 3 hours of fiddling, i managed to track down the problem in the original http response headers:
HTTP/1.1 200 OK
Cache-Control: max-age=5
Connection: keep-alive
Connection: Transfer-Encoding
Date: Fri, 30 Mar 2012 14:50:50 GMT
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Expires: Fri, 30 Mar 2012 14:50:55 GMT
Server: Apache/2.2.16 (Unix) PHP/5.3.3 mod_ssl/2.2.16 OpenSSL/1.0.0d mod_perl/2.0.4 Perl/v5.12.0
X-Powered-By: PHP/5.3.3
should be:
HTTP/1.1 200 OK
Cache-Control: max-age=5
Connection: keep-alive
Connection: Transfer-Encoding
Date: Fri, 30 Mar 2012 14:50:50 GMT
Transfer-Encoding: chunked
Content-Type: text/xml; charset=UTF-8
Expires: Fri, 30 Mar 2012 14:50:55 GMT
Server: Apache/2.2.16 (Unix) PHP/5.3.3 mod_ssl/2.2.16 OpenSSL/1.0.0d mod_perl/2.0.4 Perl/v5.12.0
X-Powered-By: PHP/5.3.3
Once i found the problem, i was able to back-find the documentation that explain the behavior:
  The supported MIME types for MSXML 6.0 are: 
  "text/xml"
  "application/xml"
  or anything that ends with "+xml", for example "application/rss+xml"
The RSS feed i'm fetching is actually a Resource Definition Format (RDF) feed, where the content type is supposed to be:
application/rdf+xml
Their use of:
text/html
is wrong on so many levels.
So the behavior i'm experiencing is by design; although frustrating - as there's no easy way to know if the responseXml is "valid".
the responseXml object will be assigned
the parseError object will be assigned
the parseError.ErrorCode is zero
the responseXml.documentElement will be nil
                My guess would be that since text/html is not supported, the underlying XML parser is not even parsing any data, which would leave the errorCode set to zero but not create an documentElement object.  If the responseText is not empty but the documentElement is, you know something went wrong, and you can test for that.
– Remy Lebeau
                Mar 30, 2012 at 21:03
                Many of the Noon feeds on that site are sending the Content-Type as text/html, but some of them, like "en_USD.xml" and "en_MXN.xml", send application/xml instead.
– Remy Lebeau
                Mar 30, 2012 at 21:12
                @WarrenP i'm falling back to if http.responseXml.documentElement = nil then result := GetXmlObjectFromXmlString(http.responseText) else result := http.responseXml; It's not a thoroughly generic solution, as sometimes there can be an xml document with no documentElement. But in this case it's good enough, because if the xml really is empty then they've done something stupid.
– Ian Boyd
                Mar 31, 2012 at 15:09
I had the same problem with YouTube services.
The responseXml object is dependent on the content-type/MIME of the response.

You could examine the response Content-Type e.g: if http.getResponseHeader('Content-Type') contains text/xml or application/xml only then you can refer to http.responseXml, otherwise it will be empty (see MSDN Remarks). Also note that the responseXml parser validation features are always turned off, for security reasons.
But, the http.responseText will always have the xml text, no matter what content type is in the response, so you can always use a new instance of DOMDocument to load the xml e.g:
http := CoXMLHTTP60.Create; // or CoServerXmlHttpRequest.Create 
http.open('GET', szUrl, False, '', '');
http.send(EmptyParam);
Assert(http.Status = 200);
doc := CreateOleObject('Msxml2.DOMDocument.6.0') as DOMDocument60; 
doc.async := False;
doc.loadXML(http.responseText); // <- load XmlHttpRequest.responseText into DOMDocument60 and use it
Assert(doc.parseError.errorCode = 0);
// do useful things with doc object...
                i don't use the DOMDocument object (and strictly speaking i don't use the XmlHttpRequest object) because the enterprise of mutton-heads thinks it's a good idea to block people's access to the internet (i.e. i have to configure a proxy - which can only be done using IServerXmlHttpRequest). Also, you can't just check for text/xml, need to also check for application/xml, or anything/anything+xml (frought with edge cases i don't even want to care about).
– Ian Boyd
                Mar 31, 2012 at 21:53
                Read my answer carefully. I'm saying that you will have a valid responseXml.documentElement object only if the response is explicitly set to text/xml. I'm saying DON'T rely on responseXml and always use responseText and DOMDocument like in the first code example. IServerXmlHttpRequest behaves the same XmlHttpRequest when the response content-type is not text/xml.
– kobik
                Mar 31, 2012 at 22:26
                i'd prefer to let the XmlHttpRequest handle the parsing of the response xml text; otherwise the xml has to go through another encode cycle (encoding as UTF-16 BSTR, then parsed again). It's possible that the documentElement is nil even when the content type is text/xml - so that's a gotcha.
– Ian Boyd
                Apr 1, 2012 at 1:46
Well, this works in Delphi XE and Delphi 7:
procedure TForm1.Button1Click(Sender: TObject);
    szUrl: string;
    http: IXMLHTTPRequest;
    doc: {$ifndef UNICODE}WideString{$else}string{$endif};
begin
    szUrl := 'http://www.bankofcanada.ca/stats/assets/rates_rss/noon/en_all.xml';
    http := CoXMLHTTP60.Create; //or CoServerXmlHttpRequest.Create
    http.open('GET', szUrl, False, '', '');
    http.setRequestHeader('Content-Type', 'text/xml;charset=UTF-8');
    http.send(EmptyParam);
    Assert(http.Status = 200);
    doc := UTF8Encode(http.responseText);
    Memo1.Lines.text := doc;
//  ShowMessage('"'+doc.xml+'"');
Hope it works for you in Delphi 5, too. Of course, any unicode characters are going to turn into ? on you, in non-unicode delphi versions.
You are retreiving the xml from the DOMDocument object itself, but you should be grabbing it from the first node in the document's tree instead, eg:
doc := http.responseXml as DOMDocument60; 
Assert(doc.parseError.errorCode = 0); 
ShowMessage('"' + doc.DocumentElement.childNodes.Item(0).xml + '"'); 
Microsoft's own examples in the documentation for DOMDocument and the xml property show exactly that kind of logic.
                At the very least it is deceiving: there is nothing in the DOMDocument yet it's valid XML.
– Ian Boyd
                Mar 31, 2012 at 15:17
        Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.