A new hoNoReadMultipartMIME flag has been added to the TIdHTTP.HTTPOptions property. The purpose of this flag is to specify whether TIdHTTP should read the body content of “multipart/…” responses, such as “multipart/x-mixed-replace” or “multipart/byteranges”, into the target TStream or to exit immediately and let the caller read the content manually instead. By default, this flag is disabled to preserve existing behavior to have TIdHTTP read the content into the target TStream until the server closes the connection.
The primary motivation for this is to handle “multipart/x-mixed-replace” server pushes, such as from webcams. TIdHTTP does not natively support server pushes (and still does not), so the previous workaround to process the push events in real-time was to use TIdEventStream as the target TStream with a OnWrite event handler assigned, or to implement a custom TStream class that overrides the virtual Write() method. Neither approach is very intuitive.
Now, if you enable the hoNoReadMultipartMIME flag, and then any “multipart/…” response is received (see UPDATE further below), you can read the MIME data directly from the TIdHTTP.IOHandler as needed, such as by using TIdTCPStream to assign TIdHTTP as the SourceStream for TIdMessageDecoderMIME and then call the decoder’s ReadHeader() and ReadBody() methods in a loop until the server stops sending pushes by sending a terminating MIME boundary, or closes the connection. For example:
var Boundary, Line: string; TCPStream: TIdTCPStream; Decoder: TIdMessageDecoder; MsgEnd: Boolean; BodyStream: TStream; begin ... HTTP.HTTPOptions := HTTP.HTTPOptions + [hoNoReadMultipartMIME]; HTTP.Get(...); if IsHeaderMediaType(HTTP.Response.ContentType, 'multipart') then begin Boundary := ExtractHeaderSubItem(HTTP.Response.ContentType, 'boundary', QuoteHTTP); repeat Line := HTTP.IOHandler.ReadLn; until (Line = ('--' + Boundary)) or (Line = ('--' + Boundary + '--')); TCPStream := TIdTCPStream.Create(HTTP); try Decoder := TIdMessageDecoderMIME.Create(nil); try MsgEnd := False; repeat TIdMessageDecoderMIME(Decoder).SourceStream := TCPStream; TIdMessageDecoderMIME(Decoder).FreeSourceStream := False; TIdMessageDecoderMIME(Decoder).MIMEBoundary := Boundary; Decoder.ReadHeader; case Decoder.PartType of mcptText: begin BodyStream := TMemoryStream.Create; try NewDecoder := Decoder.ReadBody(BodyStream, MsgEnd); try finally Decoder.Free; Decoder := NewDecoder; end; finally BodyStream.Free; end; end; mcptAttachment: begin BodyStream := TMemoryStream.Create; try NewDecoder := Decoder.ReadBody(BodyStream, MsgEnd); try finally Decoder.Free; Decoder := NewDecoder; end; finally BodyStream.Free; end; end; mcptIgnore: begin FreeAndNil(Decoder); Decoder := TIdMessageDecoderMIME.Create(nil); end; mcptEOF: begin FreeAndNil(Decoder); MsgEnd := True; end; end; until (Decoder = nil) or MsgEnd; finally Decoder.Free; end; finally TCPStream.Free; end; end; ... end;
An added benefit of this solution is that you can use the TIdHTTP.OnHeadersAvailable event to set the hoNoReadMultipartMIME flag on a per-response basis after examining the HTTP response headers before the response body content is read. That way, you can dynamically decide whether to enable the flag on certain “multipart/…” types, like “multipart/x-mixed-replace”, or to disable it on other “multipart/…” types, like “multipart/byteranges”, depending on how and when you want to parse the MIME data. – in real-time while the response is still being sent, or from a local TStream after the response has been completed. For example:
procedure TMyForm.HTTPHeadersAvailable(Sender: TObject; AHeaders: TIdHeaderList; var VContinue: Boolean); begin if IsHeaderMediaType(AHeaders.Values['Content-Type'], 'multipart/x-mixed-replace') then HTTP.HTTPOptions := HTTP.HTTPOptions + [hoNoReadMultipartMIME] else HTTP.HTTPOptions := HTTP.HTTPOptions - [hoNoReadMultipartMIME]; VContinue := True; end; ... var Resp: TStream; begin ... Resp := TMemoryStream.Create; try HTTP.Get(..., Resp); if IsHeaderMediaType(HTTP.Response.ContentType, 'multipart/x-mixed-replace') then begin end else begin end; finally Resp.Free; end; ... end;
UPDATE March 24, 2021: in earlier versions of TIdHTTP, the TCP connection would be closed when TIdHTTP is done processing a response if an HTTP keep-alive is not being used. When the hoNoReadMultipartMIME flag is enabled and an HTTP keep-alive is not used, this would cause a premature closure of the connection before user code can read the MIME data from the IOHandler. As of GitHub checkin 0F1D4047, this has now been corrected, so that the connection is no longer closed if the hoNoReadMultipartMIME flag is enabled and the response is a “multipart/…” type, regardless of whether or not an HTTP keep-alive is used.