Python HTTP Server/Client: Remote end closed connection without response error

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
I made simple HTTP Server using BaseHTTPRequestHandler . The problem is, that when I want to post some data using requests from client, I get ConnectionError . I did simple request from requests lib documentation. Also interesting thing is, that HTTP Server will receive data from client and print it to console. I don't understand how its possible.
Client:
def post_data():
    """Client method"""
    json_data = {
        'sender': 'User',
        'receiver': 'MY_SERVER',
        'message': 'Hello server! Sending some data.'}
    data_headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
    data_payload = json.dumps(json_data)
        post = requests.post('http://localhost:8080/post', data=data_payload,
                             headers=data_headers)
        print(post.status_code)
    except ConnectionError as e:
        print("CONNECTION ERROR: ")
        print(e)
HTTP Server:
def do_POST(self):
    """Server method"""
    self.send_response(200)
    print("Receiving new data ...")
    content_length = int(self.headers['Content-Length'])
    post_data = self.rfile.read(content_length)
    print(post_data)
Server result:
C:\Users\mypc\Projects\PythonFiles\httpserver>python server.py
Fri Jan  5 01:09:12 2018: HTTP Server started on port 8080.
127.0.0.1 - - [05/Jan/2018 01:09:21] "POST /post HTTP/1.1" 200 -
Receiving new data ...
b'{"sender": "User", "receiver": "MY_SERVER", "message": "Hello server! Sending some data."}'
Client result:
C:\Users\mypc\Projects\PythonFiles\httpserver>python client.py
CONNECTION ERROR:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
C:\Users\mypc\Projects\PythonFiles\httpserver>
Error without exception block:
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Python36\lib\http\client.py", line 1331, in getresponse
    response.begin()
  File "C:\Python36\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "C:\Python36\lib\http\client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\requests\adapters.py", line 440, in send
    timeout=timeout
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Python36\lib\site-packages\urllib3\util\retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Python36\lib\site-packages\urllib3\packages\six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Python36\lib\http\client.py", line 1331, in getresponse
    response.begin()
  File "C:\Python36\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "C:\Python36\lib\http\client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "client.py", line 137, in <module>
    start_parser()
  File "client.py", line 101, in start_parser
    send_requests(args.get, args.post)
  File "client.py", line 51, in send_requests
    post_data()
  File "client.py", line 129, in post_data
    headers=data_headers)
  File "C:\Python36\lib\site-packages\requests\api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Python36\lib\site-packages\requests\api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python36\lib\site-packages\requests\sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python36\lib\site-packages\requests\sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python36\lib\site-packages\requests\adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
It looks like the server is terminating the connection early without sending a full response. I've skimmed the docs and I think this is the problem (emphasis added):
send_response(code, message=None) 
  Adds a response header to the
  headers buffer and logs the accepted request. The HTTP response line
  is written to the internal buffer, followed by Server and Date
  headers. The values for these two headers are picked up from the
  version_string() and date_time_string() methods, respectively. If the
  server does not intend to send any other headers using the
  send_header() method, then send_response() should be followed by an
  end_headers() call.
  Changed in version 3.3: Headers are stored to an internal buffer and
  end_headers() needs to be called explicitly.
So you probably just need to add the call to end_headers. If you were reading an old example (prior to Python 3.3) this wouldn't have been needed.
                @isquared-KeepitReal You might be using an older version of the library. I suggest posting a new question if you're having problems so you can include your code and any errors you're seeing
– Peter Gibson
                Sep 4, 2018 at 22:59
                I  retried a couple of times, and it suddenly worked. would make sense if it's all about  too early terminated connections.
– lode
                Apr 9, 2020 at 22:59
                @AdamGross how did you fix it? will appreciate if you could answer on this question stackoverflow.com/questions/66778586/…
– young_minds1
                Mar 24, 2021 at 11:54
You can just put end_headers as Peter Gibson said
self.send_header('Content-Type', 'blabla' )
self.end_headers()
        Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.