Python处理socket.error。[Errno 104] 对方重置连接

Question 1

当使用Python 2.7和 urllib2 从一个API中检索数据时，我得到了错误 [Errno 104] Connection reset by peer 。是什么导致了这个错误，应该如何处理这个错误以使脚本不崩溃？

ticker.py

def urlopen(url):
    response = None
    request = urllib2.Request(url=url)
        response = urllib2.urlopen(request).read()
    except urllib2.HTTPError as err:
        print "HTTPError: {} ({})".format(url, err.code)
    except urllib2.URLError as err:
        print "URLError: {} ({})".format(url, err.reason)
    except httplib.BadStatusLine as err:
        print "BadStatusLine: {}".format(url)
    return response
def get_rate(from_currency="EUR", to_currency="USD"):
    url = "https://finance.yahoo.com/d/quotes.csv?f=sl1&s=%s%s=X" % (
        from_currency, to_currency)
    data = urlopen(url)
    if "%s%s" % (from_currency, to_currency) in data:
        return float(data.strip().split(",")[1])
    return None
counter = 0
while True:
    counter = counter + 1
    if counter==0 or counter%10:
        rateEurUsd = float(get_rate('EUR', 'USD'))
    # does more stuff here
Traceback (most recent call last):
  File "/var/www/testApp/python/ticker.py", line 71, in <module>
    rateEurUsd = float(get_rate('EUR', 'USD'))
  File "/var/www/testApp/python/ticker.py", line 29, in get_exchange_rate
    data = urlopen(url)
  File "/var/www/testApp/python/ticker.py", line 16, in urlopen
    response = urllib2.urlopen(request).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 438, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 625, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 438, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 625, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.7/socket.py", line 447, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 104] Connection reset by peer
error: Forever detected script exited with code: 1

Question 2


          
           
            "对等体重置连接 "是TCP/IP的等同于把电话摔回钩子上。这比仅仅是不回话，让人挂掉更有礼貌。但它不是真正有礼貌的TCP/IP交谈者所期望的FIN-ACK。(
            
             来自其他SO的回答
            
            )
           
           
            所以你对此无能为力，这是服务器的问题。
           
           
            但你可以使用
            
             try .. except
            
            块来处理这个异常。
           
           from socket import error as SocketError
import errno
    response = urllib2.urlopen(request).read()
except SocketError as e:
    if e.errno != errno.ECONNRESET:
        raise # Not error we are looking for
    pass # Handle error here.

Question 3


          
           
            
             你可以尝试在你的代码中加入一些
             
              time.sleep
             
             的调用。
            
            
             似乎服务器端限制了每个时间单位（小时、天、秒）的请求量，这是一个安全问题。你需要猜测有多少个（也许用另一个带有计数器的脚本？
            
            
             为了避免你的代码崩溃，尝试用
             
              try .. except
             
             在 urllib2 调用周围捕捉这个错误。

Question 4


          
           
            
             有一种方法可以直接在except子句中用ConnectionResetError来捕获错误，这样可以更好地隔离出正确的错误。
这个例子也捕捉到了超时的情况。
            
            from urllib.request import urlopen 
from socket import timeout
url = "http://......"
    string = urlopen(url, timeout=5).read()
except ConnectionResetError:
    print("==> ConnectionResetError")
except timeout: 
    print("==> Timeout")

Question 5


          
           
            
             
              there are 2 solution you can try.
             
             
              request too frequently.
try
              
               sleep
              
              after per request
             
             time.sleep(1)
the server detect the request client is python, so reject.
add User-Agent in header to handle this.
    headers = {
        "Content-Type": "application/json;charset=UTF-8",
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)"
        res = requests.post("url", json=req, headers=headers)