Send file using POST from a Python script

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
From: https://requests.readthedocs.io/en/latest/user/quickstart/#post-a-multipart-encoded-file
Requests makes it very simple to upload Multipart-encoded files:
with open('report.xls', 'rb') as f:
    r = requests.post('http://httpbin.org/post', files={'report.xls': f})
That's it. I'm not joking - this is one line of code. The file was sent. Let's check:
>>> r.text
  "origin": "179.13.100.4",
  "files": {
    "report.xls": "<censored...binary...data>"
  "form": {},
  "url": "http://httpbin.org/post",
  "args": {},
  "headers": {
    "Content-Length": "3196",
    "Accept-Encoding": "identity, deflate, compress, gzip",
    "Accept": "*/*",
    "User-Agent": "python-requests/0.8.0",
    "Host": "httpbin.org:80",
    "Content-Type": "multipart/form-data; boundary=127.0.0.1.502.21746.1321131593.786.1"
  "data": ""
                I'm trying the same thing & its working fine if file size is less than ~1.5 MB. else its throwing an error.. please have look at here.
– Niks Jain
                Nov 28, 2013 at 7:39
                what am trying to do is login to some site using request which i have done successfully but now i want to upload a video after logging in and the form has a different fields to be filled before submission. So how should I pass those values like videos description,videos title etc
– TaraGurung
                May 31, 2015 at 10:02
                You'd probably want to do with open('report.xls', 'rb') as f: r = requests.post('http://httpbin.org/post', files={'report.xls': f}) instead, so it closes the file again after opening.
– Hjulle
                Aug 8, 2015 at 20:49
                This answer should be updated to include Hjulle's suggestion of using the context manager to ensure file is closed.
– bmoran
                Jan 27, 2017 at 16:22
                this is not working for me, it says '405 method not allowed.'      with open(file_path, 'rb') as f:         response = requests.post(url=url, data=f, auth=HTTPBasicAuth(username=id, password=password)                            )
– Saurabh Jain
                Jul 17, 2017 at 11:31
Yes. You'd use the urllib2 module, and encode using the multipart/form-data content type. Here is some sample code to get you started -- it's a bit more than just file uploading, but you should be able to read through it and see how it works:
user_agent = "image uploader"
default_message = "Image $current of $total"
import logging
import os
from os.path import abspath, isabs, isdir, isfile, join
import random
import string
import sys
import mimetypes
import urllib2
import httplib
import time
import re
def random_string (length):
    return ''.join (random.choice (string.letters) for ii in range (length + 1))
def encode_multipart_data (data, files):
    boundary = random_string (30)
    def get_content_type (filename):
        return mimetypes.guess_type (filename)[0] or 'application/octet-stream'
    def encode_field (field_name):
        return ('--' + boundary,
                'Content-Disposition: form-data; name="%s"' % field_name,
                '', str (data [field_name]))
    def encode_file (field_name):
        filename = files [field_name]
        return ('--' + boundary,
                'Content-Disposition: form-data; name="%s"; filename="%s"' % (field_name, filename),
                'Content-Type: %s' % get_content_type(filename),
                '', open (filename, 'rb').read ())
    lines = []
    for name in data:
        lines.extend (encode_field (name))
    for name in files:
        lines.extend (encode_file (name))
    lines.extend (('--%s--' % boundary, ''))
    body = '\r\n'.join (lines)
    headers = {'content-type': 'multipart/form-data; boundary=' + boundary,
               'content-length': str (len (body))}
    return body, headers
def send_post (url, data, files):
    req = urllib2.Request (url)
    connection = httplib.HTTPConnection (req.get_host ())
    connection.request ('POST', req.get_selector (),
                        *encode_multipart_data (data, files))
    response = connection.getresponse ()
    logging.debug ('response = %s', response.read ())
    logging.debug ('Code: %s %s', response.status, response.reason)
def make_upload_file (server, thread, delay = 15, message = None,
                      username = None, email = None, password = None):
    delay = max (int (delay or '0'), 15)
    def upload_file (path, current, total):
        assert isabs (path)
        assert isfile (path)
        logging.debug ('Uploading %r to %r', path, server)
        message_template = string.Template (message or default_message)
        data = {'MAX_FILE_SIZE': '3145728',
                'sub': '',
                'mode': 'regist',
                'com': message_template.safe_substitute (current = current, total = total),
                'resto': thread,
                'name': username or '',
                'email': email or '',
                'pwd': password or random_string (20),}
        files = {'upfile': path}
        send_post (server, data, files)
        logging.info ('Uploaded %r', path)
        rand_delay = random.randint (delay, delay + 5)
        logging.debug ('Sleeping for %.2f seconds------------------------------\n\n', rand_delay)
        time.sleep (rand_delay)
    return upload_file
def upload_directory (path, upload_file):
    assert isabs (path)
    assert isdir (path)
    matching_filenames = []
    file_matcher = re.compile (r'\.(?:jpe?g|gif|png)$', re.IGNORECASE)
    for dirpath, dirnames, filenames in os.walk (path):
        for name in filenames:
            file_path = join (dirpath, name)
            logging.debug ('Testing file_path %r', file_path)
            if file_matcher.search (file_path):
                matching_filenames.append (file_path)
            else:
                logging.info ('Ignoring non-image file %r', path)
    total_count = len (matching_filenames)
    for index, file_path in enumerate (matching_filenames):
        upload_file (file_path, index + 1, total_count)
def run_upload (options, paths):
    upload_file = make_upload_file (**options)
    for arg in paths:
        path = abspath (arg)
        if isdir (path):
            upload_directory (path, upload_file)
        elif isfile (path):
            upload_file (path)
        else:
            logging.error ('No such path: %r' % path)
    logging.info ('Done!')
                On python 2.6.6 I was getting an error in Multipart boundary parsing while using this code on Windows. I had to change from string.letters to string.ascii_letters as discussed at stackoverflow.com/questions/2823316/… for this to work. The requirement on boundary is discussed here: stackoverflow.com/questions/147451/…
– amit kumar
                Jan 19, 2011 at 12:32
                calling run_upload ({'server':'', 'thread':''}, paths=['/path/to/file.txt']) causes error in this line: upload_file (path) because "upload file" requires 3 parameters so I replaces it with this line  upload_file (path, 1, 1)
– tabdulradi
                Oct 19, 2011 at 23:13
Looks like python requests does not handle extremely large multi-part files.
The documentation recommends you look into requests-toolbelt.
Here's the pertinent page from their documentation.
The only thing that stops you from using urlopen directly on a file object is the fact that the builtin file object lacks a len definition. A simple way is to create a subclass, which provides urlopen with the correct file. 
I have also modified the Content-Type header in the file below.
import os
import urllib2
class EnhancedFile(file):
    def __init__(self, *args, **keyws):
        file.__init__(self, *args, **keyws)
    def __len__(self):
        return int(os.fstat(self.fileno())[6])
theFile = EnhancedFile('a.xml', 'r')
theUrl = "http://example.com/abcde"
theHeaders= {'Content-Type': 'text/xml'}
theRequest = urllib2.Request(theUrl, theFile, theHeaders)
response = urllib2.urlopen(theRequest)
theFile.close()
for line in response:
    print line
                @robert I test your code in Python2.7 but it doesn't work.   urlopen(Request(theUrl, theFile, ...)) merely encodes the content of file as if a normal post but can not specify the correct form field.   I even try variant urlopen(theUrl, urlencode({'serverside_field_name': EnhancedFile('my_file.txt')})), it uploads a file but (of course!) with incorrect content as <open file 'my_file.txt', mode 'r' at 0x00D6B718>.   Did I miss something?
– RayLuo
                Mar 4, 2013 at 15:38
                Thanks for the answer . By using the above code I had transferred 2.2 GB raw image file by using PUT request into the webserver.
– Akshay Patil
                Jul 19, 2013 at 14:37
def test_upload_file(self):
        filename = "/Users/Ranvijay/tests/test_price_matrix.csv"
        data = {'file': open(filename, 'rb')}
        client = APIClient()
        # client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)
        response = client.post(reverse('price-matrix-csv'), data, format='multipart')
        print response
        self.assertEqual(response.status_code, status.HTTP_200_OK)
#импорт вспомогательных библиотек
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
import requests
#импорт http_file
from http_file import download_file
#создание новой сессии
s = requests.Session()
#соеденение с сервером через созданную сессию
s.get('URL_MAIN', verify=False)
#загрузка файла в 'local_filename' из 'fileUrl' через созданную сессию
download_file('local_filename', 'fileUrl', s)
                It has since moved to github.com/httplib2/httplib2. On the other hand, nowadays I would probably recommend requests instead.
– pdc
                Mar 1, 2017 at 10:03
def visit_v2(device_code, camera_code):
    image1 = MultipartParam.from_file("files", "/home/yuzx/1.txt")
    image2 = MultipartParam.from_file("files", "/home/yuzx/2.txt")
    datagen, headers = multipart_encode([('device_code', device_code), ('position', 3), ('person_data', person_data), image1, image2])
    print "".join(datagen)
    if server_port == 80:
        port_str = ""
    else:
        port_str = ":%s" % (server_port,)
    url_str = "http://" + server_ip + port_str + "/adopen/device/visit_v2"
    headers['nothing'] = 'nothing'
    request = urllib2.Request(url_str, datagen, headers)
        response = urllib2.urlopen(request)
        resp = response.read()
        print "http_status =", response.code
        result = json.loads(resp)
        print resp
        return result
    except urllib2.HTTPError, e:
        print "http_status =", e.code
        print e.read()
I tried some of the options here, but I had some issue with the headers ('files' field was empty).
A simple mock to explain how I did the post using requests and fixing the issues:
import requests
url = 'http://127.0.0.1:54321/upload'
file_to_send = '25893538.pdf'
files = {'file': (file_to_send,
                  open(file_to_send, 'rb'),
                  'application/pdf',
                  {'Expires': '0'})}
reply = requests.post(url=url, files=files)
print(reply.text)
More at https://requests.readthedocs.io/en/latest/user/quickstart/
To test this code, you could use a simple dummy server as this one (thought to run in a GNU/Linux or similar):
import os
from flask import Flask, request, render_template
rx_file_listener = Flask(__name__)
files_store = "/tmp"
@rx_file_listener.route("/upload", methods=['POST'])
def upload_file():
    storage = os.path.join(files_store, "uploaded/")
    print(storage)
    if not os.path.isdir(storage):
        os.mkdir(storage)
        for file_rx in request.files.getlist("file"):
            name = file_rx.filename
            destination = "/".join([storage, name])
            file_rx.save(destination)
        return "200"
    except Exception:
        return "500"
if __name__ == "__main__":
    rx_file_listener.run(port=54321, debug=True)
        Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.