Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I need to compile a binary file in pieces with pieces arriving in random order (yes, its a P2P project)

def write(filename, offset, data) 
    file.open(filename, "ab")
    file.seek(offset) 
    file.write(data) 
    file.close()
Say I have a 32KB write(f, o, d) at offset 1MB into file and then another 32KB write(f, o, d) at offset 0 
I end up with a file 65KB in length (i.e. the gap consisting of 0s between 32KB - 1MB is truncated/disappears) 
I am aware this may appear an incredibly stupid question, but I cannot seem to figure it out from the file.open(..) modes 
Advice gratefully received. 
*** UPDATE 
My method to write P2P pieces ended up as follows (for those who may glean some value from it) 
def writePiece(self, filename, pieceindex, bytes, ipsrc, ipdst, ts): 
    file = open(filename,"r+b")
    if not self.piecemap[ipdst].has_key(pieceindex):
        little = struct.pack('<'+'B'*len(bytes), *bytes) 
        # Seek to offset based on piece index 
        file.seek(pieceindex * self.piecesize)
        file.write(little)
        file.flush()
        self.procLog.info("Wrote (%d) bytes of piece (%d) to %s" % (len(bytes), pieceindex, filename))
    # Remember we have this piece now in case duplicates arrive 
    self.piecemap[ipdst][pieceindex] = True
    file.close()
Note: I also addressed some endian issues using struct.pack which plagued me for a while. 
For anyone wondering, the project I am working on is to analyse BT messages captured directly off the wire. 
                I need sleep - this is such a rooky question I cannot believe I asked it. Apologies.   I needed to use: file.open("r+b") to achieve my intended goal.
– codeasone
                Aug 4, 2010 at 16:24
                Now I have a file with 32KB at offset 0 and 32KB at offset 1MB with rest of the file consisting of zeroes.
– codeasone
                Aug 4, 2010 at 16:26
...         f = open(filename,'r+b')
...     except IOError:
...         f = open(filename,'wb')
...     f.seek(offset)
...     f.write(data)
...     f.close()
>>> write(filename,'1' * (1024*32),1024*1024)
>>> write(filename,'1' * (1024*32),0)
>>> os.path.getsize(filename)
1081344
                Sorry for my haste in entering the code example. To create a file with pieces of data at various offsets and otherwise filled with zeroes I used file.open("r+b") mode and just seek-ed and wrote data to the file as I needed.
– codeasone
                Aug 4, 2010 at 16:28
                You think that's what bittorrent does with a 2GB download? What are you doing to do, keep the pieces in memory?
– MattH
                Aug 4, 2010 at 20:19
                Save each piece, as it's received, to a separate file, then merge when done. I seem to recall that's what e.g. Limewire does, but I don't know about most torrent clients.
– Russell Borogove
                Aug 6, 2010 at 21:34
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.