Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I am currently trying to read a bson file to import it into a database. I can already read the file and print it as as bytes, but I end up just getting an bson.errors.InvalidBSON: objsize too large error.

This is the code that is trying to decode the file

with zip.open(name) as myfile:
    content = myfile.read()
    print(content)
    print(bson.decode_all(content))
and this is the output i get
b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'
Traceback (most recent call last):
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/bin/mongo-backup", line 11, in <module>
    load_entry_point('mongo-backup-cli', 'console_scripts', 'mongo-backup')()
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/cli.py", line 45, in restore
    restore_files(uri, db_name, file.name)
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/restore.py", line 6, in restore_files
    print(read_zip_file(file))
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/zip.py", line 24, in read_zip_file
    print(bson.decode_all(content))
bson.errors.InvalidBSON: objsize too large
print(content)
b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'
the bytes in the content variable is json encoded bson, not plain bson
if this is the output format you intend to continue to use, you need to change your code to use bson's JSON utility to load the string into a python object:
with zip.open(name) as myfile:
    content = myfile.read()
    print(content)
    print(bson.json_util.loads(content))
    #         ^--------------^
    #                | this stuff
                You are probably using the standalone bson package rather than the supported pymongo package that includes a supported bson package.  Pip uninstall bson and pip install pymongo
– bauman.space
                Aug 13, 2020 at 16:42
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.