Mongodb pymongo errors DocumentTooLarge
NickName:Jofetu Ask DateTime:2020-07-21T19:19:14

Mongodb pymongo errors DocumentTooLarge

I am trying to do the following query: Query

the problem is {"id": {"$nin": lista}}. lista is an array of ids that cointains 1.000.000 ids exceeding 98MB size. Since Mongodb has a limit of 16MB it seems i cannot send this query without getting the following error: pymongo.errors.DocumentTooLarge: BSON document too large (107382731 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.. Something I dont complete understand because the limit is supposted to be for documents inside collections.

My question is: How I can exclude a list/array of ids then?

Copyright Notice:Content Author:「Jofetu」,Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/63013365/mongodb-pymongo-errors-documenttoolarge

More about “Mongodb pymongo errors DocumentTooLarge” related questions

Mongodb pymongo errors DocumentTooLarge

I am trying to do the following query: Query the problem is {"id": {"$nin": lista}}. lista is an array of ids that cointains 1.000.000 ids exceeding 98MB size. Since Mongodb has a

Show Detail

MongoDB - pymongo.errors.DocumentTooLarge Error

I have a REST interface which POSTs data to MongoDB. The data is basically file system structure as a nested JSON structure where each level in the JSON has a key "children". This children contains...

Show Detail

pymongo.errors.DocumentTooLarge: 'aggregate' command document too large

I aggregate data using pymongo, the data is about 10 millions, I use df_list = mycol.aggregate([{'$match': {'tbl_id': {'$in': doc_list}}}, {'$project': {'_id': 0}}], allowDiskUse=True) # doc_list

Show Detail

DocumentTooLarge during query

I want to get a large number (1 million) of documents by their object id, stored in a list obj_ids. So I use docs = collection.find({'_id' : {'$in' : obj_ids}}) However, when trying to access the

Show Detail

mongodb and pymongo 16Mb limit on document size

I'm analyzing the includes in a build using the '/showIncludes', on Windows, and '-H', on *nix, flags. I'm parsing this info with a python script. Each file included is turned into an object that ...

Show Detail

MongoDB / PyMongo

I'm running a script that scrapes a quite large amount of data, and then use a function to upload that data to mongodb. I keep getting hit by connection closed errors and can't figure out why it ha...

Show Detail

Unkown error while executing heavy query on Pymongo

Pymongo .find() throwing error when find takes too long or contains a lot of documents. By wrapping everything in a try/catch and printing the exception nothing prints out. What Im thinking is tha...

Show Detail

pymongo.errors.ServerSelectionTimeoutError: RepliceSetNoPrimary - MongoDB Atlas

The problem is that whenever (I have tried 3 times) I create a new cluster on MongoDB Atlas and connect to it in Python using PyMongo, for the first 3-4 it connect painlessly but after sometime or ...

Show Detail

How to show pymongo.errors.OperationFailure details?

I am getting a pymongo OperationsFailure in python when writing to MongoDB. Is there a way to print out the detail or code attribute in addition to the traceback? Also see: http://api.mongodb.org/...

Show Detail

MongoDB $nin query exceeds BSON document size limitation

I am performing a simple query using Pymongo against a list of filenames: db.FileAssets.find({'filename': {'$nin': filenames}}}) However, the query is throwing the DocumentTooLarge error. The le...

Show Detail