Mongo DB Error while Updating - error on remote shard - caused by cursor id

Question

I have about 8 Million Documents in my Collection. And I want to remove the special Characters in one of the fields. I will post my Statement below. I am using the mongo shell in the Mongo db compass tool. The update is working about 30-50 Minutes and then throws the following error:

MongoServerError: Error on remote shard thisisjustforstack.com:27000 :: caused by :: cursor id 1272890412590646833 not found

I also see that after throwing this error, he did not update all documents.

    db.getCollection('TEST_Collection').aggregate(
[{
    $match: {
        '1List.Comment': {
            $exists: true
        }
    }
}, {
    $project: {
        '1List.Comment': 1
    }
}]
)
.forEach(function(doc,Index) {doc.1List.Comment=doc.1List.Comment.replace(/[^a-zA-Z 0-9 ]/g, '');
db.TEST_Collection.updateMany({ "_id": doc._id },{ "$set": { "1List.Comment": doc.1List.Comment } });})

Can somebody please help to get this update statement working without running in some sort of timeout? I have read something about noCursorTimeout() but I am not sure on how to use it with my statement and using it in the shell.

Thank you all!

Answer 1

Cursor timeout can't be disabled on individual aggregation cursors.

But you can set on global config:

mongod --setParameter cursorTimeoutMillis=3600000    #1 hour

Anyway I think dividing the task in small batches is a better option