Skip to main content

MongoDB BulkWrite Java API

Since version 3.2, MongoDB has introduced Bulk Update methods. In context of RDBMS, it's like SQL Batch Jobs, where SQL Statements are prepared in different chunks and a batch of statements are submitted to DB for update/insert.

Here are some important points about MongoDB Bulk Write operation..


  1. Useful in case you've huge data to update/insert.
  2. Mongo automatically prepares batches (of 1000 default) and start execution in an ordered/unordered manner.
  3. This drastically reduce DB trip time. Let's say there are 50 thousand records to update, now instead of 50k round trips to DB from your app server, using Bulk Update it would be reduced to just 50 round trips.

Let's see an example below:


List<WriteModel<Document>> updateDocuments = new ArrayList<WriteModel<Document>>();
for (Long entityId : entityIDs) {

    //Finder doc
    Document filterDocument = new Document();
    filterDocument.append("_id", entityId);

    //Update doc
    Document updateDocument = new Document();
    Document setDocument = new Document();
    setDocument.append("name", "xyz");
    setDocument.append("role", "abc");

    updateDocument.append("$set", setDocument);

    //Update option
    UpdateOptions updateOptions = new UpdateOptions();
    updateOptions.upsert(true); //if true, will create a new doc in case of unmatched find
    updateOptions.bypassDocumentValidation(true); //set true/false

    //Prepare list of Updates
    updateDocuments.add(
            new UpdateOneModel<Document>(
                    filterDocument,
                    updateDocument,
                    updateOptions));

}

//Bulk write options
BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
bulkWriteOptions.ordered(false); //False to allow parallel execution
bulkWriteOptions.bypassDocumentValidation(true);

MongoCollection<Document> mongoCollection = mongoDB.getCollection("myCollection");

BulkWriteResult bulkWriteResult = null;
try {
    //Perform bulk update
    bulkWriteResult = mongoCollection.bulkWrite(updateDocuments,
            bulkWriteOptions);
} catch (BulkWriteException e) {
    //Handle bulkwrite exception
    List<BulkWriteError> bulkWriteErrors = e.getWriteErrors();
    for (BulkWriteError bulkWriteError : bulkWriteErrors) {
        int failedIndex = bulkWriteError.getIndex();
        Long failedEntityId = entityIDs.get(failedIndex);
        System.out.println("Failed record: " + failedEntityId);
        //handle rollback
    }
}

int rowsUpdated = bulkWriteResult.getModifiedCount();


Now let's understand the process..

entityIDs: List of _id s to update

filterDocument: query filter. equivalent to SQL where clause

setDocuments: values to update. equivalent to SQL set statement

updateOptions: manner in which update should happen. 

bulkWriteOptions: write operation preferences. If entityIDs are independent of each other, you should go for un-ordered execution, simply like parallel threads.


bulkWriteErrors: Errors if any during update process 


for Bulk deletion, we just need to prepare DeleteOneModel instead of UpdateOneModel documents, rest would be same

List<WriteModel<Document>> deleteDocuments = new ArrayList<WriteModel<Document>>();
for (Long entityId : entityIDs) {

    //Finder doc
    Document filterDocument = new Document();
    filterDocument.append("_id", entityId);

    //Delete doc
    Document deleteDocument = new DeleteOneModel<Document>(filterDocument);
    //Prepare list of Deletes
    deleteDocuments.add(deleteDocument);
}


Comments

Post a Comment

Popular posts from this blog

How to create users in MongoDB

Assuming you've already completed the basic setup process mentioned in this blog . In this example, we'll see how we can create admin user and enable authorization. Connect to admin database On a cmd window, connect to MongoDB by hitting command " mongo ". By default, it connects to " test " database. Once connection is successful, switch to admin db #mongo #use admin Create Role After switching to admin db, create executiveFunction role db . runCommand ({ createRole : "executeFunctions" , privileges : [ { resource : { anyResource : true }, actions : [ "anyAction" ] } ], roles : [], writeConcern : { w : "majority" , wtimeout : 5000 } }); Create admin user In this step, we'll create admin user mongoadmin with password mongo123 db . runCommand ( { "createUser" : "mongoadmin" , "pwd" : "mongo123...

MongoDB Backup and Restore

It's a general need in MongoDB development, to take backups or restore DB with an old backup. DB backup can be done at both DB level and individual Collection level. Let's see how we can perform different backup/restore operations.. DB Backup Complete DB Assuming database to backup is "mydb", the best thing is to back it as gzip to save some space on your Server. mongodump -- archive = mydb . 2017 - 09 - 21.gz -- gzip -- db mydb This will create the archive in the directory where you're executing the command. Another way is to take backup as bson documents. mongodump -h localhost -p 27017 -d mydb -o C:\mongobackup\20170921 A directory with db name will be created under  C:\mongobackup\20170921 In case you have multiple instances running on same server, you can use --host and --port options. Individual Collection mongodump -- collection myCollection -- db mydb This will create backup with same name as that of collection (myCollection in this...