I am working on a requirement where I need to transfer all Salesforce records From Object A to Object B for this I need to use only the Jitterbit ETL Tool.
In Object A total of 40+million records, In the source I cannot use a write-back option while querying the Records. thought of using a checkbox or something but it will affect other integrations based on object A If I had that it would be easy I could query only 1million each time.
Explored different Options:
I thought of querying the id's and do order by and storing the last record id and do order by again I could repeat the process but after 3million records ids are not consistent.
finally, I decided to go with the query every month on Created Date to query the records.
I am storing the source record Id in Target as an external Id so that I won't insert duplicates.
Is there any better approach for this? did anyone has done this scenario before please let me know?
It would be a great help Thanks in advance