Continuously replicate data from Oracle to ElasticSearch

Question

Team where I'm working has a luck to work on redesigning huge legacy system, with Oracle 12 on database end. Currently this monster software has 10% of insert/update/delete operations in DB, rest 90% of operations are the select operations (searching some for entities in a million records tables). And the search of course became slow. So our first thoughts of course is to have 2 same sets of data: Oracle DB and Elastic Search DB to perform search in Elastic, but store data in Oracle and right away to replicate data to Elastic. Between Oracle and Elastic will be some logic written on C# to convert data to Elastic Stack Format But first question we are don't have answer - how to get the changed data from Oracle as soon as it changed? First approach we found - to use Continuous Query Notification But this can be used only to initiate the data transfer process. The notification can contain the RowID of changed row, but it looks like the number of changed row notification event can carry limited to 80. Second approach - Use some Open source "Data Replicator", which monitors "Transaction logs". The only replicator I found - OpenLogReplicator, I like It but I want to ask StackOverflowers, may be this will not work for my case? Please note that Oracle GoldenGate we can not use, the client would not accept this. Can someone suggest which technology to use to replicate data changes from Oracle?

Answer 1

SharePlex is a cheaper alternative to GoldenGate. Since your need is "immediate" replication you need a relatively sophisticated solution that both SharePlex and GoldenGate should be able to handle.

Answer 2

OpenLogReplicator is at this moment if the phase of creating a working prototype. It is absolutely not ready for any production or even a testing. But you can contact the author to participate in a POC to help build the tool. Or buy one of the available on the market products.