Oracle Golden Gate Cassandra Handler

49 Views Asked by At

I have a configuration of this type:

  • Oracle GoldenGate for Big Data Version 21.9.0.0.3
  • Cassandra Handler 4.17.0 downloaded using the scripts provided directly by OGG through /DependencyDownloader/cassandra.sh

The process is launched with this parmaeters file

SPECIALRUN
END RUNTIME
TARGETDB LIBFILE libggjava.so SET property=dirprm/inievo.props
REPORTCOUNT EVERY 1 MINUTES, RATE
GROUPTRANSOPS 1000
EXTFILE ${EXTFILE}
MAP U_ALLIN.*, TARGET U_ALLIN.*;

I run the process with this configuration inievo.props


#The handler properties
gg.handler.cassandra.type=cassandra
gg.handler.cassandra.mode=op
gg.handler.cassandra.contactPoints=10.152.13.7,10.152.13.8,10.152.13.9
gg.handler.cassandra.ddlHandling=CREATE,ADD,DROP
gg.handler.cassandra.compressedUpdates=true
gg.handler.cassandra.cassandraMode=async
gg.handler.cassandra.consistencyLevel=LOCAL_QUORUM
gg.handler.cassandra.dataCenter=cas

#Regex properties
gg.schemareplaceregex=[$]
gg.schemareplacestring=_

#Javawriter properties
javawriter.stats.display=TRUE
javawriter.stats.full=TRUE
javawriter.bootoptions=-Xmx1G -Xms1G -Duser.timezone=UTC -Djava.class.path=.:ggjava/ggjava.jar:./dirprm

#Logging properties
gg.log=log4j
gg.log.level=INFO
gg.report.time=3600sec

gg.classpath=/shared/Middleware/GGPRE0T5/ggjava/lib/cassandra_4.17.0/*

I have 2 type of error:

  1. Oracle GoldenGate is writing to fast and i get this error. This error is not caused by the table reported in the error log, as sometimes relaunching the same process does not cause the error to occur again. Sometimes it appears on different tables, so It is not linked to the tables themselves. To decrease this writing speed I noticed that increasing the log level in the .props file for example from INFO to DEBUG. Is there another more correct method to decrease this writing speed on the OGG side?
    =ERROR 2024-01-23 11:52:50.000560 [main] - Async statement execution on Cassandra failed for table [U_ALLIN.IVF_RISERVAPREMI] at position [-0000000000000000001].
    com.datastax.oss.driver.api.core.AllNodesFailedException: All 3 node(s) tried for the query failed (showing first 3 nodes, use getAllErrors() for more): Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=
    2186c2ff): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=2186c2ff)], Node(endPoint=bdpre35cas.gruppoitas.loc
    al/10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d-75bb857eff5a, hashCode=4d8e4253): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d
    -75bb857eff5a, hashCode=4d8e4253)], Node(endPoint=10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=bd
    pre34cas.gruppoitas.local/10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7)]
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=2186c2ff)
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d-75bb857eff5a, hashCode=4d8e4253)
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7)
    Exception in thread "main" oracle.goldengate.util.GGException: Error detected handling transaction commit event.
            at oracle.goldengate.datasource.UserExitDataSource.commitTransaction(UserExitDataSource.java:2480)
            at oracle.goldengate.datasource.UserExitDataSource.commitTx(UserExitDataSource.java:2081)
    
    Source Context :
      SourceModule            : [gglib.ggdal.adapter.java]
      SourceID                : [ggdal/Adapter/Java/JavaAdapter.cpp]
      SourceMethod            : [HandleJavaException]
      SourceLine              : [301]
      ThreadBacktrace         : [19] elements
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(CMessageContext::AddThreadContext())]
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...))]
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(_MSG_String(CSourceContext*, int, char const*, CMessageFactory::MessageDisposition))]
                              : [/shared/Middleware/GGPRE0T5/libggjava.so()]
                              : [/shared/Middleware/GGPRE0T5/libggjava.so(ggs::gglib::ggdal::CJavaRecordWriter::CommitTransaction(int, int, int, int, ggs::gglib::ggdal::CDALError&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::ggdal::CDALRecordWriter::CommitTransaction(int, int, int, int, ggs::gglib::ggdal::CDALError&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(GenericImpl::CommitTransaction(int, int, int, int))]
                              : [/shared/Middleware/GGPRE0T5/replicat(odbc_commit_transaction())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::commitAndCheckpoint(short, bool&, RepCsn&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ClassicWorker::CommitAndCheckpoint(short))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::processReplicatLoop())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::run())]
                              : [/shared/Middleware/GGPRE0T5/replicat()]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::MainThread::ExecMain())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::MainThread::Run(int, char**))]
                              : [/shared/Middleware/GGPRE0T5/replicat(main)]
                              : [/lib64/libc.so.6(__libc_start_main)]
                              : [/shared/Middleware/GGPRE0T5/replicat()]
    
    2024-01-23 11:52:50  ERROR   OGG-15051  Java or JNI exception:
    oracle.goldengate.util.GGException: Error detected handling transaction commit event.
    
    ***********************************************************************
    *                   ** Run Time Statistics **                         *
  1. The Cassandra handler cannot transform Oracle TIMESTAMP types into Cassandra TIMESTAMP but transforms them into TEXT, is there a way to make a mapping between Oracle TIMESTAMP and Cassandra TIMESTAMP?

This is the Cassandra Hanlder log

Cassandra column [dta_effetto] maps to GG column [DTA_EFFETTO] Cassandra data type [TEXT] GG index [17].
Cassandra column [dta_evento] maps to GG column [DTA_EVENTO] Cassandra data type [TEXT] GG index [18].

This are the type from Oracle side

SQL> desc table_name
Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 DTA_EFFETTO                                        TIMESTAMP(9)
 DTA_EVENTO                                         TIMESTAMP(9)
0

There are 0 best solutions below