The Microservices Architecture (MA) for Oracle GoldenGate is a new REST API Microservices-based architecture that allows you to install, configure, monitor, and manage Oracle GoldenGate services using a web-based UI.
Really there are two versions of GoldenGate now: classic and microservice. Classic architecture has standard extract, replicat, pump and receiver. It is managed by classic ggsci. Microservice Architecture (MA) has different types of processes and managed using Admin Client or using web UI. See architecture of GoldenGate MA below
Classic architecture was managed using ggsci console and had weak authentication and authorization tools. Oracle GoldenGate MA has ability to verify identity using basic authentication and using SSL client certificates.
GoldenGate MA processes
Oracle GoldenGate MA uses different types of processes to perform same tasks as GoldenGate Classic. Let’s talk a little bit about new processes:
Service Manager. Something like (and replacement of) Manager process. This is watchdog for other processes.
Administration Server. Something like ggsci console. Operates as central control entity. You use it to create and manage other processes. The key feature of Administration Server is REST API which can be accesses from any HTTP or HTTPS client.
You can add, delete or alter GoldenGate processes, edit configuration files, add users and assign roles using Administration Server.
Receiver Server. Something like collector. It can receive trail files from remote server. However it replaces multiple collectors because it is multithreaded. Receiver was designed to be protocol agnostic – so it supports HTTPS, HTTP, UDT (reliable UDP) and classic GoldenGate TCP transports. By default it uses HTTPS protocol.
Distribution Server. Something like pump. But again this multithreaded process which can handle multiple trail at the same time. So it will replace multiple pumps. And again it supports multiple protocols: WebSockets for HTTPS-based streaming, which relies on SSL security, UDT, SOCKS5, HTTP. It also support Passive mode to initiate connection from remote side.
Performance Metrics Server. This is process which collects and saves information from other processes (extracts, replicats, etc). All GoldenGate processes push information to Performance Metrics Server. Now this is the only processes which writes data to GoldenGate datastore (Berkley DB). You can use Performance Metrics Server to query various metrics, view logs, process statuses, monitor system utilization, etc.
Admin Client. It is a command line utility (similar to ggsci) used to create, configure and manage processes. Admin Client uses REST API to accomplish its tasks.
Admin Client has much more function and very usable in distributed configuration than ggsci. See table below:
|Can manage||local deployments only||all MA deployment|
|Type of access to server||ssh or local console||http or https|
|Connection to DBMS||Required||NOT required|
|Security||Operation System security||MA security|
|Connected||by default||requires separate CONNECT command|
|Login to database using||USERID+PASSWORD, USERIDALIAS||USERIDALIAS|
|Sequence of extract commands||REGISTER EXTRACT before ADD EXTRACT||REGISTER EXTRACT after ADD EXTRACT|
|Security of communications||unsecure||secure (uses SSL)|
|Transfer data using||Pump||Distribution Server|
GoldenGate MA key directories and variables
GoldenGate MA directory structure was revised according to Linux Foundation Filesystem Hierarchy Standard. GoldenGate installation consist of read-only and read-write directories.
Oracle Database Home (ORACLE_HOME). Points to Oracle Database home which is extracted or replicated by GG (/database_install_location)
Oracle GoldenGate Home (OGG_HOME). Read-Only directory contains GoldenGate binaries, libraries, etc (/ogg_install_location)
Deployment etc home (OGG_ETC_HOME). Directory containing configuration artifacts (/ogg_deployment_location/etc)
Deployment configuration home (OGG_CONF_HOME). Directory containing configuration artifacts. (OGG_ETC_HOME/conf)
Deployment security home (OGG_SSL_HOME). Directory containing security artifacts. (OGG_ETC_HOME/ssl)
Deployment variable home (OGG_VAR_HOME). Directory containing logs, reports, temporary files (?) (/ogg_deployment_location/var).
Deployment data home (OGG_DATA_HOME). Directory containing data (trail files) (OGG_VAR_HOME/lib/data).
You can change default location of stored files. Directory structure was designed to make upgrade easier.
Steps to implement Microservices Architecture
There are steps you need to finish before using GoldenGate MA:
Start Service Manager
Start and Use Admin Client
Let’s talk about each step.
There is separate document describing Installation of GG MA (https://docs.oracle.com/goldengate/c1230/gg-winux/OGGIN/toc.htm)
1. First of all download installation media for GoldenGate MA from edelivery.oracle.com or otn.oracle.com. GoldenGate MA has separate distribution.
2. Unzip and start installation using runInstaller
3. Choose database version
4. Choose directory for installation.
Installation is very easy.
Configure Service Manager
Service Manager is starting point for activities related to GoldenGate configuration. For example you can use it to create or edit deployments. One Service Manager will manage multiple deployments of GoldenGate.
1. First of all lets set OGG_HOME variable. It will be used later
2. Change directory and run oggca.sh to configure GoldenGate Service Manager (it can be secure or unsecure, for example)
[oracle@scgg1gg123 bin]$ cd $OGG_HOME/bin
[oracle@scgg1gg123 bin]$ ./oggca.sh
3. First screen will ask us about general Service Manager parameters. I used separate directory for deployments because it makes easier to upgrade GoldenGate. Also I chose to use port 10000 and to register Service Manager as daemon (good news!). On the next screen choose to create new deployment. Then choose name for deployment and directory where deployment will be created.
4. Enter Oracle Database specific variables. Then Enter information to create admin account
5. For simplicity we will choose to not use SSL/TLS security.
6. Enter ports which will be used by different processes. It is easier to enter port for Administration Server and other ports will be enter by incrementing previous port by 1
7. Specify default schema which will be used by GoldenGate (for example, ggadmin as we use usually). Then check other parameter and finish configuration
8. Run create script as root. Wait until it finished and press OK.
9. Now you can login to Service Manager and see running services.
10. Service Manager restarts automatically If you reboot server. This is because configurator created daemon OracleGoldenGate.
11. Repeat everything in this section for the second database (on another or the same server)
Configure Credentials and checkpoint tables
GoldenGate uses database connections during its work. Also we need connection to database to create some types of processes, add trandata (supplemental logging) and create checkpoint tables. Let’s prepare credentials.
1. Login to GoldenGate Service Manager on the source server using URL http://scgg1:10000/. Then click on Administration Server.
2. Enter menu by click it in the left upper corner and choose configuration
3. Click “+” to create new credentials. Then enter required information.
Credential Domain (choose any name for domain): mydomain
Credential Alias (choose name for credentials new credentials, i.e. username_dbname): ggadmin_cdb1
User ID (username and tns string for database, i.e. username@hostname:port/service_name): c##ggadmin@localhost:1521/cdb1
Password and Verify Password: passwords
5. If login was successful you will 3 new tables: Checkpoint, Trandata and Heartbeat. Let’s use them.
7. Add trandata for you schema
8. Add heartbeat tables if you want (it is not required for GoldenGate but very useful)
9. Finally you will have the following picture.
10. Configure Global checkpoint table by clicking tab “Parameter file”. Choose GLOBALS and add line “checkpointtable ggadmin.checkpoint”
11. Click Overview to return to Administration Server configuration page.
12. Repeat previous steps on both source and destination databases.
Configure extract process
We will configure standard one-way GoldenGate replication. Everything will be done through web console.
1. If you go to Administration Server we can see no extracts were created. Click on “+” button to create new extract. Then choose Integrated Extract. Click Next. Enter required information
2. On the “Parameter file” page enter other parameter which required to run extract. For example, you can enter table specification. Then click “Create and Run button”
6. You can get error. In my case I entered incorrect username. Also I’ve set subdirectory for trail files but didn’t created it. So press OK, correct error and retry to start process.
7. Then run some inserts and look into detatail by pressing Action->Details. See multiple tabs which contain different information for extract.
Configure distribution path
Now we need to create process to distribute trail data to different server. This work is done by Distribution Service. So return to GoldenGate Service Manager (http://scgg1:10000/) and click Distribution Server. You will get to Distribution Server page
1. Let’s create new distribution path by clicking “+” button. Enter required information and click “Create and Run”. I chose ogg protocol to send data because it require no additional configuration. You can loging to destination host to know Receiver Server port. My port was 10003.
2. We can look into statistics (Actions->Detail->Statistics) to see that all data generated early was sent.
3. Now let’s go to destination and create replicat to apply data. My destination GoldenGate instance is available at http://scgg2:10000/. We can check that data were received from source by clicking “Receiver Server”
1. Return to GoldenGate Service Manager and click Administration Server. Then click “+” to create Replicat
2. Enter required information.
3. Edit parameter file. For example, I’ve added MAP and INSERTALLRECORDS directive
useridalias ggadmin_orcl2 domain mydomain
MAP orcl1.ggtest.*, TARGET ggtest1.*;
4.Click button “Create and Run” and wait for staring process
5. Run some workload and check statistics of Replicat. It should work
We just finished configuration of GoldenGate in Microservice Architecture mode. It was done using web interface. This interface is great for one time task and demonstrating GoldenGate. However you should use AdminClient utility for production cases – it is more useful for production cases.