Let’s define terms which Oracle uses in Oracle Cloud Services.
- Block Storage — optimizes storage for IOPS and block-based access and provides POSIX-compliant file systems for Oracle Compute Cloud Service instances. This is just standard disk device. Sometimes it is only one drive, sometimes it is RAID device. But anyway application access it using standard disk operations
- Object Storage — scalable storage which can store large binary objects with metadata and unique ID. Multiple storage nodes form a single, shared, horizontally scalable pool. Application can access data using REST API.
Oracle Storage Cloud Service provides a low cost, reliable, secure, and scalable object-storage solution for storing unstructured data and accessing it anytime from anywhere. It is ideal for data backup, archival, file sharing, and for storing large amounts of unstructured data like logs, sensor-generated data, and VM images.
Continue reading ‘Oracle Storage Cloud Service: Creating Containers Using the REST API’ »
You work with GoldenGate then you know that there was tool GoldenGate Director for configuration, deployment, monitoring and management. Really these responsibilities are distributed among development and support. So there should be two tools: for development and for support. Oracle is following this ideology and has created two tools:
- GoldenGate Monitor (or GG plugin for OEM if you are OEM fan) — for management and monitoring
- GoldenGate Studio — for design and deployment
What do we need from development tools? I have the following list in my head:
- WYSIWYG/drag’n’drop interface
- easy moving configuration through the path Dev/QA/Prod
- collective working
GoldenGate has these features. It was created to be great scalable development tool.
Continue reading ‘GoldenGate Studio: Quick Start’ »
Virtual Machine Oracle BigData Lite is vm created by Oracle for development and demo purposes of BigData (Cloudera) and Oracle technologies working together.
You can download BigData Lite VM from Oracle site. This vm includes following software (for vm version 4.4):
Continue reading ‘Oracle BigData Lite VM introduction’ »
Интеграция данных является неотъемлемой фазой в построении оптимальных бизнес-процессов компании любого уровня и сферы деятельности. Компания Oracle предлагает лучшую в своем классе интеграционную платформу для решения задач:< ?xml:namespace prefix = "o" />
- Быстрого обогащения данных – Big Data Preparation Cloud Service
- Трансформации данных – Oracle Data Integrator
- Управления и повышения качества данных при помощи – Oracle Enterprise Data Quality
- Репликации данных в режиме реального времени – Oracle GoldenGate
- Управления слоем метаданных различных приложений – Oracle Metadata Management
Continue reading ‘Семинар «Интеграция в эпоху облачных технологий и больших данных» 25 февраля 2016 в 11:00’ »
One of GoldenGate 12.2 new features is replication support for MSSQL Standard Edition. In the previous GoldenGate 12.1 release, GoldenGate supported capture of DML only from SQL Server Enterprise Edition, due to the method of enabling Supplemental Logging, which is done via a feature only available to Enterprise Editions of SQL Server. Fortunately, beginning with Oracle GoldenGate 12.2, GoldenGate is able to enable Supplemental Logging for Standard Edition instances via a different mechanism. That mechanism to enable Supplemental Logging is done via certain SQL Server Replication components. These SQL Server Replication components are required to be installed and configured in order to enable Supplemental Logging,
Also it is very interesting that MSSQL doesn’t offer Change Data Capture for MSSQL Standard Edition. So if we have 100 offices with MSSQL and we need to collect only changed data to Data Warehouse then it is required to install MSSQL Enterprise Edition in every office because CDC is only available in MSSQL EE.
Continue reading ‘Configuring GoldenGate to replicate data from MSSQL Standard Edition (CDC, Change Data Capture)’ »
When install GoldenGate Monitor we create special credentials used by GoldenGate Agent and GoldenGate Monitor to secure communications.
Also we set this credentials while configuring GoldenGate Agent using pw_agent_util.sh.
But is not well documented how these credentials are used and how change them. So let fill these gap.
Continue reading ‘GoldenGate Monitor 12.2: how to change password for internal communications’ »
I already wrote about configuring GoldenGate agent for ODI and Enterprise Manager. Now I would like to talk about configuring it for GoldenGate Monitor. GoldenGate is interesting tool for GoldenGate monitoring. It is more powerful and flexible than GoldenGate plugin for Enterprise Manager but it is still standalone tool not integrated with OEM. So it has its own advantages and disadvantages.
From the technical standpoint configuring GoldenGate Agent for Monitor is a little bit different than configuring GoldenGate Agent for OEM and ODI. Also I see some inaccuracies in documentation and step-by-step configuration is not crystal clear. That’s why I writing this blog post — I would like to have short step-by-step instruction how to configure GoldenGate Agent for Monitor.
So let’s look at GoldenGate monitoring framework architecture
We can see
- GoldenGate Monitor has its own Server with Repository (which is Oracle Database)
- Each GoldenGate instance has Agent (Monitor Agent)
- Monitor Server and Monitor agents communicates through JMX (which can also be encrypted by SSL)
- Agent communicates with Manager to get information about GoldenGate state
Continue reading ‘Configuring GoldenGate Monitor 12.2’ »
GoldenGate pre-184.108.40.206 doesn’t store table structure in trail files and in some cases it was an issue: when we received trail file — we didn’t know source table structure from the file. We had only record and this record contained field values. But we didn’t know field names. So earlier we chose how to manage this. There are two ways:
— assume that source table is the same as target table (ASSUMETARGETDEFS). In this case first value from trail record was written to first field of target table, the second value to the second field, etc. But this method doesn’t work in general case where source and target tables are different
— extract table definitions from source and gave this definition to replicat (SOURCEDEFS). In this case trail files were parsed and applied according to the static source table definition (stored in file). This method gave us flexibility but increased number of manual work: when source table structure changes — we must regenerate DEF-file or we can lose data consistency.
Now we don’t need ASSUMETARGETDEFS and SOURCEDEFS anymore. GoldenGate automatically transfers definitions in trail files. Moreover it retransfer definition when table structure changes (but retransfer will be triggered only after DML on this table) and also retransfer will be done after switch to new trail file (also we should wait for first DML).
There are two special new record types for table definition transfer: DDR (Database Definition Record) and TDR (Table Definition Record). The first one describes database, the second one is for table structure.
Continue reading ‘GoldenGate 220.127.116.11 New Features: Metadata Encapsulation’ »
Core GoldenGate is software completely written in C/C++. But many Oracle tools are written using Java. For example, Enterprise Manager and GoldenGate Monitor. ETL tool Oracle Data Integrator was also developed using Java. All these tools work with GoldenGate and communications should be secured: there should be authentication, encryption, etc.
Oracle uses special agent to make these communications possible. We call these agent GoldenGate Agent (previously it was called JAgent). If will look at GoldenGate 11.2 or 12.1 then we see this agent is embedded: there are folders cfg and dirjar in GoldenGate Home. First folder contains GoldenGate configuration, the second one contains java libraries (jar) for GoldenGate agent execution.
There were some limitation of embedded agent: you couldn’t create multiple agents (for OEM and GG Monitor), you must remove old agent and add new one to make communications with server (Monitor or OEM) more stable. Oracle has removed GoldenGate Monitor from core GoldenGate 12.2 installation to overcome these limitations. So we should install GoldenGate Agent before configuring communications with OEM and ODI. Unfortunately GoldenGate Agent documentation is a little bit poor. So am writing this article to compensate this issue.
Continue reading ‘Configuring GoldenGate agent for ODI and Enterprise Manager (version 12.2)’ »
В GoldenGate 18.104.22.168 появилась возможность реплицировать данные, подключившись к Active Data Guard (ADG) экземпляру. В отличие от режима ALO этот режим обеспечивает Real-Time репликацию. Плюс ADG дает возможность вообще не создавать подключение к источнику — достаточно подключения к экземпляру ADG.
Давайте посмотрим, как это работает. Сначала нам нужно создать экземпляр ADG.
Continue reading ‘Настройка репликации GoldenGate из Active Data Guard база данных’ »