Robin Moffatt

127 posts published

📍 Yorkshire, UK

Using R with Jupyter Notebooks and Oracle Big Data Discovery

Using R with Jupyter Notebooks and Oracle Big Data Discovery

Oracle's Big Data Discovery [https://www.oracle.com/big-data/big-data-discovery/index.html] encompasses a good amount of exploration, transformation, and visualisation capabilities for datasets residing in your organisation’s data reservoir. Even with this though, there may come a time when your data scientists want to unleash their

Using SparkSQL and Pandas to Import Data into Hive and Big Data Discovery

Using SparkSQL and Pandas to Import Data into Hive and Big Data Discovery

Big Data Discovery [https://www.oracle.com/big-data/big-data-discovery/index.html] (BDD) is a great tool for exploring, transforming, and visualising data stored in your organisation’s Data Reservoir. I presented a workshop on it at a recent conference [https://speakerdeck.com/rmoff/unlock-the-value-in-your-big-data-reservoir-using-oracle-big-data-discovery-and-oracle-big-data-spatial-and-graph] , and got an interesting question from

Using Jupyter Notebooks with Big Data Discovery 1.2

Using Jupyter Notebooks with Big Data Discovery 1.2

New in Big Data Discovery [https://www.oracle.com/big-data/big-data-discovery/index.html] 1.2 is the addition of BDD Shell, an integration point with Python. This exposes the datasets and BDD functionality in a Python and PySpark environment, opening up huge possibilities for advanced data science work on BDD

Running Big Data Discovery Shell and Jupyter Notebook on Big Data Lite VM 4.5

Running Big Data Discovery Shell and Jupyter Notebook on Big Data Lite VM 4.5

New in Big Data Discovery [https://www.oracle.com/big-data/big-data-discovery/index.html] 1.2 is the addition of BDD Shell, an integration point with Python. This exposes the datasets and BDD functionality in a Python and PySpark environment, opening up huge possibilities for advanced data science work on BDD

OBIEE 12c – Extended Subject Areas (XSA) and the Data Set Service

OBIEE 12c – Extended Subject Areas (XSA) and the Data Set Service

One of the big changes in OBIEE 12c for end users is the ability to upload their own data sets and start analysing them directly, without needing to go through the traditional data provisioning and modelling process and associated leadtimes. The implementation of this is one of the big architectural

Changes in BI Server Cache Behaviour in OBIEE 12c : OBIS_REFRESH_CACHE

The OBIEE BI Server cache can be a great way of providing a performance boost to response times for end users - so long as it's implemented carefully. Done wrong, and you're papering over the cracks and heading for doom; done right, and it's

Under the Covers of OBIEE 12c Configuration with sysdig

OBIEE 12c has changed quite a lot in how it manages configuration. In OBIEE 11g configuration was based around system MBeans [https://docs.oracle.com/middleware/11119/biee/BIESG/configsettings.htm#BIESG1643] and the biee-domain.xml as the master copy of settings - and if you updated a configuration directly

Experiments with Elastic’s Graph Tool

Experiments with Elastic’s Graph Tool

Elastic announced their Graph [https://www.elastic.co/products/graph] tool at ElastiCON 2016 (see presentation here [https://www.elastic.co/elasticon/conf/2016/sf/graph-capabilities-in-the-elastic-stack] ). It’s part of the forthcoming X-Pack [https://www.elastic.co/v5] which bundles Graph along with other helper tools such as Shield and

New OTN Article - OBIEE Performance Analytics: Analysing the Impact of Suboptimal Design

New OTN Article - OBIEE Performance Analytics: Analysing the Impact of Suboptimal Design

I’m pleased to have recently had my first article published on the Oracle Technology Network (OTN) [http://www.oracle.com/technetwork/index.html]. You can read it in its full splendour and glory(!) over there [https://community.oracle.com/docs/DOC-993649], but I thought I’d give a bit

OBIEE Performance - Why Metrics Matter (and…Announcing obi-metrics-agent v2!)

OBIEE Performance - Why Metrics Matter (and…Announcing obi-metrics-agent v2!)

One of the first steps to improve OBIEE performance is to determine why it is slow. That may sound obvious—can’t fix it if you don’t know what you’re fixing, right? Unfortunately, the “Drunk Man anti-method [http://www.brendangregg.com/methodology.html]”, in which we merrily stumble

OBIEE 12c - Repository Password Corruption Issue

Here at Rittman Mead we’ve been working with OBIEE 12c for some time now, as part of the beta programme and more recently with clients looking to get the most out an upgrade to OBIEE 12c [https://www.rittmanmead.com/blog/2016/01/obiee12c-three-months-in-whats-the-verdict/]. We’ve also been hard

Using Linux Control Groups to Constrain Process Memory

Using Linux Control Groups to Constrain Process Memory

Linux Control Groups [http://docs.oracle.com/cd/E37670_01/E37355/html/ol_about_cgroups.html] (cgroups) are a nifty way to limit the amount of resource, such as CPU, memory, or IO throughput, that a process or group of processes may use. Frits Hoogland [https://fritshoogland.wordpress.com/2012/

Driving OBIEE User Engagement with Enhanced Usage Tracking for OBIEE

Driving OBIEE User Engagement with Enhanced Usage Tracking for OBIEE

Measuring and monitoring user interactions and behaviour with OBIEE is a key part of Rittman Mead’s User Engagement Service [http://ritt.md/ue]. By understanding and proving how users are engaging the system we can improve the experience for the user, driving up usage and ensuring maximum value for

Forays into Kafka - Enabling Flexible Data Pipelines

Forays into Kafka - Enabling Flexible Data Pipelines

One of the defining features of “Big Data” from a technologist’s point of view is the sheer number of tools and permutations at one’s disposal. Do you go Flume or Logstash? Avro or Thrift? Pig or Spark? Foo or Bar? (I made that last one up). This wealth

Introducing the Rittman Mead OBIEE Performance Analytics Service

Introducing the Rittman Mead OBIEE Performance Analytics Service

Fix Your OBIEE Performance Problems Today OBIEE is a powerful analytics tool that enables your users to make the most of the data in your organisation. Ensuring that expected response times are met is key to driving user uptake and successful user engagement with OBIEE. Rittman Mead can help diagnose

Forays into Kafka - Logstash transport / centralisation

Forays into Kafka - Logstash transport / centralisation

The holy trinity of Elasticsearch, Logstash, and Kibana (ELK) are a powerful trio of tools for data discovery [https://www.rittmanmead.com/blog/2015/04/using-the-elk-stack-to-analyse-donors-choose-data/] and systems diagnostics [https://www.rittmanmead.com/blog/2014/10/monitoring-obiee-with-elasticsearch-logstash-and-kibana/] . In a nutshell, they enable you to easily search through your log files,

Using the BI Server Metadata Web Service for Automated RPD Modifications

Using the BI Server Metadata Web Service for Automated RPD Modifications

A little-known new feature of OBIEE 11g is a web service interface to the BI Server. Called the “BI Server Metadata Web Service [https://docs.oracle.com/cd/E28280_01/bi.1111/e16364/bi_server_web_services.htm#BIEIT3235] ” it gives a route into making calls into the BI Server

Managing the OBIEE BI Server Cache from ODI 12c

Managing the OBIEE BI Server Cache from ODI 12c

I wrote recently about the OBIEE BI Server Cache and how useful it can be [https://www.rittmanmead.com/blog/2015/09/obiee-bi-server-cache-management-strategies/], but how important it is to manage it properly, both in the purging of stale data and seeding of new. In this article I want to show

OBIEE BI Server Cache Management Strategies

OBIEE BI Server Cache Management Strategies

The OBIEE BI Server cache can be one of the most effective ways of improving response times of OBIEE dashboards. By using data already in the cache it reduces load on the database, the network, and the BI Server. Should you be using it? I always describe it as the

Security patches released for OBIEE 11.1.1.7/11.1.1.9, and ODI DQ 11.1.1.3

Security patches released for OBIEE 11.1.1.7/11.1.1.9, and ODI DQ 11.1.1.3

Oracle issued their quarterly Critical Patch Update [http://www.oracle.com/technetwork/topics/security/cpujul2015-2367936.html] yesterday, and with it notice of several security issues of note: * The most serious for OBIEE (CVE-2013-2186 [https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-2186]) rates 7.5 (out of 10) on

What's New in OBIEE 11.1.1.9 for Systems Administrators and Developers

What's New in OBIEE 11.1.1.9 for Systems Administrators and Developers

After over two years since the last major release of OBIEE, Oracle released version 11.1.1.9 in May 2015. You can find the installers here [http://www.oracle.com/technetwork/middleware/bi-enterprise-edition/downloads/bi-downloads-2537285.html] and documentation here [https://docs.oracle.com/middleware/11119/biee/docs.htm]. 11.

Using the ELK Stack to Analyse Donor's Choose Data

Using the ELK Stack to Analyse Donor's Choose Data

Donor’s Choose [http://www.donorschoose.org/about] is an online charity in America through which teachers can post details of projects that need funding and donors can give money towards them. The data from the charity [http://data.donorschoose.org/]since it began in 2000 is available to download

BI Forum 2015 Preview -- OBIEE Regression Testing, and Data Discovery with the ELK stack

BI Forum 2015 Preview -- OBIEE Regression Testing, and Data Discovery with the ELK stack

I’m pleased to be presenting at both of the Rittman Mead BI Forums [https://www.rittmanmead.com/blog/biforum2015/] this year; in Brighton it’ll be my fourth time, whilst Atlanta will be my first, and my first trip to the city too. I’ve heard great things about

Analysing ODI performance with Flame Graphs

Flame Graphs [http://www.brendangregg.com/flamegraphs.html] are a visualisation that I learnt about through the excellent Linux systems performance work of Brendan Gregg [http://www.brendangregg.com/], and saw Luca Canali [https://twitter.com/lucacanalidb] talk about recently at UKOUG Tech 14 [http://tech14.ukoug.org/default.asp?

Lifting the Lid on OBIEE Internals with Linux Diagnostics Tools

There comes the point in any sufficiently complex or difficult problem diagnosis that the log files in OBIEE alone are not sufficient for building up a complete picture of what’s going on. Even with the debug/trace data that Presentation Services [https://www.rittmanmead.com/blog/2014/11/auditing-obiee-presentation-catalog-activity-with-custom-log-filters/