Rittman Mead Blog - Elasticsearch

New Performance Analytics Release

Poor performance is one of the most common complaints we hear upon arrival to customer sites, and often for good reason. When users have time to “make the rounds” before the system returns a result (if it returns a result at all), they have the right to be frustrated. Unfortunately,

elasticsearch

Streaming data from Oracle using Oracle GoldenGate and Kafka Connect

This article was also posted on the Confluent blog [http://www.confluent.io/blog/streaming-data-oracle-using-oracle-goldengate-kafka-connect/] , head over there for more great Kafka-related content! -------------------------------------------------------------------------------- Kafka Connect [http://docs.confluent.io/3.0.0/connect/index.html] is part of the Confluent Platform [http://www.confluent.io/product], providing a set

Technical

System Metrics Collectors

The need to monitor and control the system performances is not new. What is new is the trend of clever, lightweight, easy to setup, open source metric collectors in the market, along with timeseries databases to store these metrics, and user friendly front ends through which to display and analyse

Technical

Experiments with Elastic’s Graph Tool

Elastic announced their Graph [https://www.elastic.co/products/graph] tool at ElastiCON 2016 (see presentation here [https://www.elastic.co/elasticon/conf/2016/sf/graph-capabilities-in-the-elastic-stack] ). It’s part of the forthcoming X-Pack [https://www.elastic.co/v5] which bundles Graph along with other helper tools such as Shield and

Big Data

Forays into Kafka - Enabling Flexible Data Pipelines

One of the defining features of “Big Data” from a technologist’s point of view is the sheer number of tools and permutations at one’s disposal. Do you go Flume or Logstash? Avro or Thrift? Pig or Spark? Foo or Bar? (I made that last one up). This wealth

Big Data

Forays into Kafka - Logstash transport / centralisation

The holy trinity of Elasticsearch, Logstash, and Kibana (ELK) are a powerful trio of tools for data discovery [https://www.rittmanmead.com/blog/2015/04/using-the-elk-stack-to-analyse-donors-choose-data/] and systems diagnostics [https://www.rittmanmead.com/blog/2014/10/monitoring-obiee-with-elasticsearch-logstash-and-kibana/] . In a nutshell, they enable you to easily search through your log files,

Technical

Three Easy Ways to Stream Twitter Data into ElasticSearch

For the past few months a friend has been driving me crazy with all his praise for Splunk. He was going on about how easy it is to install, integrate different sources and build reports. I eventually started playing around to see if it could be used for a personal

Big Data

Using the ELK Stack to Analyse Donor's Choose Data

Donor’s Choose [http://www.donorschoose.org/about] is an online charity in America through which teachers can post details of projects that need funding and donors can give money towards them. The data from the charity [http://data.donorschoose.org/]since it began in 2000 is available to download

Technical

BI Forum 2015 Preview -- OBIEE Regression Testing, and Data Discovery with the ELK stack

I’m pleased to be presenting at both of the Rittman Mead BI Forums [https://www.rittmanmead.com/blog/biforum2015/] this year; in Brighton it’ll be my fourth time, whilst Atlanta will be my first, and my first trip to the city too. I’ve heard great things about

Technical

An Introduction to Analysing ODI Runtime Data Through Elasticsearch and Kibana 4

An important part of working with ODI is analysing the performance when it runs, and identifying steps that might be inefficient as well as variations in runtime against a baseline trend. The Operator tool in ODI itself is great for digging down into individual sessions and load plan executions, but

Technical

Analytics with Kibana and Elasticsearch through Hadoop - part 3 - Visualising the data in Kibana

In this post we will see how Kibana can be used to create visualisations over various sets of data that we have combined together. Kibana is a graphical front end for data held in ElasticSearch, which also provides the analytic capabilities. Previously we looked at where the data came from

Technical

Analytics with Kibana and Elasticsearch through Hadoop - part 2 - Getting data into Elasticsearch

Introduction In the first part of this series [https://www.rittmanmead.com/blog/2014/11/analytics-with-kibana-and-elasticsearch-through-hadoop-part-1-introduction/] I described how I made several sets of data relating to the Rittman Mead blog from various sources available through Hive. This included blog hits from the Apache webserver log, tweets, and metadata from

Technical

Monitoring OBIEE with Elasticsearch, Logstash, and Kibana

Monitoring the health of an OBIEE system and diagnosing problems that may occur is a vital task for the system’s administrator and support staff. It’s one that at Rittman Mead we help customers with implementing themselves, and also provide as a managed service [http://www.rittmanmead.com/support/