This is the begining of a multiple part series of blog posts going through setting up Elasticsearch cluster in kubernetes.
Installing Kibana for Elasticsearch on OS X Published on December 10, 2015 by Bo Andersen The first thing you have to do in order to install Kibana for Mac OS X, is to download Kibana. Kibana can be downloaded here in your browser or by running the below command in your terminal.
- Elasticsearch is a real-time, distributed storage, search, and analytics engine. It can be used for many purposes, but one context where it excels is indexing streams of semi-structured data, such as logs or decoded network packets.
- This article walks through creating a DSN for Elasticsearch data in iODBC and accessing Elasticsearch data in Microsoft Excel, all on a machine running Mac OS X. Installing the CData ODBC Drivers on Mac OS X. The CData ODBC Driver for Elasticsearch is preconfigured for the iODBC driver manager, as are many other products like Microsoft Excel.
- Part 1 - Setting up Single Node Elasticsearch
The main aim of this series of blog posts will be make notes for myself as I try to learn kubernetes and for anyone in the same position.
In this blog post, I will just concentrate on useful Kubernetes getting started resources, commands, and also with an aim of creating a single node Elasticsearch cluster.
Getting Started
I found the most helpful resource for me was the Kubernetes official website for starting to learn kubernetes. Head over to the Interactive Tutorials section of the website and spend time going through all 6 modules to cover the basics. It should only take you 1 - 2 hours. You won’t have to install anything to try it out.
The next step I took was followed the Hello Minikube tutorial. This helped me to get minikube and kubectl commands installed. (Minikube is the local development Kubernetes environment and kubectl is the command line interface used to interact with Kubernetest cluster).
Shaving the Yak!
One or two commands that I use in this post will be mac specific. Reference this guide to get more up to date and OS specific commands.
Once you’ve got the tools all installed, you can now follow along these steps to create a single node Elasticsearch cluster.
If you are using Minikube, make sure that its started properly by running this command (for mac):
Now set the Minikube context. The context is what determines which cluster kubectl is interacting with.
Verify that kubectl is configured to communicate with your cluster:
To view the nodes in the cluster, run:
Kubernetes Dashboard
Minikube includes the kubernetes dashboard as an addon which you can enable.
returns
You can enable an addon using:
You can then open the dashboard with command
Single Node Elasticsearch Cluster
Let’s start off by creating a single node elasticsearch cluster. According to the elasticsearch documentation, the current version at the time of this writing is 6.1.1. And there are three flavours of docker images. We will just use the basic image which has xpack and free license.
Run the following command to deploy elasticsearch container into our kubernetes environment exposing just the port 9200. There is no way to expose multiple ports using the
kubectl
command line currently. We will probably revisit this in a later post.To list your deployments use:
To list all pods and watch the container getting created:
Save the pod name as a variable for use in later commands.
Note: The above command will not work if you have multiple pods.
Now, you can take a look at the logs using the command:
Expose the cluster
We can verify that the cluster is running by looking at the logs. But, let’s check if elasticsearch api is responding first.
Elasticsearch Machine Learning Api
In a seperate shell window, excute the following to start a proxy into Kubernetest cluster.
Outputs:
Now, back in the other window, lets execute a
curl
command to get the response from the pod via the proxy.Update - 18 Feb 2019: In later version of kubernetes, the proxy url is at http://localhost:8001/api/v1/namespaces/default/pods/$POD_NAME/proxy/
Outputs:
Great, everything is working.
Now, lets expose this deployment to outside of Kubernetes network:
Pro tip Use
MiniKube
to open the service in your default browser.In my case, the port that was assigned to this pod was
31389
. But, we have elasticsearch cluster now running in Kubernetes!Elasticsearch For Mac
Previously, we covered the Elastic Cloud. This is a fully managed solution that may not fit all the needs. How can you run Elasticsearch locally? How can you launch Kibana? And how do you use Dev Tools? We’ll answer those questions here.
Download Elasticsearch through this link.
Download the version that matches your OS. I’m running on macOS, so the article might contain some commands specific to macOS.
Step 1 : Download and unzip Elasticsearch
Find the file in your downloads, move it to the folder in which you want to store it. On a mac, double-click on the file to unzip it.
*Step 2 *: In your terminal, go to the folder in question
(Depending on your version, this link might change)
*Step 3 *: Fire up the engines!
We’ll now start the elastic search server.
The
bin
directory will also be used to start Elasticsearch SQL CLI to interact with Elasticsearch using SQL statements for example.Another important directory is the
config
one in which we can, for example, modify the default port for Elasticsearch (9200) in the elasticsearch.yaml
file.Elasticsearch is now up and running. Let’s download Kibana now!
Kibana can be downloaded from the following link.
Download the file that matches your OS.
Step 1 : Download and unzip Kibana
Find the file in your downloads, move it to the folder in which you want to store it. On a mac, double-click on the file to unzip it.
*Step 2 *: Fire up Kibana
In your terminal, go to the folder that contains Kibana, and run this command.
If you have not modified the default port of Kibana, everything should now work. You should see a log similar to this one :
*Step 3 *: Open your browser
Now, head to the following link in your browser : http://localhost:5601.
We now have both Kibana and Elasticsearch running!