In this article, we will see how to perform complex data queries in Kibana visualization tool. If you are afraid of using kibana visualization tool then I would recommend you to first check Are you afraid of using Kibana Data Visualization Tool? article and understand how easily we can use query in kibana to get the desired output. Once you have that understanding, it will be easy to go further and understand how to query the exact data from kibana logs to narrow down the search and get results without much effort. This is more important as many times due to usage of special characters in data, it is difficult to query and might show you syntax error. Here we will see few examples that can help understand the technique.

 

How to perform complex data queries in kibana visualization tool

Also Read: How to Install and Configure Filebeat on Ubuntu/Debian

Now that you understand the basic queries to perform in Kibana to get the desired result, it is time to learn to query the complex data that includes lot of special characters and signs. Often to narrow down your search, it is important to query the exact data that you need instead of querying some parts of the data. But the problem comes when the exact data that you need to search in kibana might contain lot of special characters which you can't query as it is. You have to use a technique so that kibana understand what you are looking for.

This can be understood and performed by many ways but I will explain you the most simplistic way that will make your life easier. First, it is important to get the exact data that you need to search in kibana. I would prefer to get that data from kibana itself and then use it to narrow down my search. For example, if the exact data contains some keyword such as JHK90876 then I would straight away search this data in kibana by querying "JHK90876" in the search section. This will give me all the output that contains JHK90876 keyword.

data: JHK90876

query: "JHK90876"

Now, I need to extract the exact data from the output shown. Once I locate the data that I need to search, I will just copy it from the output instead of typing it myself so that I don't have to worry about spaces and then I would modify the data according to the format kibana accepts for search. To understand this further, let's take a quick example. Suppose the exact data that we need to search in kibana is  EmpId: "JHK90876". To search this data, we have to use below query in Kibana.

data: EmpId: "JHK90876"

query: "EmpId:\"JHK90876\""

You may notice that we have added backslash(\) before double inverted comma(") on both sides of the keyword. This is being done so that kibana understand double quotes as part of data to query. Similarly, if you have data something like below to search then you have to modify that accordingly to query. You may notice that we have not added any escape character for colon(:) and comma(,), just for double inverted comma(""). This means you just have to modify the double inverted comma("") only and leave everything as it is.

data: DeptId: 7HJ9087, EmpId: "JHK90876"

query: "DeptId: 7HJ9087, EmpId: \"JHK90876\""

Next is what would happen if I add braces({}) as well on above data? Well, that will only change the position of double inverted comma in the query. Rest remains exactly the same as explained in previous example.

data: { DeptId: 7HJ9087, EmpId: "JHK90876" }

query: "{ DeptId: 7HJ9087, EmpId:\"JHK90876\" }"

We will see one more example where data now looks like below where we have added square brackets([]) and an extra braces over the top of previous data. But as you can see there is not much modification required in the query apart from what we have been doing till now.

data: { DeptId: 7HJ9087, EmpId: "JHK90876" }, { DeptId: URS8965, EmpId: "LKJ71325" }]}

query: "{ DeptId: 7HJ9087, EmpId: \"JHK90876\" }, { DeptId: URS8965, EmpId: \"LKJ71325\" }]}"

You can further keep on adding more special characters but that does not make any difference. You will still be putting escape character("\") in front of double inverted comma("") only and then rest remains the same. Hope this is informative and helps you to perform complex queries without using of any fields.