Although Kibana is an excellent and powerful data visualization tool that provides detail and meaningful data required to troubleshoot and debug many different issues with the applications and services but sometimes it has been observed that due to usage of lucene query syntax, users find it lot difficult to frame the right query to get the data they need. Even if someone trying to filter the data, sometimes users does not know the application and service name or let's say the correct and complete name to filter all the data related to that application or service.
This makes many users hate this tool as they might need to remember all this in case they want to query or visualize any data. This especially becomes nightmare for beginners who does not have much experience with this tool. If you look into documentation, sometimes you might have to break your head to make sense out of it. You have to spend too much time to learn and understand the query to use this visualization tool. I have personally faced this problem so I completely understand the pain.
Are you afraid of using Kibana Data Visualization Tool?
Also Read: How to Install and Configure Filebeat on Ubuntu/Debian
To get rid of all these type of problems and issues, I am going to tell you the most simplistic way of using this tool to query data so that you don't have to remember anything and it can be easily used in all kind of scenarios. Let's follow the steps from beginning:-
- Login to Kibana
- Switch to your application tenant
- Go to Discover
- In query box, type "<keyword>"
- Modify date and time
- Run your query
As you can see above, in query box you have to type "<keyword>" in which keyword is any related keyword you would like to search about. It could be any incomplete service name, application name, error, message etc. Easy right? Once you run the query, you will see everything related to that keyword on output. Now you have a good starting point to search required data.
You can now get the required fields from output along with correct and complete service name, application, message etc. Take the required field from output and filter the data based on that. You don't have to run any query now, just filter the data within the right time frame. To understand this further, let's take a quick example. Let's say you searched a keyword called consumer in the query and you got below data based on that search.
Time _source May 3, 2024 @ 13:15:41.115 app.name: example-cyberithub-consumer level: INFO @metadata.fingerprint: 7v532106-d237-4e45-dbe0-5ba922411b4f @timestamp: May 3, 2024 @ 11:26:41.115 @version: 5 app.instanceId: example-cyberithub-consumer:739418709 application: example-cyberithub-consumer component: microservice-api environment: prod file: /var/www/logs/example-cyberithub-consumer/1.log index_name: ecom-app
You have all the fields mentioned above. You can take those fields and search for the specific required data. For example, we would like to search all the error for application example-cyberithub-consumer. To do that, I would take app.name field from above output and put it in my filter and then for error, I would take level field from above output and query for ERROR. To do this, follow below steps:-
- Select Add filter
- In edit filter, select app.name in field section.
- Then select is in Operator section
- Finally enter example-cyberithub-consumer in Value section.
Above filter will query all the data related to application example-cyberithub-consumer. To filter all the errors related to this application, follow below steps:-
- Select Add filter
- In edit filter, select level in field section.
- Then select is in Operator section
- Finally enter ERROR in Value section.
Now you have two filter set which will give all the error logs related to application example-cyberithub-consumer in selected timeframe. If you don't get the search result then expand your timeframe and search the data again. You will see the required results shown on the output. Expand the data to see in detail. That's all needed to be done !!