I've started using ELK stack (Elasticsearch, Logstash, Kibana) as a default go-to option whenever I require a log management solution in a project that I work on. Analysing and searching through the logs is much nicer when done through Kibana's web UI then grepping through the log files as you would do if you had no log management. However, from time to time, there still comes a moment where I wish I could just tail -f the log file to see what's happening right now in this moment. After googling around and not finding what I wanted, I ended up building Elktail, a command line utility that allows you to search and tail your logs stored in Elasticsearch.
Using Elktail, you can connect to the Elasticsearch instance that hosts your logstash logs and tail them. The experience is similar to good old tail -f. You can also add search terms (just like you would in the search bar in Kibana), which will yield results similar to grepping the log file.
Here's how you would connect to Elasticsearch instance running on elastic.example.com in order to tail logstash logs:
elktail --url http://elastic.example.com
Elktail also supports basic authentication and ssh tunneling in case your Elasticsearch cluster is not publicly available over the internet but you do have ssh access to it.
Once you can list your logs on the command line, several additional workflows open up. For example, I often need to extract and e-mail part of the log file to my fellow developers or attach it together with a bug report in the issue tracker. I just want to copy paste 100 relevant lines, but it's often hard to do that in Kibana, at least with satisfying results. Copy-pasting picks up html formatting and Kibana's pagination will get in the way. While I could share the link to the search, it's often not easy to setup the search in a way that isolates the crux of the issue.
Here are some examples of how you could use Elktail. Examples are based on the log format for Spring Boot applications, and ELK setup is explained in one of the previous posts.
Tail only messages with info level
elktail level:INFO (you need to specify --url only once, elktail will remember it for subsequent invocations)
Just do a search for a string
elktail DispatcherServlet
List last 2 execeptions and their stacktraces
elktail -l -n 2 tags:stacktrace
Change the display format to only show timestamp, level and log message fields:
elktail -f "%@timestamp %level %logmessage"
If you would like to try it out yourself, head over to the Elktail github page and download it.
