How to retrieve values from json output in Bash

Written by Claudio Kuenzler - 0 comments

Published on April 29th 2016 - last updated on October 15th 2021 - Listed in Linux Coding Shell

---

Sure, one could get a json output with curl and then do all kinds of grepping, sedding and awking. But there is actually a cool way, to get the value directly by accessing the json objects. The magic is done by jshon. 

Note/Update October 15th 2021: This article about jshon was written in 2016. I have meanwhile discovered jq and switched to it, mainly because jq is under ongoing development compared to jshon.

Fortunately this (cli-)program doesn't need to be compiled manually, in Ubuntu it can be downloaded from the official repos:

$ sudo apt-get install jshon

Now let's do some json'ing. Let's take the ElasticSearch cluster stats for example. As you may know, you can get the current ES cluster statistics and information from the following URL: http://your.es.node:9200/_cluster/stats (for a human readable output you can use http://your.es.node:9200/_cluster/stats?human&pretty). 

The json output is separated into several objects. You can use jshon to list them:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -k
timestamp
cluster_name
status
indices
nodes

As you see above, I directly accessed the ES URL and piped the output to jshon. The -k parameter returns a list of keys. In this case we got 5 keys.

The value I am looking for is within the "indices" key, so I first display the values of the full key with the -e parameter (returns json value):

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices
{
 "count": 13,
 "shards": {
  "total": 61,
  "primaries": 61,
  "replication": 0.0,
  "index": {
   "shards": {
    "min": 1,
    "max": 5,
    "avg": 4.6923076923076925
   },
   "primaries": {
    "min": 1,
    "max": 5,
    "avg": 4.6923076923076925
   },
   "replication": {
    "min": 0.0,
    "max": 0.0,
    "avg": 0.0
   }
  }
 },
 "docs": {
  "count": 935249,
  "deleted": 196434
 },
 "store": {
  "size": "11.9gb",
  "size_in_bytes": 12860502682,
  "throttle_time": "3.7h",
  "throttle_time_in_millis": 13374212
 },
 "fielddata": {
  "memory_size": "7.9mb",
  "memory_size_in_bytes": 8328212,
  "evictions": 0
 },
 "filter_cache": {
  "memory_size": "78.1mb",
  "memory_size_in_bytes": 81991016,
  "evictions": 72989502
 },
 "id_cache": {
  "memory_size": "0b",
  "memory_size_in_bytes": 0
 },
 "completion": {
  "size": "0b",
  "size_in_bytes": 0
 },
 "segments": {
  "count": 453,
  "memory": "17.9mb",
  "memory_in_bytes": 18873642,
  "index_writer_memory": "0b",
  "index_writer_memory_in_bytes": 0,
  "index_writer_max_memory": "190.1mb",
  "index_writer_max_memory_in_bytes": 199421333,
  "version_map_memory": "0b",
  "version_map_memory_in_bytes": 0,
  "fixed_bit_set": "3.2mb",
  "fixed_bit_set_memory_in_bytes": 3448160
 },
 "percolate": {
  "total": 0,
  "get_time": "0s",
  "time_in_millis": 0,
  "current": 0,
  "memory_size_in_bytes": -1,
  "memory_size": "-1b",
  "queries": 0
 }
}

So this is the full output, but not the single value of what I actually wanted. To do this, the subkeys within the "indices" key can be shown and used the same way:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -k
percolate
shards
count
store
docs
fielddata
filter_cache
id_cache
completion
segments

So now we got a nice list of the subkeys which we can access directly with an additional -e:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -e store
{
 "size": "11.9gb",
 "size_in_bytes": 12860502682,
 "throttle_time": "3.7h",
 "throttle_time_in_millis": 13374212
}

And to get the final single value I wanted from the start, this can be retrieved by using yet another -e parameter:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -e store -e "size_in_bytes"
12860502682

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.