Using bash variables in jq

by on February 10, 2021 Share
  • jq
  • linux
  • technical

Due to the special characters used in json, the easiest way to use jq with inline scripts it by putting it between single quotes. That makes it impossible to use bash variables inside your script. Fortunately, jq has an --arg parameter to create a predefined variable from an external source.

You can use it to define a $foo variable with the contents of bash variable $FOO, for example:

$ export FOO="bar"
$ jq -n --arg foo "$FOO" '{foo: $foo}'
{
  "foo": "bar"
}

You might notice the -n parameter: This tells jq to use null as input instead of reading json from stdin like it normally would. I’m using it in all examples on this page as it makes them a bit shorter. In real world usage you probably won’t need it.

As usual in bash scripts you can also execute a command inline:

$ jq -n --arg hostname "$(hostname)" '{foo: $hostname}'
{
  "foo": "cypher"
}

Every variable created with --arg is treated as a string. That might cause some unexpected behaviour when you work with numbers, like when you attempt to add a number to an argument. Comparisons between strings and numbers seem to work fine, but I wouldn’t count on it (pun intended).

$ echo '{"a": 3.0}' | jq --arg foo 5 '$foo + .a'
jq: error (at <stdin>:1): string ("5") and number (3) cannot be added

There are two ways to solve this: either convert the variable to a number with tonumber or use --argjson instead. I prefer to use tonumber because that way I’m sure jq forces it to be a number. As my argument may be the result of an earlier command, I may be using an error message as input instead of the expected numeric result.

$ jq -n --arg foo 5 '($foo|tonumber) + 1'
6

$ jq -n --argjson foo 5 '$foo + 1'
6

Variables from files

It’s also possible to load variables from files. With --slurpfile we can read a file containing json objects. An array containing the separate objects is then made available as a predefined variable. Note that even if your file contains only a single json object, as the variable is still an array, the [0] is required to access it.

$ cat foo.json
{"foo": "bar"}
{"bar": "foo"}

$ jq -n --slurpfile foo foo.json '$foo[0].foo'
"bar"

A second option is --rawfile, creating a string variable with the exact contents of the file. Note that as jq outputs json, reading and printing the file from the previous example results an escaped string.

$ jq -n --rawfile foo foo.json '$foo'
"{\"foo\": \"bar\"}\n{\"bar\": \"foo\"}\n"

That’s it!