How do I obtain a raw json string from an environment variable in python?

Question

In BASH, I can store JSON into an environment variable via jq:

export FOO=$(curl ... | jq -Rs)

Now, if I load that into python:

bar=os.environ["FOO"]

bar will have something like: {\\"this\\":\\"that\\"}, which is a problem when loading with json:

json.loads(bar) # error

---

I have tried a few things including repr, rf'{os.environ["FOO"]}' and so on, but there doesn't seem to be an internally managed way to drop the extra slashes.

How do I drop the extra slashes via string functions? I'd prefer not to simply replace them with a single slash, as I might have to touch that code again sometime in the future.

Answer 1

Add the -r flag to output as raw text from jq. Here is an example using echo that gives the escaped results:

echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | jq -Rs
# gives:
"{\"fruit\":{\"name\":\"apple\",\"color\":\"green\",\"price\":1.20}}\n"

But adding -r gives a result that is still in JSON format without all of the escaped quotes:

echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | jq -Rsr
# gives:
{"fruit":{"name":"apple","color":"green","price":1.20}}

Which is parseable by Python.

export FOO=$(echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | jq -Rsr)

import os
import json

json.loads(os.environ['FOO'])
# returns:
# {'fruit': {'name': 'apple', 'color': 'green', 'price': 1.2}}

Answer 2

I tried to reconstruct this:

export FOO=$(curl https://jsonplaceholder.typicode.com/todos/1|jq -Rs)

This is what is stored from the looks in bash:

"{\n \"userId\": 1,\n \"id\": 1,\n \"title\": \"delectus aut autem\",\n \"completed\": false\n}"

Then in python shell:

>>> import json
>>> import os
>>> json.loads(json.loads(os.environ["FOO"]))
{'userId': 1, 'id': 1, 'title': 'delectus aut autem', 'completed': False}

So basically json.loads() doesn't give you any guarantees regarding the type of Python object you get, but it tries to de-serialize whatever you feed it and so a nested de-serialization does the job. But of course, it is better, if you can make sure that the environment contains the data in the correct format in the first place as the other answer suggests.