JValue in Json4s (Scala): How to create an empty JValue

Question

I want to create an empty JValue to be able to parse JSON objects together.

As for now I am creating the JValue containing {} and then I am parsing the other objects to it and then in the end I remove the first row using an RDD, but I would like to create

var JValue: JValue = JValue.empty

from the beginning to be able to skip the removing part.

Is it possible to create an empty JValue?

import org.json4s._
import org.json4s.jackson.JsonMethods._

var JValue: JValue = parse("{}")
val a = parse(""" {"name":"Scott", "age":33} """)
val b = parse(""" {"name":"Scott", "location":"London"} """)

JValue = JValue.++(a)
JValue = JValue.++(b)

val df = spark.read.json(Seq(compact(render(JValue ))) toDS())
val rdd = df.rdd.first()
val removeFirstRow = df.rdd.filter(row => row != rdd)
val newDataFrame = spark.createDataFrame(removeFirstRow,df.schema)

Answer 1

If I understand correctly what you are trying to achieve, you can start from an empty array like so:

var JValue: JValue = JArray(List.empty)

Calling the ++ method on the empty array will result in the items being added to that array, as defined here.

The final result is the following object:

[ {
  "name" : "Scott",
  "age" : 33
}, {
  "name" : "Scott",
  "location" : "London"
} ]

If you want to play around with the resulting code, you can have a look at this worksheet on Scastie (please bear in mind that I did not pull in the Spark dependency there and I'm not 100% sure that would work anyway in Scastie).

As you can notice in the code I linked above, you can also just to a ++ b to obtain the same result, so you don't have to necessarily start from the empty array.

As a further note, you may want to rename JValue to something different to avoid weird errors in which you cannot tell apart the variable and the JValue type. Usually in Scala types are capitalized and variables are not. But of course, try to work towards the existing practices of your codebase.