How to Use DocumentDb, LINQ, and F# to return multiple properties?

Question

Getting an error when running the F# version of the code below, while the C# version below works. Any ideas about on how to return multiple documentdb properties from a LINQ query with F# in general?

2016-12-29T23:57:08.504 Exception while executing function: Functions.GetTags. mscorlib: Exception has been thrown by the target of an invocation. mscorlib: One or more errors occurred. Microsoft.Azure.Documents.Client: Constructor invocation is not supported.

C#

var userTagsQuery = 
userDocument.Where(user => user.topics != null && user.education != null)
.Select(user => new {topics=user.topics, education=user.education});

F#

type UserTagRecord = {topics : string list; education : string list}
let userTagsQuery = 
user.Where(fun user -> user.topics <> null && user.education <> null)
.Select(fun user -> {topics=user.topics :?> string list; education=user.education :?> string list})

Answer 1

The error says "Constructor invocation is not supported". This is a sensible limitation, most LINQ providers have it.

In your F# code, the constructor call is the record creation. F# records are compiled as classes with a bunch of read-only properties and a single constructor that takes values for those properties as parameters. According to the error message, calling this constructor is not supported. Bad luck.

An interesting thing to notice is that C# anonymous types (employed in your C# code) work in exactly same way as F# records - they are classes with a bunch of read-only properties and a single constructor, - and yet, they are supported. Handing C# anonymous types as a special case is, again, a common practice for LINQ providers. Many LINQ providers would handle it in a more generalized way that would cover F# records as well, but in this case it is apparently not the case. If I were you, I would open an issue about it.

What you can try is replace your record with a class with mutable properties, and construct it with property initialization syntax:

type UserTagRecord() = 
   member val topics : string list = [] with get, set
   member val education : string list = [] with get, set

let userTagsQuery = 
   user
      .Where(fun user -> user.topics <> null && user.education <> null)
      .Select(fun user -> UserTagRecord( topics=user.topics :?> string list, education=user.education :?> string list ) )

I would also go out on a limb here and suggest that you might have further trouble with using F# lists. First, DocumentDb might not like them by themselves, and second, I don't know what user.topics and user.education are, but I'm pretty sure that they're not a subclass of string list, so your casts will probably fail.

Answer 2

I gave this a decent shot trying many forms of types: custom, structs, CLIMutable records, mutable fields, anonymous records, tuples, structs but ultimately either the FSharp quotations conversion or the CosmosClient Expression tree parsing rejected me.

But, between the Cosmos Client and F#, it does tolerate projecting things into an array...

---

(Will polish and re-post eventually; this will likely wind up in Equinox.CosmosStore)

---

My best hack without resorting to C# is for now to project the items VERY carefully as an obj[]

1. do the query, yielding an IQueryable<obj[]>:

   let all: IQueryable<obj[]> =
       ...
      .Select(fun i -> [| i._etag; i.u[0] |] : obj[])
   

   (that's a string and a JSON Object in my case)

2. let ToQueryDefinition yield you the query

3. make sure you supply a STJ serializer to the CosmosClient

   let ser = Equinox.CosmosStore.Core.CosmosJsonSerializer(System.Text.Json.JsonSerializerOptions())
   let client = new CosmosClient(cs, clientOptions = CosmosClientOptions(Serializer = ser))
   

4. Parse it using a JsonIsomorphism:

   [<StjConverter(typeof<StjResultParser>)>]
   type Result = { etag: string; unfold: Unfold<string> }
   and StjResultParser() =
       inherit FsCodec.SystemTextJson.JsonIsomorphism<Result, System.Text.Json.JsonElement[]>()
       let serdes = FsCodec.SystemTextJson.Serdes.Default
       override _.UnPickle input = { etag = serdes.Deserialize input[0]; unfold = serdes.Deserialize input[1] }
       override _.Pickle value = invalidOp "readOnly"
   

5. Walk the results using something like, e.g.:

   let private taskEnum (iterator: FeedIterator<'T>) = taskSeq {
       while iterator.HasMoreResults do
           let! response = iterator.ReadNextAsync()
           yield response.Diagnostics.GetClientElapsedTime(), response.RequestCharge, response.Resource }
   
   let fetch<'T> (desc: string) (container: Container) (queryDefinition: QueryDefinition) = taskSeq {
       if Log.IsEnabled Serilog.Events.LogEventLevel.Debug then Log.Debug("CosmosQuery.fetch {desc} {query}", desc, queryDefinition.QueryText)
       let sw = System.Diagnostics.Stopwatch.StartNew()
       let iterator = container.GetItemQueryIterator<'T>(queryDefinition)
       let mutable responses, items, totalRtt, totalRu = 0, 0, TimeSpan.Zero, 0.
       try for rtt, rc, response in taskEnum iterator do
               responses <- responses + 1
               totalRu <- totalRu + rc
               totalRtt <- totalRtt + rtt
               for item in response do
                   items <- items + 1
                   yield item
       finally Log.Information("CosmosQuery.fetch {desc} found {count} ({trips}r, {totalRtt:N0}ms) {rc}RU {latency}ms",
                               desc, items, responses, totalRtt.TotalMilliseconds, totalRu, sw.ElapsedMilliseconds) }
   

In the end you can walk the results like so:

all.ToQueryDefinition()
|> CosmosStoreQuery.fetch<Result> "items" source.Container
|> TaskSeq.iter (fun x -> printfn $"%s{x.etag} %O{x.unfold}")

Answer 3

Based on this answer, I arrived at:

open System.Linq.Expressions

let expandExpr find replace =
    { new ExpressionVisitor() with
        override _.Visit node =
            if node = find then replace
            else base.Visit node }

// We want to generate a projection statement of the shape: VALUE {"sn": root["p"], "snap": root["u"][0].["d"]}
// In C#, one could `.Select(x => new { sn = x.p, snap = x.u[0].d })`
// However the Cosmos SDK does not support F# (or C#) records yet https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3728
type SnAndSnap<'State>() =
    member val sn: FsCodec.StreamName = Unchecked.defaultof<_> with get, set
    member val snap: 'State = Unchecked.defaultof<_> with get, set
    // This hack generates an expression equivalent to the 
    static member FromItemExpr<'I>(u: Expression<System.Func<Item<'I>, 'I>>) =
        let t = typedefof<SnAndSnap<'I>>
        let param = Expression.Parameter(typedefof<Item<'I>>, "x")
        let dummy = Unchecked.defaultof<SnAndSnap<'I>>
        let etagProp = t.GetMember(nameof dummy.sn)[0]
        let unfoldProp = t.GetMember(nameof dummy.snap)[0]
        let ux = (Query.expandExpr u.Parameters[0] param).Visit(u.Body)
        Expression.Lambda<System.Func<Item<'I>, SnAndSnap<'State>>>(
            Expression.MemberInit(
                Expression.New(t.GetConstructor [||]),
                [|  Expression.Bind(etagProp, Expression.PropertyOrField(param, nameof Unchecked.defaultof<Item<'I>>.p)) :> MemberBinding
                    Expression.Bind(unfoldProp, ux) |]),
            [| param |])

Used like this:

q.Select(Indexed.SnAndSnap<'S>.FromItemExpr(fun x -> x.u[0].d))