Billboarding Anchor's Child Entity in RealityKit

Question

I'm trying to billboard an attachment so that it remains pointed at the user wherever they're viewing the content from. I pulled the component and system from the various Apple example projects (Diorama) that have the billboarding system.

Playing around with the system and component I can add a model entity to the scene, tag it with the component and it works perfectly, all the time. As the camera moves it tracks it perfectly. Even when nested under other empty entities or off center or odd rotation.

Great! I wanted to apply this to an attachment that is shown from a model entity that is anchored to a horizontal plane and all of the sudden it doesn't work at all.

I create the anchor:

let anchor = AnchorEntity(.plane(.horizontal, classification: .table, minimumBounds: [0.01, 0.01]))

if let lookAtText = attachments.entity(for: "LookAtMe") {
    lookAtText.position = [0,0.5,0]
    lookAtText.components.set(BillboardComponent())
    lookAtText.name = "Look At Me"
    anchor.addChild(lookAtText)
}
content.add(anchor)

The attachment shows above the anchor as expected but does not rotate correctly or billboard correctly.

I don't understand why adding it as a child to the anchor entity suddenly breaks completely unrelated systems.

Am I doing something wrong or is this some sort of privacy issue? I can't find any documentation that using the look at api from an anchored entity is somehow forbidden.