Unity NGO: Delay with throwing things at other people

Question

I am currently making a multiplayer game in Unity with Netcode for GameObjects, and I wanted to have a feature in the game, where if you throw a object at another player, the player dies.

Currently, I check on the side of the client, who has throws the object, if the thrown object hits another player. If it does, the client sends an Server RPC with the other Player's Client ID, to tell the server, that this player should die. Then the Server sends another Client RPC to everybody, including the specific Client ID, and if that ID matches with one's own, this Player dies.

(I think, this is a pretty bad way to do this, but maybe someone can give me a better approach :) )

And now, in my recent playtests, people complain dying, when they haven't even been hit. So I investigated, and found out, that if you walk in a straight line, and another player throws an object at you, it looks like the object has been thrown right next to you, but you die anyways.

Here are some code snippets, I am currently using for my project:

("thisPlayer" is a tag, only the local Player has)

BoxScript.cs

public void OnCollisionEnter(Collision collision) {
    hitSounds[UnityEngine.Random.Range(0, hitSounds.Count)].Play();

    if (OwnerClientId == NetworkManager.Singleton.LocalClientId) {
        if (collision.transform.CompareTag("otherPlayer")) {
           killPlayerServerRpc(collision.transform.GetComponent<NetworkObject().OwnerClientId);
        }
    }
}

...

[Rpc(SendTo.Server)]
private void killPlayerServerRpc(ulong victim) {
    killPlayerClientRpc(victim);
}

[Rpc(SendTo.ClientsAndHost)]
private void killPlayerClientRpc(ulong victim) {
    if (victim == NetworkManager.Singleton.LocalClientId) {
        GameObject.FindGameObjectWithTag("thisPlayer").GetComponent<PlayerDeath>().Die();
    }
    BoxBody.velocity = new Vector3(0, 0, 0);
}

Thank you!