SpriteKit collision detection not working as expected

Question

So I have this code inside my GameScene class (child of SKScene):

physicsWorld.contactDelegate = self
physicsWorld.gravity = CGVector(dx: 0, dy: -9.8)
addChild(balloon)
addChild(monkey)
monkey.physicsBody = SKPhysicsBody(rectangleOf: monkey.frame.size)
monkey.physicsBody?.isDynamic = false
monkey.physicsBody?.categoryBitMask = 0
monkey.physicsBody?.contactTestBitMask = 1//Balloon.categoryBitMask

balloon.physicsBody = SKPhysicsBody(rectangleOf: balloon.frame.size)
balloon.physicsBody?.isDynamic = false
balloon.physicsBody?.categoryBitMask = 1
balloon.physicsBody?.contactTestBitMask = 0
balloon.start()

I can see that the balloon and the monkey node have touched each other in the simulator, however, nothing happens. I have also conformed to the SKPhysicsContactDelegate protocol, like so:

extension GameScene: SKPhysicsContactDelegate {
    func didBegin(_ contact: SKPhysicsContact) {
        print("CONTACT!!!")
    }
}

Edit: I set isDynamic = true and it printed "CONTACT!!" as expected, however I dont want the objects to be affecting each other's position