I'm confused about whether a network of linear activation perceptrons can model non-linear functions.
According to the book I'm reading (Mitchell), every boolean function can be represented by some network of interconnected units based on the perceptron. This means that XOR, which is non-linear, can be represented by it.
However, later in the book, the author states "multiple layers of cascaded linear units still produce only linear functions", and uses it as motivation to introduce the Sigmoid activation function.
What's going on?
I wonder if the problem is how hidden layers are defined. I understand that if the output of a first perceptron is ∑wi*xi + b, which is then the input of a second perceptron, then the resulting output is still linear, because a linear transformation of a linear transformation is still linear. But the output of a perceptron is 0/1 based on a threshold.
I found an example on the web that models XOR using a network of linear activations.
This question has been asked before, however, the answers haven't been thorough or specific.
I think what he means in the book is that a network of perceptrons with non-linear activation functions can solve XOR.
Otherwise you could collapse all the linear operations in the network in a single matrix and you would have a single perceptron capable of solving the XOR problem.