Technology
Gradients of a Dense Layer: Matrices and Jacobians
In a dense layer with z equal to Wx plus b, the Jacobian matrix of the output with respect to the input is the weight matrix W itself. From it come the two training rules: the gradient with respect to the weights is delta times x transposed, and the gradient with respect to the input is W transposed times delta.