Convolutional NN
Foundations of CNN
matrixs convolution
Edge detection
Vertical / horizontial
conv-forward(tf.nn.cov2d)
m a t r i x ( 6 × 6 ) ∗ f i l t e r ( 3 × 3 ) = m a t r i x ( 4 × 4 ) matrix(6\times6)*filter(3\times3)=matrix(4\times4) matrix(6×6)∗filter(3×3)=matrix(4×4)
m a t r i x ( n × n ) ∗ f i l t e r ( f × f ) = m a t r i x ( ( n − f + 1 ) × ( n − f + 1 ) ) matrix(n\times n)*filter(f\times f)=matrix((n-f+1)\times (n-f+1)) matrix(n×n)∗filter(f×f)=matrix((n−f+1)×(n−f+1))
Padding
problems:
- shrinking output
- throw away infomation from edge
m a t r i x ( ( n + 2 p ) × ( n + 2 p ) ) ∗ f i l t e r ( f × f ) = m a t r i x ( ( n + 2 p − f + 1 ) × ( n + 2 p − f + 1 ) matrix((n+2p)\times (n+2p))*filter(f\times f)=matrix((n+2p-f+1)\times (n+2p-f+1) matrix((n+2p)×(n+2p))∗filter(f×f)=matrix((n+2p−f+1)×(n+2p−f+1)
valid convolution / same convolution
“same” : p = f − 1 2 ( o d d f ) p = \frac{f-1}{2} (odd\;f) p=2f−1(oddf)
Stride
m a t r i x ( ( n + 2 p ) × ( n + 2 p ) ) ∗ f i l t e r ( f × f ) = m a t r i x ( ( ( n + 2 p − f ) s + 1 ) × ( ( n + 2 p − f ) s + 1 ) matrix((n+2p)\times (n+2p))*filter(f\times f)=matrix((\frac{(n+2p-f)}{s}+1)\times (\frac{(n+2p-f)}{s}+1) matrix((n+2p)×(n+2p))∗filter(f×f)=matrix((s(n+2p−f)+1)×(s(n+2p−f)+1)
Volume convolution
m a t r i x ( n × n × n c ) ∗ f i l t e r ( f × f × n c ) = m a t r i x ( ( n − f + 1 ) × ( n − f + 1 ) × n c ) matrix(n\times n\times n_c)*filter(f\times f\times n_c)=matrix((n-f+1)\times (n-f+1)\times n_c) matrix(n×n×nc)∗filter(f×f×nc)=matrix((n−f+1)×(n−f+1)×nc)
