¤Þ¤È¤áÃæ


4061529021
¤Î¼«½¬¥á¥â


½çÅÁÇÅ·¿¥Í¥Ã¥È¥ï¡¼¥¯ (feed forward neural network)

³èÀ­²½´Ø¿ô f(u) ¤Ë²¿¤ò»È¤¦¤«

²¿¤ò³Ø½¬¤¹¤ë¤Î¤«?

¸ûÇ۹߲¼Ë¡ (gradient decent method) ¤Ë¤è¤ëºÇŬ²½

²áŬ¹ç (overfitting, overlerning)

½Å¤ß¸º¿ê (weight decay)

\(E_{t}(w) = \frac{1}{N_{t}} \sum_{n \in D_{t}} E_{n}(w) + \frac{ \lambda }{2} \|w\|^{2} \\ \lambda = 0.01 〜 0.00001 \)

¸íº¹´Ø¿ô¤Ë¡¢½Å¤ß¤ÎÆó¾èϤιà¤òÄɲ乤ë
¢ª ¸íº¹´Ø¿ô¤¬¾®¤µ¤¯¤Ê¤ë¤è¤¦¤Ë³Ø½¬¤ò¿Ê¤á¤ë¤Î¤Ç¡¢¤è¤ê¾®¤µ¤Ê w ¤¬Áª¹¥¤µ¤ì¤ë

½Å¤ß¾å¸Â

\(\sum_i w_{ji}^2 < c \)

³Æ j Áؤˤª¤¤¤Æ¡¢½Å¤ß¤ÎÆó¾èϤ¬¡¢Äê¿ô c ¤è¤ê¾®¤µ¤¯¤Ê¤ë¤è¤¦¤Ë¤¹¤ë¡£

while ( square_sum(w[j]) > c ) {
  for (i = 0; i < size; i++) {
    w[j][i] = w[j][i] * 0.99;
  }
}

¥É¥í¥Ã¥×¥¢¥¦¥È (dropout)


³Ø½¬¤Î¥È¥ê¥Ã¥¯

Àµµ¬²½ (normalization, standardization)

¥Ç¡¼¥¿³ÈÄ¥ (data augmentation)

¥â¥Ç¥ëÊ¿¶Ñ (model avaraging)

³Ø½¬·¸¿ô

\(w^{(t+1)} = w^{(t)} - \varepsilon \bigtriangledown E \)

¥â¥á¥ó¥¿¥à (momentum)

¥µ¥ó¥×¥ë¤Î½ç½ø¤ò¹©Éפ¹¤ë

¸íº¹µÕÅÁÇÅË¡ (back-propagation)

[¤Ï¤¸¤á¤Ë] bias ¤Î¼è¤ê°·¤¤Êý¤Ë¤Ä¤¤¤Æ

¹Í¤¨Êý

backpropagation.png

·×»»ÊýË¡


³ÆÁؤγèÀ­²½´Ø¿ô¤Î f'(u)

½ÐÎÏÁØ¤Î¸íº¹ ¦Ä(L)

\(\frac{\partial E}{\partial w_{ji}}\) ¤Î¸¡»»ÊýË¡

¸ûÇ۾üºÌäÂê

½Å¤ß¤Î½é´üÃÍ

Íð¿ô¤Ç·èÄꤹ¤ë

¼çÀ®Ê¬Ê¬ÀÏ

¥Ç¡¼¥¿¤ÎÇò¿§²½

¼«¸ÊÉä¹æ²½

¥¹¥Ñ¡¼¥¹ÀµÂ§²½

¥Ç¥Î¥¤¥¸¥ó¥°

denoiseing.png

¾ö¹þ¤ß¥Ë¥å¡¼¥é¥ë¥Í¥Ã¥È (Convolution Neural Network)

¾ö¤ß¹þ¤ßÁØ (Convolution)

convolution_calc.png

¥×¡¼¥ê¥ó¥°ÁØ

pooling.png

Àµµ¬²½ÁØ(LCN)

¥³¥ó¥È¥é¥¹¥ÈÀµµ¬²½ÆþÎϲèÁü (1,2,...,N) ¤Î¥³¥ó¥È¥é¥¹¥È¤òÀµµ¬²½¤·¤Æ³Ø½¬¤·¤ä¤¹¤¯¤¹¤ë
¶É½ê¥³¥ó¥È¥é¥¹¥ÈÀµµ¬²½²èÁüǧ¼±¤Î¥Í¥Ã¥È¥ï¡¼¥¯¤ËÁȤ߹þ¤à¡£¾ö¹þ¤ß¤ä¥×¡¼¥ê¥ó¥°¤Çµ±ÅÙ¤¬Ë°Ï¤¹¤ë¤Î¤òËɤ°
¸º»»Àµµ¬²½(i,j)¤Î²èÁÇÃͤò(i,j)¼þÊդνŤßÉÕ¤­Ê¿¶Ñ¤Ë¤¹¤ë
½ü»»Àµµ¬²½(i,j)¤Î²èÁÇÃͤò(i,j)¼þÊդνŤßÉÕ¤­Ê¬»¶¤Ç³ä¤ë¡£ÊѲ½¤Î¾¯¤Ê¤¤²Õ½ê¤Îǻø¤ò¶¯Ä´¤¹¤ë½èÍý¤ò¹Ô¤¦

¸ûÇۤη׻»ÊýË¡

ºÆµ¢·¿¥Ë¥å¡¼¥é¥ë¥Í¥Ã¥È (RNN/LSTM/CTC)

RNN

ñ½ç¤Ê RNN ¤Î¼ÂÁõ

LSTM

a graph image


Culture, Random Memorandum 2015Q3


*1 Àµµ¬²½¸å¤ÎÎΰèP¤Î²èÁÇÃͤιç·×¤òƱ¤¸¤Ë¤¹¤ë
*2 ÊÑ´¹¤¹¤ë²èÁÇ(i,j)¤ò½Å»ë¤¹¤ë
*3 Àµµ¬²½¸å¤ÎÎΰèP¤Î²èÁÇÃͤιç·×¤òƱ¤¸¤Ë¤¹¤ë
*4 ÊÑ´¹¤¹¤ë²èÁÇ(i,j)¤ò½Å»ë¤¹¤ë

źÉÕ¥Õ¥¡¥¤¥ë: filesrnn2.png 1787·ï [¾ÜºÙ] filesrnn1.png 1800·ï [¾ÜºÙ] filernn.png 1788·ï [¾ÜºÙ] fileconvolution_delta.png 1800·ï [¾ÜºÙ] filelcn.png 1990·ï [¾ÜºÙ] filepooling.png 2210·ï [¾ÜºÙ] filergb_convolution.png 2206·ï [¾ÜºÙ] fileconvolution_calc.png 2197·ï [¾ÜºÙ] fileconvolution_network.png 2244·ï [¾ÜºÙ] fileconvolution.png 2157·ï [¾ÜºÙ] filedenoiseing.png 1992·ï [¾ÜºÙ] fileself_encode2.png 1968·ï [¾ÜºÙ] fileself_encode.png 1909·ï [¾ÜºÙ] fileprincipal_analysis.png 1945·ï [¾ÜºÙ] filefloat.png 840·ï [¾ÜºÙ] filedouble.png 885·ï [¾ÜºÙ] filebackpropagation.png 1871·ï [¾ÜºÙ] filemomentum2.png 1986·ï [¾ÜºÙ] filemomentum1.png 2009·ï [¾ÜºÙ]

¥È¥Ã¥×   ÊÔ½¸ Åà·ë º¹Ê¬ ¥Ð¥Ã¥¯¥¢¥Ã¥× źÉÕ Ê£À½ ̾Á°Êѹ¹ ¥ê¥í¡¼¥É   ¿·µ¬ °ìÍ÷ ñ¸ì¸¡º÷ ºÇ½ª¹¹¿·   ¥Ø¥ë¥×   ºÇ½ª¹¹¿·¤ÎRSS   sitemap
Last-modified: 2018-09-27 (ÌÚ) 00:45:03 (2261d)
Short-URL: http://at-sushi.com/pukiwiki/index.php?cmd=s&k=0889ad68c1
ISBN10
ISBN13
9784061426061