自適應(yīng)參數(shù)化ReLU，是一種動(dòng)態(tài)的ReLU（Dynamic ReLU），在2019年5月投稿至IEEE Trans. on Industrial Electronics，2020年1月錄用，2020年2月在IEEE網(wǎng)站發(fā)布預(yù)印本。

本文在調(diào)參記錄25的基礎(chǔ)上，將自適應(yīng)參數(shù)化ReLU中間層的神經(jīng)元個(gè)數(shù)，從2個(gè)增加到4個(gè)，同時(shí)添加了一個(gè)Dropout層，繼續(xù)測(cè)試其在Cifar10數(shù)據(jù)集上的效果。

自適應(yīng)參數(shù)化ReLU的基本原理：

Keras程序：

#!/usr/bin/env python3

實(shí)驗(yàn)結(jié)果：

Using TensorFlow backend. x_train shape: (50000, 32, 32, 3) 50000 train samples 10000 test samples Epoch 1/1000 420s 840ms/step - loss: 6.8132 - acc: 0.3731 - val_loss: 6.0072 - val_acc: 0.4935 Epoch 2/1000 333s 667ms/step - loss: 5.5637 - acc: 0.5070 - val_loss: 4.9312 - val_acc: 0.5947 Epoch 3/1000 333s 667ms/step - loss: 4.6506 - acc: 0.5734 - val_loss: 4.1140 - val_acc: 0.6551 Epoch 4/1000 333s 667ms/step - loss: 3.9385 - acc: 0.6189 - val_loss: 3.4676 - val_acc: 0.6973 Epoch 5/1000 333s 667ms/step - loss: 3.3667 - acc: 0.6524 - val_loss: 2.9511 - val_acc: 0.7314 Epoch 6/1000 333s 667ms/step - loss: 2.8977 - acc: 0.6816 - val_loss: 2.5364 - val_acc: 0.7556 Epoch 7/1000 334s 667ms/step - loss: 2.5121 - acc: 0.7062 - val_loss: 2.1815 - val_acc: 0.7762 Epoch 8/1000 334s 667ms/step - loss: 2.2071 - acc: 0.7228 - val_loss: 1.8975 - val_acc: 0.7940 Epoch 9/1000 333s 667ms/step - loss: 1.9563 - acc: 0.7382 - val_loss: 1.6740 - val_acc: 0.8045 Epoch 10/1000 333s 667ms/step - loss: 1.7436 - acc: 0.7521 - val_loss: 1.4899 - val_acc: 0.8178 Epoch 11/1000 333s 667ms/step - loss: 1.5743 - acc: 0.7647 - val_loss: 1.3368 - val_acc: 0.8307 Epoch 12/1000 333s 667ms/step - loss: 1.4340 - acc: 0.7721 - val_loss: 1.2230 - val_acc: 0.8355 Epoch 13/1000 334s 667ms/step - loss: 1.3226 - acc: 0.7815 - val_loss: 1.1185 - val_acc: 0.8426 Epoch 14/1000 334s 667ms/step - loss: 1.2300 - acc: 0.7885 - val_loss: 1.0387 - val_acc: 0.8458 Epoch 15/1000 333s 667ms/step - loss: 1.1585 - acc: 0.7961 - val_loss: 1.0022 - val_acc: 0.8457 Epoch 16/1000 334s 667ms/step - loss: 1.0832 - acc: 0.8069 - val_loss: 0.9392 - val_acc: 0.8491 Epoch 17/1000 333s 667ms/step - loss: 1.0364 - acc: 0.8119 - val_loss: 0.8801 - val_acc: 0.8604 Epoch 18/1000 333s 667ms/step - loss: 0.9910 - acc: 0.8164 - val_loss: 0.8453 - val_acc: 0.8661 Epoch 19/1000 333s 666ms/step - loss: 0.9597 - acc: 0.8208 - val_loss: 0.8281 - val_acc: 0.8665 Epoch 20/1000 333s 666ms/step - loss: 0.9316 - acc: 0.8248 - val_loss: 0.7990 - val_acc: 0.8686 Epoch 21/1000 333s 666ms/step - loss: 0.9131 - acc: 0.8259 - val_loss: 0.7708 - val_acc: 0.8773 Epoch 22/1000 333s 666ms/step - loss: 0.8881 - acc: 0.8341 - val_loss: 0.7715 - val_acc: 0.8744 Epoch 23/1000 333s 666ms/step - loss: 0.8734 - acc: 0.8352 - val_loss: 0.7602 - val_acc: 0.8800 Epoch 24/1000 333s 667ms/step - loss: 0.8614 - acc: 0.8374 - val_loss: 0.7653 - val_acc: 0.8762 Epoch 25/1000 333s 666ms/step - loss: 0.8512 - acc: 0.8415 - val_loss: 0.7291 - val_acc: 0.8862 Epoch 26/1000 333s 666ms/step - loss: 0.8386 - acc: 0.8446 - val_loss: 0.7418 - val_acc: 0.8838 Epoch 27/1000 333s 666ms/step - loss: 0.8346 - acc: 0.8471 - val_loss: 0.7217 - val_acc: 0.8891 Epoch 28/1000 334s 668ms/step - loss: 0.8243 - acc: 0.8515 - val_loss: 0.7233 - val_acc: 0.8860 Epoch 29/1000 335s 670ms/step - loss: 0.8208 - acc: 0.8516 - val_loss: 0.7140 - val_acc: 0.8889 Epoch 30/1000 336s 671ms/step - loss: 0.8181 - acc: 0.8540 - val_loss: 0.7033 - val_acc: 0.8956 Epoch 31/1000 335s 669ms/step - loss: 0.8054 - acc: 0.8546 - val_loss: 0.7127 - val_acc: 0.8903 Epoch 32/1000 336s 672ms/step - loss: 0.8017 - acc: 0.8582 - val_loss: 0.7283 - val_acc: 0.8879 Epoch 33/1000 334s 669ms/step - loss: 0.8039 - acc: 0.8604 - val_loss: 0.7117 - val_acc: 0.8904 Epoch 34/1000 332s 663ms/step - loss: 0.8007 - acc: 0.8604 - val_loss: 0.7053 - val_acc: 0.8927 Epoch 35/1000 334s 668ms/step - loss: 0.7940 - acc: 0.8636 - val_loss: 0.7236 - val_acc: 0.8891 Epoch 36/1000 332s 663ms/step - loss: 0.7910 - acc: 0.8634 - val_loss: 0.6960 - val_acc: 0.8989 Epoch 37/1000 334s 669ms/step - loss: 0.7845 - acc: 0.8663 - val_loss: 0.7109 - val_acc: 0.8913 Epoch 38/1000 334s 668ms/step - loss: 0.7870 - acc: 0.8655 - val_loss: 0.7001 - val_acc: 0.8968 Epoch 39/1000 335s 670ms/step - loss: 0.7830 - acc: 0.8680 - val_loss: 0.7081 - val_acc: 0.8951 Epoch 40/1000 335s 670ms/step - loss: 0.7846 - acc: 0.8704 - val_loss: 0.6885 - val_acc: 0.9064 Epoch 41/1000 332s 664ms/step - loss: 0.7823 - acc: 0.8703 - val_loss: 0.6980 - val_acc: 0.9037 Epoch 42/1000 333s 667ms/step - loss: 0.7770 - acc: 0.8712 - val_loss: 0.7001 - val_acc: 0.9009 Epoch 43/1000 333s 665ms/step - loss: 0.7745 - acc: 0.8725 - val_loss: 0.6988 - val_acc: 0.9041 Epoch 44/1000 335s 671ms/step - loss: 0.7789 - acc: 0.8717 - val_loss: 0.6864 - val_acc: 0.9074 Epoch 45/1000 335s 670ms/step - loss: 0.7716 - acc: 0.8760 - val_loss: 0.6904 - val_acc: 0.9069 Epoch 46/1000 334s 669ms/step - loss: 0.7711 - acc: 0.8753 - val_loss: 0.6904 - val_acc: 0.9040 Epoch 47/1000 332s 665ms/step - loss: 0.7680 - acc: 0.8788 - val_loss: 0.7124 - val_acc: 0.9002 Epoch 48/1000 335s 670ms/step - loss: 0.7685 - acc: 0.8784 - val_loss: 0.6935 - val_acc: 0.9093 Epoch 49/1000 335s 670ms/step - loss: 0.7673 - acc: 0.8788 - val_loss: 0.7104 - val_acc: 0.8989 Epoch 50/1000 335s 670ms/step - loss: 0.7675 - acc: 0.8774 - val_loss: 0.7000 - val_acc: 0.9055 Epoch 51/1000 335s 670ms/step - loss: 0.7629 - acc: 0.8819 - val_loss: 0.7205 - val_acc: 0.8998 Epoch 52/1000 333s 666ms/step - loss: 0.7658 - acc: 0.8803 - val_loss: 0.6880 - val_acc: 0.9074 Epoch 53/1000 335s 670ms/step - loss: 0.7699 - acc: 0.8792 - val_loss: 0.7064 - val_acc: 0.9058 Epoch 54/1000 333s 665ms/step - loss: 0.7619 - acc: 0.8832 - val_loss: 0.7095 - val_acc: 0.9044 Epoch 55/1000 335s 670ms/step - loss: 0.7662 - acc: 0.8807 - val_loss: 0.6995 - val_acc: 0.9083 Epoch 56/1000 335s 671ms/step - loss: 0.7618 - acc: 0.8826 - val_loss: 0.7061 - val_acc: 0.9056 Epoch 57/1000 333s 665ms/step - loss: 0.7647 - acc: 0.8826 - val_loss: 0.6994 - val_acc: 0.9077 Epoch 58/1000 334s 668ms/step - loss: 0.7628 - acc: 0.8817 - val_loss: 0.6965 - val_acc: 0.9124 Epoch 59/1000 335s 670ms/step - loss: 0.7628 - acc: 0.8835 - val_loss: 0.7090 - val_acc: 0.9020 Epoch 60/1000 335s 669ms/step - loss: 0.7594 - acc: 0.8835 - val_loss: 0.7074 - val_acc: 0.9051 Epoch 61/1000 335s 669ms/step - loss: 0.7555 - acc: 0.8861 - val_loss: 0.7233 - val_acc: 0.8998 Epoch 62/1000 332s 665ms/step - loss: 0.7617 - acc: 0.8850 - val_loss: 0.6922 - val_acc: 0.9105 Epoch 63/1000 335s 670ms/step - loss: 0.7543 - acc: 0.8872 - val_loss: 0.7026 - val_acc: 0.9055 Epoch 64/1000 335s 671ms/step - loss: 0.7562 - acc: 0.8880 - val_loss: 0.6993 - val_acc: 0.9092 Epoch 65/1000 333s 665ms/step - loss: 0.7572 - acc: 0.8868 - val_loss: 0.6902 - val_acc: 0.9107 Epoch 66/1000 335s 670ms/step - loss: 0.7576 - acc: 0.8858 - val_loss: 0.6948 - val_acc: 0.9098 Epoch 67/1000 335s 670ms/step - loss: 0.7573 - acc: 0.8875 - val_loss: 0.7264 - val_acc: 0.9005 Epoch 68/1000 335s 670ms/step - loss: 0.7501 - acc: 0.8893 - val_loss: 0.7128 - val_acc: 0.9069 Epoch 69/1000 334s 669ms/step - loss: 0.7503 - acc: 0.8908 - val_loss: 0.7080 - val_acc: 0.9072 Epoch 70/1000 335s 670ms/step - loss: 0.7567 - acc: 0.8882 - val_loss: 0.7090 - val_acc: 0.9081 ... Epoch 292/1000 334s 668ms/step - loss: 0.7486 - acc: 0.9087 - val_loss: 0.7229 - val_acc: 0.9221 Epoch 293/1000 334s 668ms/step - loss: 0.7537 - acc: 0.9070 - val_loss: 0.7409 - val_acc: 0.9157 Epoch 294/1000 334s 667ms/step - loss: 0.7549 - acc: 0.9068 - val_loss: 0.7365 - val_acc: 0.9167 Epoch 295/1000 332s 664ms/step - loss: 0.7520 - acc: 0.9078 - val_loss: 0.7265 - val_acc: 0.9186 Epoch 296/1000 332s 663ms/step - loss: 0.7542 - acc: 0.9078 - val_loss: 0.7219 - val_acc: 0.9187 Epoch 297/1000 336s 671ms/step - loss: 0.7538 - acc: 0.9076 - val_loss: 0.7297 - val_acc: 0.9201 Epoch 298/1000 333s 665ms/step - loss: 0.7492 - acc: 0.9099 - val_loss: 0.7118 - val_acc: 0.9251 Epoch 299/1000 334s 669ms/step - loss: 0.7511 - acc: 0.9088 - val_loss: 0.7181 - val_acc: 0.9232 Epoch 300/1000 334s 667ms/step - loss: 0.7525 - acc: 0.9084 - val_loss: 0.7298 - val_acc: 0.9202 Epoch 301/1000 lr changed to 0.010000000149011612 334s 668ms/step - loss: 0.6468 - acc: 0.9450 - val_loss: 0.6315 - val_acc: 0.9491 Epoch 302/1000 335s 671ms/step - loss: 0.5879 - acc: 0.9639 - val_loss: 0.6157 - val_acc: 0.9529 Epoch 303/1000 333s 665ms/step - loss: 0.5643 - acc: 0.9690 - val_loss: 0.6061 - val_acc: 0.9528 Epoch 304/1000 335s 670ms/step - loss: 0.5450 - acc: 0.9722 - val_loss: 0.5968 - val_acc: 0.9525 Epoch 305/1000 336s 672ms/step - loss: 0.5290 - acc: 0.9744 - val_loss: 0.5860 - val_acc: 0.9559 Epoch 306/1000 336s 672ms/step - loss: 0.5164 - acc: 0.9766 - val_loss: 0.5746 - val_acc: 0.9556 Epoch 307/1000 334s 667ms/step - loss: 0.5026 - acc: 0.9779 - val_loss: 0.5686 - val_acc: 0.9548 Epoch 308/1000 332s 664ms/step - loss: 0.4952 - acc: 0.9779 - val_loss: 0.5672 - val_acc: 0.9566 Epoch 309/1000 332s 664ms/step - loss: 0.4826 - acc: 0.9799 - val_loss: 0.5587 - val_acc: 0.9563 Epoch 310/1000 332s 664ms/step - loss: 0.4717 - acc: 0.9812 - val_loss: 0.5531 - val_acc: 0.9546 Epoch 311/1000 332s 664ms/step - loss: 0.4622 - acc: 0.9825 - val_loss: 0.5421 - val_acc: 0.9564 Epoch 312/1000 334s 668ms/step - loss: 0.4557 - acc: 0.9813 - val_loss: 0.5400 - val_acc: 0.9541 Epoch 313/1000 333s 667ms/step - loss: 0.4473 - acc: 0.9829 - val_loss: 0.5319 - val_acc: 0.9572 Epoch 314/1000 333s 666ms/step - loss: 0.4363 - acc: 0.9848 - val_loss: 0.5311 - val_acc: 0.9553 Epoch 315/1000 333s 665ms/step - loss: 0.4285 - acc: 0.9844 - val_loss: 0.5253 - val_acc: 0.9545 Epoch 316/1000 334s 668ms/step - loss: 0.4214 - acc: 0.9851 - val_loss: 0.5162 - val_acc: 0.9572 Epoch 317/1000 332s 665ms/step - loss: 0.4124 - acc: 0.9860 - val_loss: 0.5116 - val_acc: 0.9553 Epoch 318/1000 333s 666ms/step - loss: 0.4077 - acc: 0.9858 - val_loss: 0.5074 - val_acc: 0.9548 Epoch 319/1000 335s 671ms/step - loss: 0.3978 - acc: 0.9875 - val_loss: 0.5031 - val_acc: 0.9539 Epoch 320/1000 334s 669ms/step - loss: 0.3944 - acc: 0.9857 - val_loss: 0.4901 - val_acc: 0.9569 Epoch 321/1000 332s 663ms/step - loss: 0.3840 - acc: 0.9883 - val_loss: 0.4926 - val_acc: 0.9554 Epoch 322/1000 333s 667ms/step - loss: 0.3826 - acc: 0.9861 - val_loss: 0.4884 - val_acc: 0.9580 Epoch 323/1000 332s 664ms/step - loss: 0.3751 - acc: 0.9871 - val_loss: 0.4797 - val_acc: 0.9560 Epoch 324/1000 332s 665ms/step - loss: 0.3695 - acc: 0.9878 - val_loss: 0.4787 - val_acc: 0.9554 Epoch 325/1000 333s 665ms/step - loss: 0.3639 - acc: 0.9876 - val_loss: 0.4677 - val_acc: 0.9575 Epoch 326/1000 331s 662ms/step - loss: 0.3563 - acc: 0.9885 - val_loss: 0.4603 - val_acc: 0.9573 Epoch 327/1000 334s 667ms/step - loss: 0.3514 - acc: 0.9886 - val_loss: 0.4573 - val_acc: 0.9569 Epoch 328/1000 333s 667ms/step - loss: 0.3465 - acc: 0.9888 - val_loss: 0.4584 - val_acc: 0.9561 Epoch 329/1000 331s 661ms/step - loss: 0.3449 - acc: 0.9878 - val_loss: 0.4575 - val_acc: 0.9556 Epoch 330/1000 332s 665ms/step - loss: 0.3371 - acc: 0.9889 - val_loss: 0.4506 - val_acc: 0.9575 Epoch 331/1000 333s 665ms/step - loss: 0.3314 - acc: 0.9888 - val_loss: 0.4413 - val_acc: 0.9565 Epoch 332/1000 332s 664ms/step - loss: 0.3275 - acc: 0.9891 - val_loss: 0.4430 - val_acc: 0.9553 Epoch 333/1000 332s 665ms/step - loss: 0.3246 - acc: 0.9884 - val_loss: 0.4350 - val_acc: 0.9560 Epoch 334/1000 334s 667ms/step - loss: 0.3180 - acc: 0.9893 - val_loss: 0.4385 - val_acc: 0.9547 Epoch 335/1000 333s 667ms/step - loss: 0.3179 - acc: 0.9880 - val_loss: 0.4342 - val_acc: 0.9556 Epoch 336/1000 332s 665ms/step - loss: 0.3107 - acc: 0.9897 - val_loss: 0.4227 - val_acc: 0.9578 Epoch 337/1000 332s 664ms/step - loss: 0.3082 - acc: 0.9886 - val_loss: 0.4264 - val_acc: 0.9553 Epoch 338/1000 334s 668ms/step - loss: 0.3031 - acc: 0.9895 - val_loss: 0.4251 - val_acc: 0.9563 Epoch 339/1000 332s 664ms/step - loss: 0.2994 - acc: 0.9899 - val_loss: 0.4240 - val_acc: 0.9542 Epoch 340/1000 334s 668ms/step - loss: 0.2966 - acc: 0.9895 - val_loss: 0.4186 - val_acc: 0.9547 Epoch 341/1000 332s 664ms/step - loss: 0.2945 - acc: 0.9891 - val_loss: 0.4112 - val_acc: 0.9548 Epoch 342/1000 334s 668ms/step - loss: 0.2888 - acc: 0.9904 - val_loss: 0.4151 - val_acc: 0.9532 Epoch 343/1000 335s 670ms/step - loss: 0.2866 - acc: 0.9895 - val_loss: 0.4121 - val_acc: 0.9556 Epoch 344/1000 334s 667ms/step - loss: 0.2835 - acc: 0.9896 - val_loss: 0.4018 - val_acc: 0.9561 Epoch 345/1000 334s 669ms/step - loss: 0.2811 - acc: 0.9892 - val_loss: 0.4020 - val_acc: 0.9547 Epoch 346/1000 332s 665ms/step - loss: 0.2768 - acc: 0.9897 - val_loss: 0.4021 - val_acc: 0.9536 Epoch 347/1000 334s 668ms/step - loss: 0.2755 - acc: 0.9888 - val_loss: 0.4000 - val_acc: 0.9536 Epoch 348/1000 335s 670ms/step - loss: 0.2729 - acc: 0.9889 - val_loss: 0.3904 - val_acc: 0.9571 Epoch 349/1000 333s 665ms/step - loss: 0.2679 - acc: 0.9905 - val_loss: 0.3914 - val_acc: 0.9542 Epoch 350/1000 334s 667ms/step - loss: 0.2672 - acc: 0.9892 - val_loss: 0.3913 - val_acc: 0.9564 Epoch 351/1000 334s 667ms/step - loss: 0.2647 - acc: 0.9890 - val_loss: 0.3941 - val_acc: 0.9546 Epoch 352/1000 332s 664ms/step - loss: 0.2624 - acc: 0.9893 - val_loss: 0.3894 - val_acc: 0.9529 Epoch 353/1000 332s 664ms/step - loss: 0.2581 - acc: 0.9901 - val_loss: 0.3924 - val_acc: 0.9522 Epoch 354/1000 334s 668ms/step - loss: 0.2564 - acc: 0.9897 - val_loss: 0.3930 - val_acc: 0.9518 Epoch 355/1000 333s 666ms/step - loss: 0.2560 - acc: 0.9898 - val_loss: 0.3842 - val_acc: 0.9536 Epoch 356/1000 331s 663ms/step - loss: 0.2569 - acc: 0.9885 - val_loss: 0.3790 - val_acc: 0.9524 Epoch 357/1000 333s 666ms/step - loss: 0.2509 - acc: 0.9895 - val_loss: 0.3791 - val_acc: 0.9548 Epoch 358/1000 333s 667ms/step - loss: 0.2499 - acc: 0.9895 - val_loss: 0.3759 - val_acc: 0.9552 Epoch 359/1000 333s 666ms/step - loss: 0.2519 - acc: 0.9876 - val_loss: 0.3735 - val_acc: 0.9541 Epoch 360/1000 333s 666ms/step - loss: 0.2476 - acc: 0.9884 - val_loss: 0.3767 - val_acc: 0.9516 Epoch 361/1000 331s 661ms/step - loss: 0.2478 - acc: 0.9877 - val_loss: 0.3617 - val_acc: 0.9554 Epoch 362/1000 331s 662ms/step - loss: 0.2410 - acc: 0.9899 - val_loss: 0.3677 - val_acc: 0.9530 Epoch 363/1000 333s 666ms/step - loss: 0.2420 - acc: 0.9887 - val_loss: 0.3678 - val_acc: 0.9524 Epoch 364/1000 334s 669ms/step - loss: 0.2412 - acc: 0.9885 - val_loss: 0.3599 - val_acc: 0.9552 Epoch 365/1000 333s 666ms/step - loss: 0.2382 - acc: 0.9887 - val_loss: 0.3655 - val_acc: 0.9536 Epoch 366/1000 332s 664ms/step - loss: 0.2359 - acc: 0.9892 - val_loss: 0.3612 - val_acc: 0.9543 Epoch 367/1000 332s 663ms/step - loss: 0.2349 - acc: 0.9891 - val_loss: 0.3675 - val_acc: 0.9526 Epoch 368/1000 334s 668ms/step - loss: 0.2315 - acc: 0.9899 - val_loss: 0.3626 - val_acc: 0.9516 Epoch 369/1000 335s 670ms/step - loss: 0.2295 - acc: 0.9902 - val_loss: 0.3695 - val_acc: 0.9526 Epoch 370/1000 333s 667ms/step - loss: 0.2323 - acc: 0.9880 - val_loss: 0.3635 - val_acc: 0.9530 Epoch 371/1000 337s 673ms/step - loss: 0.2310 - acc: 0.9890 - val_loss: 0.3666 - val_acc: 0.9531 Epoch 372/1000 336s 672ms/step - loss: 0.2307 - acc: 0.9881 - val_loss: 0.3628 - val_acc: 0.9515 Epoch 373/1000 331s 663ms/step - loss: 0.2324 - acc: 0.9865 - val_loss: 0.3545 - val_acc: 0.9551 Epoch 374/1000 332s 663ms/step - loss: 0.2304 - acc: 0.9879 - val_loss: 0.3614 - val_acc: 0.9524 Epoch 375/1000 337s 673ms/step - loss: 0.2280 - acc: 0.9876 - val_loss: 0.3659 - val_acc: 0.9521 Epoch 376/1000 332s 664ms/step - loss: 0.2246 - acc: 0.9884 - val_loss: 0.3631 - val_acc: 0.9504 Epoch 377/1000 332s 664ms/step - loss: 0.2285 - acc: 0.9870 - val_loss: 0.3560 - val_acc: 0.9500 ... Epoch 577/1000 335s 670ms/step - loss: 0.1963 - acc: 0.9889 - val_loss: 0.3402 - val_acc: 0.9501 Epoch 578/1000 334s 669ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3376 - val_acc: 0.9499 Epoch 579/1000 335s 669ms/step - loss: 0.2001 - acc: 0.9875 - val_loss: 0.3440 - val_acc: 0.9472 Epoch 580/1000 334s 668ms/step - loss: 0.1990 - acc: 0.9884 - val_loss: 0.3421 - val_acc: 0.9494 Epoch 581/1000 333s 667ms/step - loss: 0.2014 - acc: 0.9869 - val_loss: 0.3274 - val_acc: 0.9535 Epoch 582/1000 334s 668ms/step - loss: 0.2016 - acc: 0.9871 - val_loss: 0.3455 - val_acc: 0.9487 Epoch 583/1000 334s 668ms/step - loss: 0.2005 - acc: 0.9877 - val_loss: 0.3417 - val_acc: 0.9483 Epoch 584/1000 332s 664ms/step - loss: 0.1984 - acc: 0.9877 - val_loss: 0.3477 - val_acc: 0.9466 Epoch 585/1000 335s 669ms/step - loss: 0.2025 - acc: 0.9867 - val_loss: 0.3508 - val_acc: 0.9441 Epoch 586/1000 334s 669ms/step - loss: 0.1995 - acc: 0.9879 - val_loss: 0.3432 - val_acc: 0.9480 Epoch 587/1000 334s 668ms/step - loss: 0.1969 - acc: 0.9888 - val_loss: 0.3486 - val_acc: 0.9476 Epoch 588/1000 332s 663ms/step - loss: 0.2015 - acc: 0.9871 - val_loss: 0.3385 - val_acc: 0.9488 Epoch 589/1000 332s 663ms/step - loss: 0.1994 - acc: 0.9877 - val_loss: 0.3444 - val_acc: 0.9456 Epoch 590/1000 334s 668ms/step - loss: 0.2010 - acc: 0.9878 - val_loss: 0.3489 - val_acc: 0.9457 Epoch 591/1000 334s 668ms/step - loss: 0.2010 - acc: 0.9878 - val_loss: 0.3371 - val_acc: 0.9497 Epoch 592/1000 332s 664ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3478 - val_acc: 0.9455 Epoch 593/1000 334s 669ms/step - loss: 0.1993 - acc: 0.9875 - val_loss: 0.3397 - val_acc: 0.9492 Epoch 594/1000 332s 663ms/step - loss: 0.2002 - acc: 0.9877 - val_loss: 0.3415 - val_acc: 0.9504 Epoch 595/1000 332s 663ms/step - loss: 0.1973 - acc: 0.9885 - val_loss: 0.3457 - val_acc: 0.9482 Epoch 596/1000 334s 667ms/step - loss: 0.2007 - acc: 0.9877 - val_loss: 0.3558 - val_acc: 0.9450 Epoch 597/1000 334s 667ms/step - loss: 0.1961 - acc: 0.9890 - val_loss: 0.3313 - val_acc: 0.9523 Epoch 598/1000 334s 668ms/step - loss: 0.1961 - acc: 0.9889 - val_loss: 0.3423 - val_acc: 0.9487 Epoch 599/1000 332s 664ms/step - loss: 0.1957 - acc: 0.9887 - val_loss: 0.3471 - val_acc: 0.9463 Epoch 600/1000 332s 664ms/step - loss: 0.1979 - acc: 0.9886 - val_loss: 0.3447 - val_acc: 0.9484 Epoch 601/1000 lr changed to 0.0009999999776482583 334s 668ms/step - loss: 0.1872 - acc: 0.9920 - val_loss: 0.3180 - val_acc: 0.9540 Epoch 602/1000 332s 663ms/step - loss: 0.1760 - acc: 0.9955 - val_loss: 0.3097 - val_acc: 0.9569 Epoch 603/1000 332s 663ms/step - loss: 0.1743 - acc: 0.9963 - val_loss: 0.3071 - val_acc: 0.9586 Epoch 604/1000 333s 667ms/step - loss: 0.1715 - acc: 0.9969 - val_loss: 0.3063 - val_acc: 0.9585 Epoch 605/1000 331s 663ms/step - loss: 0.1706 - acc: 0.9973 - val_loss: 0.3042 - val_acc: 0.9592 Epoch 606/1000 331s 663ms/step - loss: 0.1700 - acc: 0.9974 - val_loss: 0.3082 - val_acc: 0.9582 Epoch 607/1000 331s 663ms/step - loss: 0.1689 - acc: 0.9977 - val_loss: 0.3073 - val_acc: 0.9596 Epoch 608/1000 332s 664ms/step - loss: 0.1679 - acc: 0.9976 - val_loss: 0.3063 - val_acc: 0.9584 Epoch 609/1000 335s 670ms/step - loss: 0.1671 - acc: 0.9980 - val_loss: 0.3053 - val_acc: 0.9597 Epoch 610/1000 332s 663ms/step - loss: 0.1669 - acc: 0.9981 - val_loss: 0.3050 - val_acc: 0.9595 Epoch 611/1000 332s 664ms/step - loss: 0.1660 - acc: 0.9982 - val_loss: 0.3054 - val_acc: 0.9599 Epoch 612/1000 334s 668ms/step - loss: 0.1646 - acc: 0.9984 - val_loss: 0.3064 - val_acc: 0.9591 Epoch 613/1000 333s 666ms/step - loss: 0.1655 - acc: 0.9983 - val_loss: 0.3022 - val_acc: 0.9598 Epoch 614/1000 334s 668ms/step - loss: 0.1639 - acc: 0.9987 - val_loss: 0.3009 - val_acc: 0.9603 Epoch 615/1000 334s 667ms/step - loss: 0.1640 - acc: 0.9983 - val_loss: 0.3035 - val_acc: 0.9594 Epoch 616/1000 334s 669ms/step - loss: 0.1635 - acc: 0.9985 - val_loss: 0.3031 - val_acc: 0.9606 Epoch 617/1000 335s 669ms/step - loss: 0.1635 - acc: 0.9983 - val_loss: 0.3056 - val_acc: 0.9596 Epoch 618/1000 333s 666ms/step - loss: 0.1629 - acc: 0.9986 - val_loss: 0.3040 - val_acc: 0.9593 Epoch 619/1000 334s 667ms/step - loss: 0.1622 - acc: 0.9987 - val_loss: 0.3030 - val_acc: 0.9605 Epoch 620/1000 333s 666ms/step - loss: 0.1621 - acc: 0.9985 - val_loss: 0.3035 - val_acc: 0.9598 Epoch 621/1000 332s 664ms/step - loss: 0.1617 - acc: 0.9986 - val_loss: 0.3034 - val_acc: 0.9611 Epoch 622/1000 334s 668ms/step - loss: 0.1613 - acc: 0.9986 - val_loss: 0.3025 - val_acc: 0.9612 Epoch 623/1000 334s 668ms/step - loss: 0.1610 - acc: 0.9986 - val_loss: 0.3012 - val_acc: 0.9610 Epoch 624/1000 332s 664ms/step - loss: 0.1601 - acc: 0.9988 - val_loss: 0.3005 - val_acc: 0.9621 Epoch 625/1000 332s 664ms/step - loss: 0.1609 - acc: 0.9984 - val_loss: 0.3018 - val_acc: 0.9606 Epoch 626/1000 334s 667ms/step - loss: 0.1593 - acc: 0.9990 - val_loss: 0.3002 - val_acc: 0.9607 Epoch 627/1000 332s 663ms/step - loss: 0.1592 - acc: 0.9989 - val_loss: 0.3015 - val_acc: 0.9611 Epoch 628/1000 331s 663ms/step - loss: 0.1591 - acc: 0.9987 - val_loss: 0.2975 - val_acc: 0.9616 Epoch 629/1000 332s 663ms/step - loss: 0.1583 - acc: 0.9990 - val_loss: 0.2989 - val_acc: 0.9618 Epoch 630/1000 334s 667ms/step - loss: 0.1579 - acc: 0.9990 - val_loss: 0.2999 - val_acc: 0.9610 Epoch 631/1000 332s 663ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.3003 - val_acc: 0.9598 Epoch 632/1000 334s 669ms/step - loss: 0.1579 - acc: 0.9988 - val_loss: 0.2996 - val_acc: 0.9611 Epoch 633/1000 332s 664ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.2974 - val_acc: 0.9614 Epoch 634/1000 334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3010 - val_acc: 0.9612 Epoch 635/1000 332s 664ms/step - loss: 0.1565 - acc: 0.9990 - val_loss: 0.3000 - val_acc: 0.9608 Epoch 636/1000 334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3001 - val_acc: 0.9606 Epoch 637/1000 333s 665ms/step - loss: 0.1558 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9612 Epoch 638/1000 334s 668ms/step - loss: 0.1559 - acc: 0.9989 - val_loss: 0.2975 - val_acc: 0.9605 Epoch 639/1000 334s 668ms/step - loss: 0.1550 - acc: 0.9991 - val_loss: 0.2985 - val_acc: 0.9618 Epoch 640/1000 335s 670ms/step - loss: 0.1550 - acc: 0.9992 - val_loss: 0.2998 - val_acc: 0.9613 Epoch 641/1000 334s 668ms/step - loss: 0.1546 - acc: 0.9991 - val_loss: 0.2982 - val_acc: 0.9609 Epoch 642/1000 332s 663ms/step - loss: 0.1543 - acc: 0.9992 - val_loss: 0.2988 - val_acc: 0.9606 Epoch 643/1000 333s 666ms/step - loss: 0.1537 - acc: 0.9992 - val_loss: 0.2999 - val_acc: 0.9604 Epoch 644/1000 331s 663ms/step - loss: 0.1536 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9611 Epoch 645/1000 333s 666ms/step - loss: 0.1535 - acc: 0.9990 - val_loss: 0.2981 - val_acc: 0.9606 Epoch 646/1000 331s 663ms/step - loss: 0.1530 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9619 Epoch 647/1000 334s 668ms/step - loss: 0.1527 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9612 Epoch 648/1000 334s 669ms/step - loss: 0.1529 - acc: 0.9989 - val_loss: 0.2992 - val_acc: 0.9607 Epoch 649/1000 335s 670ms/step - loss: 0.1527 - acc: 0.9991 - val_loss: 0.2995 - val_acc: 0.9603 Epoch 650/1000 333s 666ms/step - loss: 0.1522 - acc: 0.9991 - val_loss: 0.2980 - val_acc: 0.9594 Epoch 651/1000 334s 669ms/step - loss: 0.1517 - acc: 0.9991 - val_loss: 0.2965 - val_acc: 0.9597 Epoch 652/1000 333s 667ms/step - loss: 0.1518 - acc: 0.9992 - val_loss: 0.2968 - val_acc: 0.9610 Epoch 653/1000 332s 663ms/step - loss: 0.1510 - acc: 0.9992 - val_loss: 0.2958 - val_acc: 0.9600 Epoch 654/1000 334s 667ms/step - loss: 0.1510 - acc: 0.9991 - val_loss: 0.2962 - val_acc: 0.9605 Epoch 655/1000 332s 664ms/step - loss: 0.1507 - acc: 0.9991 - val_loss: 0.2951 - val_acc: 0.9609 Epoch 656/1000 335s 669ms/step - loss: 0.1502 - acc: 0.9991 - val_loss: 0.2944 - val_acc: 0.9601 Epoch 657/1000 333s 667ms/step - loss: 0.1503 - acc: 0.9990 - val_loss: 0.2938 - val_acc: 0.9610 Epoch 658/1000 332s 664ms/step - loss: 0.1495 - acc: 0.9993 - val_loss: 0.2957 - val_acc: 0.9613 Epoch 659/1000 333s 667ms/step - loss: 0.1501 - acc: 0.9990 - val_loss: 0.2962 - val_acc: 0.9603 Epoch 660/1000 332s 664ms/step - loss: 0.1492 - acc: 0.9992 - val_loss: 0.2962 - val_acc: 0.9607 Epoch 661/1000 333s 666ms/step - loss: 0.1488 - acc: 0.9993 - val_loss: 0.2975 - val_acc: 0.9593 Epoch 662/1000 331s 663ms/step - loss: 0.1487 - acc: 0.9992 - val_loss: 0.2947 - val_acc: 0.9608 Epoch 663/1000 334s 667ms/step - loss: 0.1483 - acc: 0.9993 - val_loss: 0.2956 - val_acc: 0.9598 Epoch 664/1000 334s 668ms/step - loss: 0.1483 - acc: 0.9991 - val_loss: 0.2938 - val_acc: 0.9608 Epoch 665/1000 334s 668ms/step - loss: 0.1479 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9607 Epoch 666/1000 334s 667ms/step - loss: 0.1478 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9607 Epoch 667/1000 332s 663ms/step - loss: 0.1471 - acc: 0.9993 - val_loss: 0.2930 - val_acc: 0.9606 Epoch 668/1000 334s 667ms/step - loss: 0.1471 - acc: 0.9992 - val_loss: 0.2920 - val_acc: 0.9609 Epoch 669/1000 333s 667ms/step - loss: 0.1467 - acc: 0.9992 - val_loss: 0.2909 - val_acc: 0.9613 Epoch 670/1000 334s 668ms/step - loss: 0.1463 - acc: 0.9994 - val_loss: 0.2927 - val_acc: 0.9613 Epoch 671/1000 334s 667ms/step - loss: 0.1461 - acc: 0.9993 - val_loss: 0.2942 - val_acc: 0.9601 Epoch 672/1000 331s 663ms/step - loss: 0.1455 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9611 Epoch 673/1000 331s 663ms/step - loss: 0.1452 - acc: 0.9994 - val_loss: 0.2908 - val_acc: 0.9613 Epoch 674/1000 331s 662ms/step - loss: 0.1453 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9595 Epoch 675/1000 331s 662ms/step - loss: 0.1449 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9615 Epoch 676/1000 333s 665ms/step - loss: 0.1450 - acc: 0.9993 - val_loss: 0.2908 - val_acc: 0.9613 Epoch 677/1000 333s 666ms/step - loss: 0.1442 - acc: 0.9995 - val_loss: 0.2916 - val_acc: 0.9612 Epoch 678/1000 332s 664ms/step - loss: 0.1439 - acc: 0.9995 - val_loss: 0.2944 - val_acc: 0.9609 Epoch 679/1000 334s 667ms/step - loss: 0.1440 - acc: 0.9994 - val_loss: 0.2921 - val_acc: 0.9612 Epoch 680/1000 331s 663ms/step - loss: 0.1432 - acc: 0.9996 - val_loss: 0.2919 - val_acc: 0.9613 Epoch 681/1000 334s 669ms/step - loss: 0.1436 - acc: 0.9992 - val_loss: 0.2935 - val_acc: 0.9599 Epoch 682/1000 332s 665ms/step - loss: 0.1431 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9608 Epoch 683/1000 333s 666ms/step - loss: 0.1434 - acc: 0.9991 - val_loss: 0.2910 - val_acc: 0.9613 Epoch 684/1000 333s 665ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2907 - val_acc: 0.9612 Epoch 685/1000 332s 664ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9611 Epoch 686/1000 333s 666ms/step - loss: 0.1423 - acc: 0.9991 - val_loss: 0.2946 - val_acc: 0.9606 Epoch 687/1000 331s 661ms/step - loss: 0.1417 - acc: 0.9994 - val_loss: 0.2935 - val_acc: 0.9616 Epoch 688/1000 334s 667ms/step - loss: 0.1416 - acc: 0.9995 - val_loss: 0.2922 - val_acc: 0.9612 Epoch 689/1000 333s 667ms/step - loss: 0.1413 - acc: 0.9994 - val_loss: 0.2932 - val_acc: 0.9609 Epoch 690/1000 334s 667ms/step - loss: 0.1413 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9616 Epoch 691/1000 332s 665ms/step - loss: 0.1413 - acc: 0.9990 - val_loss: 0.2935 - val_acc: 0.9619 Epoch 692/1000 333s 666ms/step - loss: 0.1410 - acc: 0.9993 - val_loss: 0.2952 - val_acc: 0.9612 Epoch 693/1000 333s 667ms/step - loss: 0.1407 - acc: 0.9992 - val_loss: 0.2952 - val_acc: 0.9602 Epoch 694/1000 331s 662ms/step - loss: 0.1404 - acc: 0.9992 - val_loss: 0.2940 - val_acc: 0.9600 Epoch 695/1000 333s 667ms/step - loss: 0.1400 - acc: 0.9995 - val_loss: 0.2934 - val_acc: 0.9603 Epoch 696/1000 334s 668ms/step - loss: 0.1396 - acc: 0.9994 - val_loss: 0.2940 - val_acc: 0.9602 Epoch 697/1000 333s 665ms/step - loss: 0.1394 - acc: 0.9994 - val_loss: 0.2925 - val_acc: 0.9599 Epoch 698/1000 333s 667ms/step - loss: 0.1393 - acc: 0.9994 - val_loss: 0.2920 - val_acc: 0.9611 Epoch 699/1000 333s 667ms/step - loss: 0.1392 - acc: 0.9992 - val_loss: 0.2884 - val_acc: 0.9611 Epoch 700/1000 333s 666ms/step - loss: 0.1383 - acc: 0.9994 - val_loss: 0.2877 - val_acc: 0.9609 Epoch 701/1000 331s 661ms/step - loss: 0.1385 - acc: 0.9993 - val_loss: 0.2887 - val_acc: 0.9605 Epoch 702/1000 332s 665ms/step - loss: 0.1387 - acc: 0.9992 - val_loss: 0.2883 - val_acc: 0.9606 Epoch 703/1000 331s 661ms/step - loss: 0.1380 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9603 Epoch 704/1000 331s 662ms/step - loss: 0.1376 - acc: 0.9994 - val_loss: 0.2886 - val_acc: 0.9598 Epoch 705/1000 334s 668ms/step - loss: 0.1379 - acc: 0.9993 - val_loss: 0.2891 - val_acc: 0.9608 Epoch 706/1000 333s 666ms/step - loss: 0.1372 - acc: 0.9994 - val_loss: 0.2902 - val_acc: 0.9595 Epoch 707/1000 333s 666ms/step - loss: 0.1370 - acc: 0.9995 - val_loss: 0.2895 - val_acc: 0.9594 Epoch 708/1000 333s 665ms/step - loss: 0.1370 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9594 Epoch 709/1000 331s 662ms/step - loss: 0.1366 - acc: 0.9993 - val_loss: 0.2897 - val_acc: 0.9594 Epoch 710/1000 331s 662ms/step - loss: 0.1362 - acc: 0.9996 - val_loss: 0.2913 - val_acc: 0.9589 Epoch 711/1000 334s 667ms/step - loss: 0.1362 - acc: 0.9993 - val_loss: 0.2890 - val_acc: 0.9609 Epoch 712/1000 333s 666ms/step - loss: 0.1360 - acc: 0.9993 - val_loss: 0.2924 - val_acc: 0.9584 Epoch 713/1000 333s 666ms/step - loss: 0.1361 - acc: 0.9992 - val_loss: 0.2912 - val_acc: 0.9587 Epoch 714/1000 331s 661ms/step - loss: 0.1354 - acc: 0.9993 - val_loss: 0.2904 - val_acc: 0.9581 Epoch 715/1000 331s 662ms/step - loss: 0.1354 - acc: 0.9994 - val_loss: 0.2901 - val_acc: 0.9581 Epoch 716/1000 333s 666ms/step - loss: 0.1351 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9582 Epoch 717/1000 332s 664ms/step - loss: 0.1349 - acc: 0.9992 - val_loss: 0.2881 - val_acc: 0.9585 Epoch 718/1000 331s 662ms/step - loss: 0.1345 - acc: 0.9995 - val_loss: 0.2870 - val_acc: 0.9596 Epoch 719/1000 331s 662ms/step - loss: 0.1344 - acc: 0.9994 - val_loss: 0.2863 - val_acc: 0.9600 Epoch 720/1000 333s 667ms/step - loss: 0.1343 - acc: 0.9993 - val_loss: 0.2881 - val_acc: 0.9592 Epoch 721/1000 334s 667ms/step - loss: 0.1338 - acc: 0.9995 - val_loss: 0.2869 - val_acc: 0.9595 Epoch 722/1000 331s 661ms/step - loss: 0.1336 - acc: 0.9995 - val_loss: 0.2853 - val_acc: 0.9599 Epoch 723/1000 334s 668ms/step - loss: 0.1334 - acc: 0.9994 - val_loss: 0.2856 - val_acc: 0.9603 Epoch 724/1000 331s 661ms/step - loss: 0.1330 - acc: 0.9995 - val_loss: 0.2843 - val_acc: 0.9600 Epoch 725/1000 332s 664ms/step - loss: 0.1330 - acc: 0.9993 - val_loss: 0.2853 - val_acc: 0.9601 Epoch 726/1000 331s 661ms/step - loss: 0.1329 - acc: 0.9994 - val_loss: 0.2819 - val_acc: 0.9600 Epoch 727/1000 331s 662ms/step - loss: 0.1326 - acc: 0.9994 - val_loss: 0.2816 - val_acc: 0.9596 Epoch 728/1000 333s 665ms/step - loss: 0.1324 - acc: 0.9993 - val_loss: 0.2826 - val_acc: 0.9598 Epoch 729/1000 333s 666ms/step - loss: 0.1323 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9597 Epoch 730/1000 331s 662ms/step - loss: 0.1321 - acc: 0.9993 - val_loss: 0.2831 - val_acc: 0.9604 Epoch 731/1000 333s 666ms/step - loss: 0.1314 - acc: 0.9995 - val_loss: 0.2833 - val_acc: 0.9597 Epoch 732/1000 333s 666ms/step - loss: 0.1315 - acc: 0.9994 - val_loss: 0.2808 - val_acc: 0.9609 Epoch 733/1000 331s 662ms/step - loss: 0.1309 - acc: 0.9994 - val_loss: 0.2831 - val_acc: 0.9593 Epoch 734/1000 332s 665ms/step - loss: 0.1308 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9592 Epoch 735/1000 331s 662ms/step - loss: 0.1306 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9596 Epoch 736/1000 332s 664ms/step - loss: 0.1305 - acc: 0.9993 - val_loss: 0.2838 - val_acc: 0.9590 Epoch 737/1000 331s 662ms/step - loss: 0.1302 - acc: 0.9994 - val_loss: 0.2829 - val_acc: 0.9593 Epoch 738/1000 333s 666ms/step - loss: 0.1304 - acc: 0.9993 - val_loss: 0.2845 - val_acc: 0.9580 Epoch 739/1000 333s 666ms/step - loss: 0.1296 - acc: 0.9996 - val_loss: 0.2878 - val_acc: 0.9579 Epoch 740/1000 333s 666ms/step - loss: 0.1298 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587 Epoch 741/1000 333s 666ms/step - loss: 0.1294 - acc: 0.9995 - val_loss: 0.2859 - val_acc: 0.9593 Epoch 742/1000 333s 665ms/step - loss: 0.1290 - acc: 0.9995 - val_loss: 0.2840 - val_acc: 0.9589 Epoch 743/1000 333s 666ms/step - loss: 0.1288 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9591 Epoch 744/1000 332s 665ms/step - loss: 0.1286 - acc: 0.9995 - val_loss: 0.2868 - val_acc: 0.9587 Epoch 745/1000 331s 662ms/step - loss: 0.1288 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587 Epoch 746/1000 331s 661ms/step - loss: 0.1285 - acc: 0.9993 - val_loss: 0.2841 - val_acc: 0.9589 Epoch 747/1000 331s 662ms/step - loss: 0.1282 - acc: 0.9995 - val_loss: 0.2841 - val_acc: 0.9589 Epoch 748/1000 334s 667ms/step - loss: 0.1281 - acc: 0.9994 - val_loss: 0.2846 - val_acc: 0.9585 Epoch 749/1000 333s 665ms/step - loss: 0.1276 - acc: 0.9995 - val_loss: 0.2846 - val_acc: 0.9597 Epoch 750/1000 333s 665ms/step - loss: 0.1272 - acc: 0.9996 - val_loss: 0.2839 - val_acc: 0.9589 Epoch 751/1000 333s 667ms/step - loss: 0.1272 - acc: 0.9995 - val_loss: 0.2831 - val_acc: 0.9594 Epoch 752/1000 332s 665ms/step - loss: 0.1273 - acc: 0.9994 - val_loss: 0.2828 - val_acc: 0.9592 Epoch 753/1000 333s 666ms/step - loss: 0.1271 - acc: 0.9994 - val_loss: 0.2823 - val_acc: 0.9592 Epoch 754/1000 333s 667ms/step - loss: 0.1267 - acc: 0.9995 - val_loss: 0.2826 - val_acc: 0.9598 ... Epoch 899/1000 333s 666ms/step - loss: 0.1018 - acc: 0.9994 - val_loss: 0.2702 - val_acc: 0.9570 Epoch 900/1000 333s 666ms/step - loss: 0.1019 - acc: 0.9992 - val_loss: 0.2697 - val_acc: 0.9573 Epoch 901/1000 lr changed to 9.999999310821295e-05 331s 661ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.2677 - val_acc: 0.9572 Epoch 902/1000 333s 666ms/step - loss: 0.1014 - acc: 0.9994 - val_loss: 0.2691 - val_acc: 0.9573 Epoch 903/1000 331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2666 - val_acc: 0.9587 Epoch 904/1000 335s 669ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2680 - val_acc: 0.9575 Epoch 905/1000 334s 669ms/step - loss: 0.1016 - acc: 0.9994 - val_loss: 0.2659 - val_acc: 0.9581 Epoch 906/1000 334s 667ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2659 - val_acc: 0.9578 Epoch 907/1000 333s 666ms/step - loss: 0.1010 - acc: 0.9996 - val_loss: 0.2642 - val_acc: 0.9582 Epoch 908/1000 333s 665ms/step - loss: 0.1008 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9574 Epoch 909/1000 331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2651 - val_acc: 0.9573 Epoch 910/1000 332s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2656 - val_acc: 0.9579 Epoch 911/1000 333s 666ms/step - loss: 0.1013 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9577 Epoch 912/1000 331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2654 - val_acc: 0.9578 Epoch 913/1000 334s 667ms/step - loss: 0.1006 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9576 Epoch 914/1000 333s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9579 Epoch 915/1000 333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2649 - val_acc: 0.9586 Epoch 916/1000 332s 665ms/step - loss: 0.1009 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9577 Epoch 917/1000 332s 664ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2648 - val_acc: 0.9584 Epoch 918/1000 331s 662ms/step - loss: 0.1005 - acc: 0.9997 - val_loss: 0.2652 - val_acc: 0.9586 Epoch 919/1000 333s 666ms/step - loss: 0.1012 - acc: 0.9994 - val_loss: 0.2642 - val_acc: 0.9590 Epoch 920/1000 331s 662ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9586 Epoch 921/1000 334s 669ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2645 - val_acc: 0.9579 Epoch 922/1000 333s 665ms/step - loss: 0.1012 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591 Epoch 923/1000 332s 665ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2639 - val_acc: 0.9583 Epoch 924/1000 331s 663ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2630 - val_acc: 0.9586 Epoch 925/1000 335s 669ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588 Epoch 926/1000 335s 670ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9583 Epoch 927/1000 336s 673ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2637 - val_acc: 0.9587 Epoch 928/1000 333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2641 - val_acc: 0.9584 Epoch 929/1000 336s 672ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9591 Epoch 930/1000 339s 679ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9581 Epoch 931/1000 338s 676ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2625 - val_acc: 0.9583 Epoch 932/1000 338s 675ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9584 Epoch 933/1000 331s 662ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9586 Epoch 934/1000 334s 668ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9583 Epoch 935/1000 334s 667ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2626 - val_acc: 0.9584 Epoch 936/1000 333s 667ms/step - loss: 0.1007 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9598 Epoch 937/1000 332s 665ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590 Epoch 938/1000 332s 663ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9590 Epoch 939/1000 333s 666ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2636 - val_acc: 0.9592 Epoch 940/1000 333s 666ms/step - loss: 0.1004 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9587 Epoch 941/1000 333s 665ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2644 - val_acc: 0.9587 Epoch 942/1000 333s 666ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9587 Epoch 943/1000 335s 669ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2625 - val_acc: 0.9595 Epoch 944/1000 334s 668ms/step - loss: 0.1009 - acc: 0.9994 - val_loss: 0.2635 - val_acc: 0.9585 Epoch 945/1000 334s 668ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2652 - val_acc: 0.9586 Epoch 946/1000 331s 662ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9588 Epoch 947/1000 333s 666ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2638 - val_acc: 0.9583 Epoch 948/1000 333s 667ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2634 - val_acc: 0.9594 Epoch 949/1000 331s 662ms/step - loss: 0.1002 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9591 Epoch 950/1000 333s 667ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2616 - val_acc: 0.9591 Epoch 951/1000 333s 666ms/step - loss: 0.1000 - acc: 0.9998 - val_loss: 0.2634 - val_acc: 0.9590 Epoch 952/1000 331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2619 - val_acc: 0.9593 Epoch 953/1000 334s 668ms/step - loss: 0.0997 - acc: 0.9998 - val_loss: 0.2638 - val_acc: 0.9587 Epoch 954/1000 333s 666ms/step - loss: 0.1003 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9593 Epoch 955/1000 333s 666ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2630 - val_acc: 0.9587 Epoch 956/1000 331s 662ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9587 Epoch 957/1000 333s 667ms/step - loss: 0.1001 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9588 Epoch 958/1000 332s 665ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591 Epoch 959/1000 333s 667ms/step - loss: 0.1003 - acc: 0.9997 - val_loss: 0.2625 - val_acc: 0.9591 Epoch 960/1000 333s 665ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2639 - val_acc: 0.9595 Epoch 961/1000 331s 662ms/step - loss: 0.1003 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9590 Epoch 962/1000 333s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595 Epoch 963/1000 333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2635 - val_acc: 0.9594 Epoch 964/1000 331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2623 - val_acc: 0.9593 Epoch 965/1000 331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9593 Epoch 966/1000 333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9592 Epoch 967/1000 331s 661ms/step - loss: 0.0995 - acc: 0.9998 - val_loss: 0.2636 - val_acc: 0.9594 Epoch 968/1000 331s 663ms/step - loss: 0.0998 - acc: 0.9997 - val_loss: 0.2642 - val_acc: 0.9585 Epoch 969/1000 336s 672ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9595 Epoch 970/1000 333s 667ms/step - loss: 0.1000 - acc: 0.9997 - val_loss: 0.2633 - val_acc: 0.9596 Epoch 971/1000 332s 665ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9589 Epoch 972/1000 331s 662ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2655 - val_acc: 0.9600 Epoch 973/1000 333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9590 Epoch 974/1000 331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2641 - val_acc: 0.9588 Epoch 975/1000 333s 667ms/step - loss: 0.0996 - acc: 0.9998 - val_loss: 0.2651 - val_acc: 0.9593 Epoch 976/1000 333s 666ms/step - loss: 0.1000 - acc: 0.9995 - val_loss: 0.2647 - val_acc: 0.9591 Epoch 977/1000 334s 668ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9589 Epoch 978/1000 331s 662ms/step - loss: 0.0997 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9585 Epoch 979/1000 333s 665ms/step - loss: 0.0994 - acc: 0.9998 - val_loss: 0.2641 - val_acc: 0.9584 Epoch 980/1000 333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588 Epoch 981/1000 331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9592 Epoch 982/1000 333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9584 Epoch 983/1000 331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9604 Epoch 984/1000 334s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9599 Epoch 985/1000 334s 668ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595 Epoch 986/1000 334s 668ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9596 Epoch 987/1000 333s 666ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9595 Epoch 988/1000 332s 665ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9598 Epoch 989/1000 333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9592 Epoch 990/1000 331s 662ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9598 Epoch 991/1000 331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9596 Epoch 992/1000 331s 662ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9595 Epoch 993/1000 333s 667ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9596 Epoch 994/1000 333s 666ms/step - loss: 0.0993 - acc: 0.9997 - val_loss: 0.2638 - val_acc: 0.9593 Epoch 995/1000 331s 662ms/step - loss: 0.0990 - acc: 0.9998 - val_loss: 0.2639 - val_acc: 0.9595 Epoch 996/1000 333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2643 - val_acc: 0.9585 Epoch 997/1000 331s 662ms/step - loss: 0.0993 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9590 Epoch 998/1000 334s 667ms/step - loss: 0.0993 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590 Epoch 999/1000 331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2638 - val_acc: 0.9596 Epoch 1000/1000 333s 666ms/step - loss: 0.0991 - acc: 0.9997 - val_loss: 0.2636 - val_acc: 0.9596 Train loss: 0.0978851483464241 Train accuracy: 0.9999800000190735 Test loss: 0.26395464450120926 Test accuracy: 0.9592000007629394

最終的實(shí)驗(yàn)結(jié)果是95.92%。其實(shí)，在第670個(gè)epoch的時(shí)候，已經(jīng)到了96.13%，最后反而降了。

參考文獻(xiàn)：

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, DOI: 10.1109/TIE.2020.2972458, Date of Publication: 13 February 2020

Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis?ieeexplore.ieee.org

總結(jié)

以上是生活随笔為你收集整理的input自适应_一种Dynamic ReLU：自适应参数化ReLU（调参记录26）Cifar10~95.92%的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。