1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
| epoch 1 | 200/ 1081 batches | lr 30.00 | ms/batch 332.10 | loss 6.47 | ppl 648.68
| epoch 1 | 400/ 1081 batches | lr 30.00 | ms/batch 321.14 | loss 4.87 | ppl 130.97
| epoch 1 | 600/ 1081 batches | lr 30.00 | ms/batch 324.12 | loss 3.96 | ppl 52.20
| epoch 1 | 800/ 1081 batches | lr 30.00 | ms/batch 323.84 | loss 3.58 | ppl 35.88
| epoch 1 | 1000/ 1081 batches | lr 30.00 | ms/batch 322.87 | loss 3.36 | ppl 28.71
-----------------------------------------------------------------------------------------
| end of epoch 1 | time: 378.06s | valid loss 4.44 | valid ppl 85.19
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 2 | 200/ 1081 batches | lr 30.00 | ms/batch 329.21 | loss 3.51 | ppl 33.36
| epoch 2 | 400/ 1081 batches | lr 30.00 | ms/batch 324.86 | loss 3.36 | ppl 28.89
| epoch 2 | 600/ 1081 batches | lr 30.00 | ms/batch 326.57 | loss 2.93 | ppl 18.76
| epoch 2 | 800/ 1081 batches | lr 30.00 | ms/batch 319.19 | loss 2.85 | ppl 17.22
| epoch 2 | 1000/ 1081 batches | lr 30.00 | ms/batch 323.80 | loss 2.76 | ppl 15.80
-----------------------------------------------------------------------------------------
| end of epoch 2 | time: 377.65s | valid loss 4.27 | valid ppl 71.17
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 3 | 200/ 1081 batches | lr 30.00 | ms/batch 327.43 | loss 2.98 | ppl 19.62
| epoch 3 | 400/ 1081 batches | lr 30.00 | ms/batch 329.49 | loss 2.96 | ppl 19.24
| epoch 3 | 600/ 1081 batches | lr 30.00 | ms/batch 323.91 | loss 2.59 | ppl 13.39
| epoch 3 | 800/ 1081 batches | lr 30.00 | ms/batch 324.70 | loss 2.57 | ppl 13.03
| epoch 3 | 1000/ 1081 batches | lr 30.00 | ms/batch 323.03 | loss 2.51 | ppl 12.33
-----------------------------------------------------------------------------------------
| end of epoch 3 | time: 378.10s | valid loss 4.18 | valid ppl 65.69
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 4 | 200/ 1081 batches | lr 30.00 | ms/batch 331.71 | loss 2.73 | ppl 15.32
| epoch 4 | 400/ 1081 batches | lr 30.00 | ms/batch 326.49 | loss 2.72 | ppl 15.18
| epoch 4 | 600/ 1081 batches | lr 30.00 | ms/batch 321.82 | loss 2.39 | ppl 10.94
| epoch 4 | 800/ 1081 batches | lr 30.00 | ms/batch 324.35 | loss 2.39 | ppl 10.90
| epoch 4 | 1000/ 1081 batches | lr 30.00 | ms/batch 316.77 | loss 2.33 | ppl 10.32
-----------------------------------------------------------------------------------------
| end of epoch 4 | time: 376.85s | valid loss 4.11 | valid ppl 61.22
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 5 | 200/ 1081 batches | lr 30.00 | ms/batch 326.00 | loss 2.56 | ppl 12.93
| epoch 5 | 400/ 1081 batches | lr 30.00 | ms/batch 320.03 | loss 2.58 | ppl 13.26
| epoch 5 | 600/ 1081 batches | lr 30.00 | ms/batch 321.69 | loss 2.28 | ppl 9.79
| epoch 5 | 800/ 1081 batches | lr 30.00 | ms/batch 323.70 | loss 2.30 | ppl 9.93
| epoch 5 | 1000/ 1081 batches | lr 30.00 | ms/batch 324.63 | loss 2.21 | ppl 9.09
-----------------------------------------------------------------------------------------
| end of epoch 5 | time: 376.65s | valid loss 4.11 | valid ppl 60.81
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 6 | 200/ 1081 batches | lr 30.00 | ms/batch 325.99 | loss 2.46 | ppl 11.65
| epoch 6 | 400/ 1081 batches | lr 30.00 | ms/batch 325.63 | loss 2.47 | ppl 11.85
| epoch 6 | 600/ 1081 batches | lr 30.00 | ms/batch 324.08 | loss 2.18 | ppl 8.85
| epoch 6 | 800/ 1081 batches | lr 30.00 | ms/batch 322.38 | loss 2.21 | ppl 9.08
| epoch 6 | 1000/ 1081 batches | lr 30.00 | ms/batch 322.02 | loss 2.15 | ppl 8.56
-----------------------------------------------------------------------------------------
| end of epoch 6 | time: 376.52s | valid loss 4.09 | valid ppl 59.67
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 7 | 200/ 1081 batches | lr 30.00 | ms/batch 327.93 | loss 2.37 | ppl 10.66
| epoch 7 | 400/ 1081 batches | lr 30.00 | ms/batch 329.17 | loss 2.40 | ppl 11.03
| epoch 7 | 600/ 1081 batches | lr 30.00 | ms/batch 324.01 | loss 2.11 | ppl 8.24
| epoch 7 | 800/ 1081 batches | lr 30.00 | ms/batch 326.44 | loss 2.15 | ppl 8.56
| epoch 7 | 1000/ 1081 batches | lr 30.00 | ms/batch 322.63 | loss 2.09 | ppl 8.09
-----------------------------------------------------------------------------------------
| end of epoch 7 | time: 376.37s | valid loss 4.04 | valid ppl 56.80
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 8 | 200/ 1081 batches | lr 30.00 | ms/batch 327.02 | loss 2.30 | ppl 9.99
| epoch 8 | 400/ 1081 batches | lr 30.00 | ms/batch 317.09 | loss 2.34 | ppl 10.36
| epoch 8 | 600/ 1081 batches | lr 30.00 | ms/batch 322.57 | loss 2.06 | ppl 7.84
| epoch 8 | 800/ 1081 batches | lr 30.00 | ms/batch 325.41 | loss 2.09 | ppl 8.06
| epoch 8 | 1000/ 1081 batches | lr 30.00 | ms/batch 322.70 | loss 2.03 | ppl 7.60
-----------------------------------------------------------------------------------------
| end of epoch 8 | time: 376.62s | valid loss 4.06 | valid ppl 57.71
-----------------------------------------------------------------------------------------
| epoch 9 | 200/ 1081 batches | lr 30.00 | ms/batch 326.11 | loss 2.26 | ppl 9.56
| epoch 9 | 400/ 1081 batches | lr 30.00 | ms/batch 322.61 | loss 2.29 | ppl 9.86
| epoch 9 | 600/ 1081 batches | lr 30.00 | ms/batch 322.66 | loss 2.02 | ppl 7.56
| epoch 9 | 800/ 1081 batches | lr 30.00 | ms/batch 322.64 | loss 2.04 | ppl 7.72
| epoch 9 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.81 | loss 1.97 | ppl 7.17
-----------------------------------------------------------------------------------------
| end of epoch 9 | time: 379.63s | valid loss 4.05 | valid ppl 57.25
-----------------------------------------------------------------------------------------
| epoch 10 | 200/ 1081 batches | lr 30.00 | ms/batch 326.10 | loss 2.19 | ppl 8.98
| epoch 10 | 400/ 1081 batches | lr 30.00 | ms/batch 326.89 | loss 2.23 | ppl 9.33
| epoch 10 | 600/ 1081 batches | lr 30.00 | ms/batch 329.47 | loss 1.98 | ppl 7.27
| epoch 10 | 800/ 1081 batches | lr 30.00 | ms/batch 324.90 | loss 2.00 | ppl 7.41
| epoch 10 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.67 | loss 1.96 | ppl 7.11
-----------------------------------------------------------------------------------------
| end of epoch 10 | time: 380.34s | valid loss 4.04 | valid ppl 56.72
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 11 | 200/ 1081 batches | lr 30.00 | ms/batch 322.57 | loss 2.16 | ppl 8.68
| epoch 11 | 400/ 1081 batches | lr 30.00 | ms/batch 321.98 | loss 2.22 | ppl 9.20
| epoch 11 | 600/ 1081 batches | lr 30.00 | ms/batch 321.70 | loss 1.95 | ppl 7.06
| epoch 11 | 800/ 1081 batches | lr 30.00 | ms/batch 320.59 | loss 1.97 | ppl 7.15
| epoch 11 | 1000/ 1081 batches | lr 30.00 | ms/batch 319.73 | loss 1.91 | ppl 6.74
-----------------------------------------------------------------------------------------
| end of epoch 11 | time: 375.85s | valid loss 4.03 | valid ppl 56.06
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 12 | 200/ 1081 batches | lr 30.00 | ms/batch 326.03 | loss 2.13 | ppl 8.42
| epoch 12 | 400/ 1081 batches | lr 30.00 | ms/batch 325.38 | loss 2.18 | ppl 8.83
| epoch 12 | 600/ 1081 batches | lr 30.00 | ms/batch 326.67 | loss 1.92 | ppl 6.83
| epoch 12 | 800/ 1081 batches | lr 30.00 | ms/batch 320.86 | loss 1.94 | ppl 6.95
| epoch 12 | 1000/ 1081 batches | lr 30.00 | ms/batch 321.92 | loss 1.88 | ppl 6.54
-----------------------------------------------------------------------------------------
| end of epoch 12 | time: 376.20s | valid loss 4.01 | valid ppl 55.28
-----------------------------------------------------------------------------------------
Saving Normal!
| epoch 13 | 200/ 1081 batches | lr 30.00 | ms/batch 328.38 | loss 2.11 | ppl 8.23
| epoch 13 | 400/ 1081 batches | lr 30.00 | ms/batch 322.65 | loss 2.14 | ppl 8.47
| epoch 13 | 600/ 1081 batches | lr 30.00 | ms/batch 325.03 | loss 1.89 | ppl 6.62
| epoch 13 | 800/ 1081 batches | lr 30.00 | ms/batch 321.91 | loss 1.91 | ppl 6.75
| epoch 13 | 1000/ 1081 batches | lr 30.00 | ms/batch 326.44 | loss 1.86 | ppl 6.43
-----------------------------------------------------------------------------------------
| end of epoch 13 | time: 375.46s | valid loss 4.02 | valid ppl 55.84
-----------------------------------------------------------------------------------------
| epoch 14 | 200/ 1081 batches | lr 30.00 | ms/batch 314.80 | loss 2.07 | ppl 7.94
| epoch 14 | 400/ 1081 batches | lr 30.00 | ms/batch 323.74 | loss 2.12 | ppl 8.34
| epoch 14 | 600/ 1081 batches | lr 30.00 | ms/batch 318.27 | loss 1.87 | ppl 6.52
| epoch 14 | 800/ 1081 batches | lr 30.00 | ms/batch 325.38 | loss 1.90 | ppl 6.72
| epoch 14 | 1000/ 1081 batches | lr 30.00 | ms/batch 322.33 | loss 1.83 | ppl 6.25
-----------------------------------------------------------------------------------------
| end of epoch 14 | time: 375.99s | valid loss 4.02 | valid ppl 55.70
-----------------------------------------------------------------------------------------
| epoch 15 | 200/ 1081 batches | lr 30.00 | ms/batch 320.61 | loss 2.06 | ppl 7.85
| epoch 15 | 400/ 1081 batches | lr 30.00 | ms/batch 322.74 | loss 2.10 | ppl 8.18
| epoch 15 | 600/ 1081 batches | lr 30.00 | ms/batch 324.20 | loss 1.85 | ppl 6.34
| epoch 15 | 800/ 1081 batches | lr 30.00 | ms/batch 325.72 | loss 1.88 | ppl 6.53
| epoch 15 | 1000/ 1081 batches | lr 30.00 | ms/batch 321.70 | loss 1.81 | ppl 6.10
-----------------------------------------------------------------------------------------
| end of epoch 15 | time: 375.81s | valid loss 4.02 | valid ppl 55.94
-----------------------------------------------------------------------------------------
| epoch 16 | 200/ 1081 batches | lr 30.00 | ms/batch 326.27 | loss 2.03 | ppl 7.61
| epoch 16 | 400/ 1081 batches | lr 30.00 | ms/batch 323.36 | loss 2.08 | ppl 8.01
| epoch 16 | 600/ 1081 batches | lr 30.00 | ms/batch 319.92 | loss 1.83 | ppl 6.21
| epoch 16 | 800/ 1081 batches | lr 30.00 | ms/batch 327.85 | loss 1.84 | ppl 6.32
| epoch 16 | 1000/ 1081 batches | lr 30.00 | ms/batch 320.66 | loss 1.79 | ppl 6.00
-----------------------------------------------------------------------------------------
| end of epoch 16 | time: 375.14s | valid loss 4.03 | valid ppl 56.31
-----------------------------------------------------------------------------------------
| epoch 17 | 200/ 1081 batches | lr 30.00 | ms/batch 326.99 | loss 2.01 | ppl 7.45
| epoch 17 | 400/ 1081 batches | lr 30.00 | ms/batch 321.51 | loss 2.06 | ppl 7.83
| epoch 17 | 600/ 1081 batches | lr 30.00 | ms/batch 317.95 | loss 1.81 | ppl 6.13
| epoch 17 | 800/ 1081 batches | lr 30.00 | ms/batch 323.17 | loss 1.84 | ppl 6.32
| epoch 17 | 1000/ 1081 batches | lr 30.00 | ms/batch 328.34 | loss 1.77 | ppl 5.88
-----------------------------------------------------------------------------------------
| end of epoch 17 | time: 375.97s | valid loss 4.01 | valid ppl 55.41
-----------------------------------------------------------------------------------------
| epoch 18 | 200/ 1081 batches | lr 30.00 | ms/batch 323.53 | loss 1.98 | ppl 7.28
| epoch 18 | 400/ 1081 batches | lr 30.00 | ms/batch 320.05 | loss 2.03 | ppl 7.64
| epoch 18 | 600/ 1081 batches | lr 30.00 | ms/batch 318.99 | loss 1.81 | ppl 6.09
| epoch 18 | 800/ 1081 batches | lr 30.00 | ms/batch 322.42 | loss 1.82 | ppl 6.16
| epoch 18 | 1000/ 1081 batches | lr 30.00 | ms/batch 323.10 | loss 1.77 | ppl 5.87
-----------------------------------------------------------------------------------------
| end of epoch 18 | time: 375.63s | valid loss 4.02 | valid ppl 55.65
-----------------------------------------------------------------------------------------
Switching!
| epoch 19 | 200/ 1081 batches | lr 30.00 | ms/batch 332.88 | loss 1.97 | ppl 7.14
| epoch 19 | 400/ 1081 batches | lr 30.00 | ms/batch 332.28 | loss 2.01 | ppl 7.48
| epoch 19 | 600/ 1081 batches | lr 30.00 | ms/batch 327.77 | loss 1.79 | ppl 5.97
| epoch 19 | 800/ 1081 batches | lr 30.00 | ms/batch 333.72 | loss 1.81 | ppl 6.10
| epoch 19 | 1000/ 1081 batches | lr 30.00 | ms/batch 335.08 | loss 1.75 | ppl 5.74
-----------------------------------------------------------------------------------------
| end of epoch 19 | time: 386.95s | valid loss 3.93 | valid ppl 51.09
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 20 | 200/ 1081 batches | lr 30.00 | ms/batch 338.21 | loss 1.96 | ppl 7.07
| epoch 20 | 400/ 1081 batches | lr 30.00 | ms/batch 332.95 | loss 2.01 | ppl 7.43
| epoch 20 | 600/ 1081 batches | lr 30.00 | ms/batch 330.24 | loss 1.77 | ppl 5.90
| epoch 20 | 800/ 1081 batches | lr 30.00 | ms/batch 336.25 | loss 1.78 | ppl 5.95
| epoch 20 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.58 | loss 1.74 | ppl 5.68
-----------------------------------------------------------------------------------------
| end of epoch 20 | time: 386.39s | valid loss 3.93 | valid ppl 50.93
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 21 | 200/ 1081 batches | lr 30.00 | ms/batch 331.72 | loss 1.94 | ppl 6.95
| epoch 21 | 400/ 1081 batches | lr 30.00 | ms/batch 326.43 | loss 1.98 | ppl 7.25
| epoch 21 | 600/ 1081 batches | lr 30.00 | ms/batch 329.44 | loss 1.76 | ppl 5.79
| epoch 21 | 800/ 1081 batches | lr 30.00 | ms/batch 329.25 | loss 1.77 | ppl 5.90
| epoch 21 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.93 | loss 1.72 | ppl 5.60
-----------------------------------------------------------------------------------------
| end of epoch 21 | time: 386.87s | valid loss 3.92 | valid ppl 50.62
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 22 | 200/ 1081 batches | lr 30.00 | ms/batch 334.85 | loss 1.93 | ppl 6.86
| epoch 22 | 400/ 1081 batches | lr 30.00 | ms/batch 333.16 | loss 1.97 | ppl 7.15
| epoch 22 | 600/ 1081 batches | lr 30.00 | ms/batch 329.15 | loss 1.74 | ppl 5.72
| epoch 22 | 800/ 1081 batches | lr 30.00 | ms/batch 330.47 | loss 1.77 | ppl 5.87
| epoch 22 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.91 | loss 1.71 | ppl 5.51
-----------------------------------------------------------------------------------------
| end of epoch 22 | time: 386.67s | valid loss 3.92 | valid ppl 50.39
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 23 | 200/ 1081 batches | lr 30.00 | ms/batch 332.34 | loss 1.91 | ppl 6.78
| epoch 23 | 400/ 1081 batches | lr 30.00 | ms/batch 331.38 | loss 1.95 | ppl 7.06
| epoch 23 | 600/ 1081 batches | lr 30.00 | ms/batch 333.49 | loss 1.73 | ppl 5.66
| epoch 23 | 800/ 1081 batches | lr 30.00 | ms/batch 330.66 | loss 1.75 | ppl 5.76
| epoch 23 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.11 | loss 1.69 | ppl 5.42
-----------------------------------------------------------------------------------------
| end of epoch 23 | time: 386.25s | valid loss 3.92 | valid ppl 50.27
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 24 | 200/ 1081 batches | lr 30.00 | ms/batch 339.96 | loss 1.90 | ppl 6.69
| epoch 24 | 400/ 1081 batches | lr 30.00 | ms/batch 335.48 | loss 1.95 | ppl 6.99
| epoch 24 | 600/ 1081 batches | lr 30.00 | ms/batch 333.59 | loss 1.70 | ppl 5.49
| epoch 24 | 800/ 1081 batches | lr 30.00 | ms/batch 330.30 | loss 1.74 | ppl 5.70
| epoch 24 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.11 | loss 1.69 | ppl 5.39
-----------------------------------------------------------------------------------------
| end of epoch 24 | time: 386.30s | valid loss 3.92 | valid ppl 50.16
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 25 | 200/ 1081 batches | lr 30.00 | ms/batch 335.03 | loss 1.89 | ppl 6.61
| epoch 25 | 400/ 1081 batches | lr 30.00 | ms/batch 333.45 | loss 1.92 | ppl 6.85
| epoch 25 | 600/ 1081 batches | lr 30.00 | ms/batch 334.29 | loss 1.70 | ppl 5.50
| epoch 25 | 800/ 1081 batches | lr 30.00 | ms/batch 331.86 | loss 1.73 | ppl 5.63
| epoch 25 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.78 | loss 1.68 | ppl 5.38
-----------------------------------------------------------------------------------------
| end of epoch 25 | time: 385.96s | valid loss 3.91 | valid ppl 50.04
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 26 | 200/ 1081 batches | lr 30.00 | ms/batch 339.20 | loss 1.88 | ppl 6.54
| epoch 26 | 400/ 1081 batches | lr 30.00 | ms/batch 329.29 | loss 1.91 | ppl 6.78
| epoch 26 | 600/ 1081 batches | lr 30.00 | ms/batch 338.71 | loss 1.69 | ppl 5.44
| epoch 26 | 800/ 1081 batches | lr 30.00 | ms/batch 331.23 | loss 1.71 | ppl 5.55
| epoch 26 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.35 | loss 1.66 | ppl 5.27
-----------------------------------------------------------------------------------------
| end of epoch 26 | time: 386.40s | valid loss 3.91 | valid ppl 49.93
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 27 | 200/ 1081 batches | lr 30.00 | ms/batch 334.17 | loss 1.86 | ppl 6.42
| epoch 27 | 400/ 1081 batches | lr 30.00 | ms/batch 331.82 | loss 1.90 | ppl 6.69
| epoch 27 | 600/ 1081 batches | lr 30.00 | ms/batch 331.93 | loss 1.68 | ppl 5.37
| epoch 27 | 800/ 1081 batches | lr 30.00 | ms/batch 330.40 | loss 1.72 | ppl 5.56
| epoch 27 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.77 | loss 1.65 | ppl 5.21
-----------------------------------------------------------------------------------------
| end of epoch 27 | time: 386.36s | valid loss 3.91 | valid ppl 49.83
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 28 | 200/ 1081 batches | lr 30.00 | ms/batch 334.46 | loss 1.85 | ppl 6.33
| epoch 28 | 400/ 1081 batches | lr 30.00 | ms/batch 327.13 | loss 1.89 | ppl 6.62
| epoch 28 | 600/ 1081 batches | lr 30.00 | ms/batch 334.44 | loss 1.68 | ppl 5.37
| epoch 28 | 800/ 1081 batches | lr 30.00 | ms/batch 331.94 | loss 1.71 | ppl 5.51
| epoch 28 | 1000/ 1081 batches | lr 30.00 | ms/batch 336.20 | loss 1.63 | ppl 5.09
-----------------------------------------------------------------------------------------
| end of epoch 28 | time: 386.23s | valid loss 3.91 | valid ppl 49.73
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 29 | 200/ 1081 batches | lr 30.00 | ms/batch 334.11 | loss 1.85 | ppl 6.34
| epoch 29 | 400/ 1081 batches | lr 30.00 | ms/batch 329.25 | loss 1.89 | ppl 6.63
| epoch 29 | 600/ 1081 batches | lr 30.00 | ms/batch 339.03 | loss 1.67 | ppl 5.32
| epoch 29 | 800/ 1081 batches | lr 30.00 | ms/batch 332.18 | loss 1.68 | ppl 5.36
| epoch 29 | 1000/ 1081 batches | lr 30.00 | ms/batch 331.69 | loss 1.64 | ppl 5.17
-----------------------------------------------------------------------------------------
| end of epoch 29 | time: 386.17s | valid loss 3.90 | valid ppl 49.65
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 30 | 200/ 1081 batches | lr 30.00 | ms/batch 334.98 | loss 1.83 | ppl 6.24
| epoch 30 | 400/ 1081 batches | lr 30.00 | ms/batch 329.40 | loss 1.87 | ppl 6.50
| epoch 30 | 600/ 1081 batches | lr 30.00 | ms/batch 333.54 | loss 1.66 | ppl 5.25
| epoch 30 | 800/ 1081 batches | lr 30.00 | ms/batch 328.76 | loss 1.69 | ppl 5.43
| epoch 30 | 1000/ 1081 batches | lr 30.00 | ms/batch 337.20 | loss 1.64 | ppl 5.14
-----------------------------------------------------------------------------------------
| end of epoch 30 | time: 385.63s | valid loss 3.90 | valid ppl 49.54
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 31 | 200/ 1081 batches | lr 30.00 | ms/batch 338.33 | loss 1.82 | ppl 6.18
| epoch 31 | 400/ 1081 batches | lr 30.00 | ms/batch 331.08 | loss 1.86 | ppl 6.43
| epoch 31 | 600/ 1081 batches | lr 30.00 | ms/batch 333.08 | loss 1.65 | ppl 5.22
| epoch 31 | 800/ 1081 batches | lr 30.00 | ms/batch 331.99 | loss 1.67 | ppl 5.31
| epoch 31 | 1000/ 1081 batches | lr 30.00 | ms/batch 336.18 | loss 1.62 | ppl 5.05
-----------------------------------------------------------------------------------------
| end of epoch 31 | time: 386.02s | valid loss 3.90 | valid ppl 49.46
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 32 | 200/ 1081 batches | lr 30.00 | ms/batch 335.20 | loss 1.81 | ppl 6.12
| epoch 32 | 400/ 1081 batches | lr 30.00 | ms/batch 331.51 | loss 1.86 | ppl 6.40
| epoch 32 | 600/ 1081 batches | lr 30.00 | ms/batch 334.79 | loss 1.64 | ppl 5.15
| epoch 32 | 800/ 1081 batches | lr 30.00 | ms/batch 334.04 | loss 1.66 | ppl 5.28
| epoch 32 | 1000/ 1081 batches | lr 30.00 | ms/batch 334.13 | loss 1.62 | ppl 5.05
-----------------------------------------------------------------------------------------
| end of epoch 32 | time: 385.49s | valid loss 3.90 | valid ppl 49.34
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 33 | 200/ 1081 batches | lr 30.00 | ms/batch 334.79 | loss 1.81 | ppl 6.09
| epoch 33 | 400/ 1081 batches | lr 30.00 | ms/batch 336.62 | loss 1.85 | ppl 6.36
| epoch 33 | 600/ 1081 batches | lr 30.00 | ms/batch 333.30 | loss 1.63 | ppl 5.09
| epoch 33 | 800/ 1081 batches | lr 30.00 | ms/batch 333.51 | loss 1.65 | ppl 5.19
| epoch 33 | 1000/ 1081 batches | lr 30.00 | ms/batch 333.77 | loss 1.61 | ppl 5.02
-----------------------------------------------------------------------------------------
| end of epoch 33 | time: 385.99s | valid loss 3.90 | valid ppl 49.23
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 34 | 200/ 1081 batches | lr 30.00 | ms/batch 331.56 | loss 1.80 | ppl 6.07
| epoch 34 | 400/ 1081 batches | lr 30.00 | ms/batch 336.32 | loss 1.83 | ppl 6.25
| epoch 34 | 600/ 1081 batches | lr 30.00 | ms/batch 336.19 | loss 1.61 | ppl 5.02
| epoch 34 | 800/ 1081 batches | lr 30.00 | ms/batch 331.02 | loss 1.65 | ppl 5.20
| epoch 34 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.05 | loss 1.60 | ppl 4.94
-----------------------------------------------------------------------------------------
| end of epoch 34 | time: 385.59s | valid loss 3.89 | valid ppl 49.13
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 35 | 200/ 1081 batches | lr 30.00 | ms/batch 335.11 | loss 1.79 | ppl 5.99
| epoch 35 | 400/ 1081 batches | lr 30.00 | ms/batch 328.63 | loss 1.83 | ppl 6.21
| epoch 35 | 600/ 1081 batches | lr 30.00 | ms/batch 335.98 | loss 1.61 | ppl 5.02
| epoch 35 | 800/ 1081 batches | lr 30.00 | ms/batch 331.66 | loss 1.63 | ppl 5.11
| epoch 35 | 1000/ 1081 batches | lr 30.00 | ms/batch 335.89 | loss 1.59 | ppl 4.92
-----------------------------------------------------------------------------------------
| end of epoch 35 | time: 385.41s | valid loss 3.89 | valid ppl 49.02
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 36 | 200/ 1081 batches | lr 30.00 | ms/batch 335.39 | loss 1.77 | ppl 5.86
| epoch 36 | 400/ 1081 batches | lr 30.00 | ms/batch 328.02 | loss 1.82 | ppl 6.19
| epoch 36 | 600/ 1081 batches | lr 30.00 | ms/batch 329.77 | loss 1.61 | ppl 5.00
| epoch 36 | 800/ 1081 batches | lr 30.00 | ms/batch 329.08 | loss 1.62 | ppl 5.07
| epoch 36 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.41 | loss 1.59 | ppl 4.91
-----------------------------------------------------------------------------------------
| end of epoch 36 | time: 386.11s | valid loss 3.89 | valid ppl 48.91
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 37 | 200/ 1081 batches | lr 30.00 | ms/batch 330.03 | loss 1.77 | ppl 5.85
| epoch 37 | 400/ 1081 batches | lr 30.00 | ms/batch 333.44 | loss 1.81 | ppl 6.13
| epoch 37 | 600/ 1081 batches | lr 30.00 | ms/batch 332.75 | loss 1.60 | ppl 4.94
| epoch 37 | 800/ 1081 batches | lr 30.00 | ms/batch 327.89 | loss 1.62 | ppl 5.05
| epoch 37 | 1000/ 1081 batches | lr 30.00 | ms/batch 333.83 | loss 1.56 | ppl 4.78
-----------------------------------------------------------------------------------------
| end of epoch 37 | time: 385.77s | valid loss 3.89 | valid ppl 48.79
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 38 | 200/ 1081 batches | lr 30.00 | ms/batch 334.21 | loss 1.75 | ppl 5.77
| epoch 38 | 400/ 1081 batches | lr 30.00 | ms/batch 331.16 | loss 1.81 | ppl 6.09
| epoch 38 | 600/ 1081 batches | lr 30.00 | ms/batch 327.69 | loss 1.59 | ppl 4.92
| epoch 38 | 800/ 1081 batches | lr 30.00 | ms/batch 333.82 | loss 1.63 | ppl 5.08
| epoch 38 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.58 | loss 1.55 | ppl 4.73
-----------------------------------------------------------------------------------------
| end of epoch 38 | time: 386.54s | valid loss 3.89 | valid ppl 48.67
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 39 | 200/ 1081 batches | lr 30.00 | ms/batch 333.50 | loss 1.74 | ppl 5.71
| epoch 39 | 400/ 1081 batches | lr 30.00 | ms/batch 331.01 | loss 1.79 | ppl 6.00
| epoch 39 | 600/ 1081 batches | lr 30.00 | ms/batch 335.61 | loss 1.59 | ppl 4.88
| epoch 39 | 800/ 1081 batches | lr 30.00 | ms/batch 332.61 | loss 1.60 | ppl 4.97
| epoch 39 | 1000/ 1081 batches | lr 30.00 | ms/batch 330.35 | loss 1.56 | ppl 4.76
-----------------------------------------------------------------------------------------
| end of epoch 39 | time: 385.67s | valid loss 3.88 | valid ppl 48.57
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 40 | 200/ 1081 batches | lr 30.00 | ms/batch 338.89 | loss 1.74 | ppl 5.71
| epoch 40 | 400/ 1081 batches | lr 30.00 | ms/batch 334.24 | loss 1.78 | ppl 5.96
| epoch 40 | 600/ 1081 batches | lr 30.00 | ms/batch 328.72 | loss 1.57 | ppl 4.79
| epoch 40 | 800/ 1081 batches | lr 30.00 | ms/batch 332.18 | loss 1.60 | ppl 4.95
| epoch 40 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.37 | loss 1.55 | ppl 4.73
-----------------------------------------------------------------------------------------
| end of epoch 40 | time: 385.90s | valid loss 3.88 | valid ppl 48.48
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 41 | 200/ 1081 batches | lr 30.00 | ms/batch 333.54 | loss 1.73 | ppl 5.65
| epoch 41 | 400/ 1081 batches | lr 30.00 | ms/batch 334.45 | loss 1.78 | ppl 5.95
| epoch 41 | 600/ 1081 batches | lr 30.00 | ms/batch 327.31 | loss 1.57 | ppl 4.81
| epoch 41 | 800/ 1081 batches | lr 30.00 | ms/batch 330.45 | loss 1.61 | ppl 4.99
| epoch 41 | 1000/ 1081 batches | lr 30.00 | ms/batch 335.40 | loss 1.55 | ppl 4.70
-----------------------------------------------------------------------------------------
| end of epoch 41 | time: 385.66s | valid loss 3.88 | valid ppl 48.37
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 42 | 200/ 1081 batches | lr 30.00 | ms/batch 338.79 | loss 1.71 | ppl 5.54
| epoch 42 | 400/ 1081 batches | lr 30.00 | ms/batch 330.36 | loss 1.78 | ppl 5.91
| epoch 42 | 600/ 1081 batches | lr 30.00 | ms/batch 327.10 | loss 1.57 | ppl 4.81
| epoch 42 | 800/ 1081 batches | lr 30.00 | ms/batch 327.99 | loss 1.58 | ppl 4.85
| epoch 42 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.02 | loss 1.54 | ppl 4.65
-----------------------------------------------------------------------------------------
| end of epoch 42 | time: 385.98s | valid loss 3.88 | valid ppl 48.25
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 43 | 200/ 1081 batches | lr 30.00 | ms/batch 332.72 | loss 1.71 | ppl 5.53
| epoch 43 | 400/ 1081 batches | lr 30.00 | ms/batch 332.06 | loss 1.77 | ppl 5.84
| epoch 43 | 600/ 1081 batches | lr 30.00 | ms/batch 325.43 | loss 1.56 | ppl 4.77
| epoch 43 | 800/ 1081 batches | lr 30.00 | ms/batch 330.96 | loss 1.57 | ppl 4.82
| epoch 43 | 1000/ 1081 batches | lr 30.00 | ms/batch 332.92 | loss 1.53 | ppl 4.63
-----------------------------------------------------------------------------------------
| end of epoch 43 | time: 385.95s | valid loss 3.87 | valid ppl 48.13
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 44 | 200/ 1081 batches | lr 30.00 | ms/batch 335.44 | loss 1.70 | ppl 5.49
| epoch 44 | 400/ 1081 batches | lr 30.00 | ms/batch 334.82 | loss 1.76 | ppl 5.79
| epoch 44 | 600/ 1081 batches | lr 30.00 | ms/batch 329.24 | loss 1.56 | ppl 4.75
| epoch 44 | 800/ 1081 batches | lr 30.00 | ms/batch 329.88 | loss 1.57 | ppl 4.83
| epoch 44 | 1000/ 1081 batches | lr 30.00 | ms/batch 331.64 | loss 1.53 | ppl 4.61
-----------------------------------------------------------------------------------------
| end of epoch 44 | time: 385.51s | valid loss 3.87 | valid ppl 48.00
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 45 | 200/ 1081 batches | lr 30.00 | ms/batch 333.00 | loss 1.71 | ppl 5.52
| epoch 45 | 400/ 1081 batches | lr 30.00 | ms/batch 333.19 | loss 1.74 | ppl 5.71
| epoch 45 | 600/ 1081 batches | lr 30.00 | ms/batch 328.17 | loss 1.55 | ppl 4.71
| epoch 45 | 800/ 1081 batches | lr 30.00 | ms/batch 331.88 | loss 1.57 | ppl 4.80
| epoch 45 | 1000/ 1081 batches | lr 30.00 | ms/batch 333.65 | loss 1.53 | ppl 4.60
-----------------------------------------------------------------------------------------
| end of epoch 45 | time: 385.96s | valid loss 3.87 | valid ppl 47.87
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 46 | 200/ 1081 batches | lr 30.00 | ms/batch 336.09 | loss 1.69 | ppl 5.44
| epoch 46 | 400/ 1081 batches | lr 30.00 | ms/batch 333.59 | loss 1.73 | ppl 5.65
| epoch 46 | 600/ 1081 batches | lr 30.00 | ms/batch 331.95 | loss 1.53 | ppl 4.61
| epoch 46 | 800/ 1081 batches | lr 30.00 | ms/batch 330.10 | loss 1.56 | ppl 4.78
| epoch 46 | 1000/ 1081 batches | lr 30.00 | ms/batch 331.11 | loss 1.51 | ppl 4.51
-----------------------------------------------------------------------------------------
| end of epoch 46 | time: 385.33s | valid loss 3.87 | valid ppl 47.75
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 47 | 200/ 1081 batches | lr 30.00 | ms/batch 330.38 | loss 1.68 | ppl 5.37
| epoch 47 | 400/ 1081 batches | lr 30.00 | ms/batch 334.35 | loss 1.73 | ppl 5.65
| epoch 47 | 600/ 1081 batches | lr 30.00 | ms/batch 330.04 | loss 1.54 | ppl 4.68
| epoch 47 | 800/ 1081 batches | lr 30.00 | ms/batch 334.15 | loss 1.57 | ppl 4.79
| epoch 47 | 1000/ 1081 batches | lr 30.00 | ms/batch 330.84 | loss 1.51 | ppl 4.54
-----------------------------------------------------------------------------------------
| end of epoch 47 | time: 385.93s | valid loss 3.86 | valid ppl 47.64
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 48 | 200/ 1081 batches | lr 30.00 | ms/batch 332.89 | loss 1.69 | ppl 5.44
| epoch 48 | 400/ 1081 batches | lr 30.00 | ms/batch 329.11 | loss 1.74 | ppl 5.67
| epoch 48 | 600/ 1081 batches | lr 30.00 | ms/batch 332.61 | loss 1.53 | ppl 4.62
| epoch 48 | 800/ 1081 batches | lr 30.00 | ms/batch 329.45 | loss 1.56 | ppl 4.75
| epoch 48 | 1000/ 1081 batches | lr 30.00 | ms/batch 329.09 | loss 1.51 | ppl 4.50
-----------------------------------------------------------------------------------------
| end of epoch 48 | time: 385.58s | valid loss 3.86 | valid ppl 47.53
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 49 | 200/ 1081 batches | lr 30.00 | ms/batch 330.06 | loss 1.67 | ppl 5.33
| epoch 49 | 400/ 1081 batches | lr 30.00 | ms/batch 336.61 | loss 1.73 | ppl 5.65
| epoch 49 | 600/ 1081 batches | lr 30.00 | ms/batch 333.48 | loss 1.53 | ppl 4.63
| epoch 49 | 800/ 1081 batches | lr 30.00 | ms/batch 337.63 | loss 1.55 | ppl 4.70
| epoch 49 | 1000/ 1081 batches | lr 30.00 | ms/batch 334.49 | loss 1.51 | ppl 4.51
-----------------------------------------------------------------------------------------
| end of epoch 49 | time: 385.12s | valid loss 3.86 | valid ppl 47.42
-----------------------------------------------------------------------------------------
Saving Averaged!
| epoch 50 | 200/ 1081 batches | lr 30.00 | ms/batch 332.24 | loss 1.69 | ppl 5.40
| epoch 50 | 400/ 1081 batches | lr 30.00 | ms/batch 329.75 | loss 1.73 | ppl 5.62
| epoch 50 | 600/ 1081 batches | lr 30.00 | ms/batch 334.46 | loss 1.52 | ppl 4.56
| epoch 50 | 800/ 1081 batches | lr 30.00 | ms/batch 335.88 | loss 1.55 | ppl 4.72
| epoch 50 | 1000/ 1081 batches | lr 30.00 | ms/batch 330.69 | loss 1.50 | ppl 4.46
-----------------------------------------------------------------------------------------
| end of epoch 50 | time: 385.40s | valid loss 3.86 | valid ppl 47.31
-----------------------------------------------------------------------------------------
Saving Averaged!
=========================================================================================
| End of training | test loss 3.39 | test ppl 29.81
=========================================================================================
Tip!
Press p or n or
Models
Annotations
Connect to our Discord channel