Loss without regularization

[Likelihoood]

[VinceShieh]

regularization is done in the weights update

[Likelihoood]

Yes, I know it is done in weights update.
I mean that after each iteration the printing loss does not contain reg loss( r1 * ||w|| + r2 * ||v|| ).
I add a function to calculate reg loss:

and after each iteration, print the reg loss

But...
Due to the weights being updated in each sample of each iteration, the weight which are used to cal reg loss are not the same one as the beginning of the iteration.
Actually the tr_loss has the same situation.

Please help consider it .

[VinceShieh]

hmm, I suggest that, 1. make a toy sample (ex. 3 samples with 5 feature) 2. do not add regularization 3. print the weights at the beginning of each iteration and right after each update, IN THE SAME FUNCTION CALL 4. add one-way regularization and go to step 3 and check again.