Feature Request: display_interval per plot line instead of per graph

[paulguerrero]

First of all, thanks for this interesting alternative to Tensorboard.

One feature I am still missing is the ability to set a display_interval per plot line instead of per graph. In my training runs, I usually run the validation steps less frequently than training steps. It seems the display_interval however, is over actual samples given to Losswise, not over the iteration number passed in the x parameter.

So, for example, if I pass validation samples at x values (100, 200, 300, ...), and set display_interval to 100, then the validation loss plot will only update in x value intervals of 10000 instead of 100, which is longer than I would like. Lowering display_interval makes the training samples too noisy, since I pass training samples at x values (1, 2, 3, ...). So the training and validation values are effectively averaged over largely different interval sizes in x (although over the same raw number of samples).

My current work-around of putting training and validation in different plots with different display_intervals is not ideal either, because x-axis of the plots is never quite synchronized.

One way this could be solved would be to allow an option to average over same intervals in x instead of same number of raw samples, or an alternative could be to allow setting different display_intervals for different plot lines in a Graph.

[nicodjimenez]

Thanks for the kinds words on Losswise

It seems the display_interval however, is over actual samples given to Losswise, not over the iteration number passed in the x parameter.

If that's a case, that's a bug. Losswise should only perform averaging when the difference in consecutive x-values is 1, eg. for the training loss:

https://github.com/Losswise/losswise-python/blob/master/losswise/__init__.py#L161

Any idea what might help resolve this bug?

I will take a harder look at the code next week regardless.

[paulguerrero]

Sorry I misinterpreted the issue then. The real issue seems to be that I don't pass the x values for my validation graph as multiples of the display_interval. The issue is then in lines 158-159:

if self.max_iter is not None and x < self.max_iter - 1 and x % self.display_interval != 0:
    return

If I read the code correctly, if the x are not passed as multiples of the display_interval, it will be stored in tracked_value_list, but never be added to the plot. So instead, something like this could be used:

if self.max_iter is not None and x < self.max_iter - 1 and x - self.last_plotted_x < self.display_interval:
    return