Online
Online Training¶
Online training, also called 'test time training' occurs during generation to enables model parameters to dynamically evolve as they see additional samples after base training occurs. This differs from in-context learning where models can adapt to new tasks without parameter changes by updating prompts with examples or improved instructions.
The Surprising Effectiveness of Test-Time Training for Abstract Reasoning
The authors show in their paper
Online-LoRA
The authors provide code and show in their paper a manner of enabling online-lora to finetune pre-trained Vision Transformers (ViT)s in real time, thereby addressing the issues with reherasl buffers. They use dynamic loss values to ensure automatic recognition of data distribution shipfts and a novel online weight regularization strategy for combining different model parameters.
