The use of synthetic training data such as interpolated sequence values is common, but fraught with the issue of your synthetic signal generating features that are not true to real life. 

Instead, you might think about about appending the values of two long-period sin waves per input element to the sequence going in to your first linear / fully-connected layer. The simplest thing that could possibly work would be to interpret minute of the day and day of the month as the values of your sin wave! (Or perhaps minute/hour if all training sequences are quite short.) Since you’re doing sequence prediction, you would add the appropriate values for each image(?) in the sequence being evaluated.   

With that additional signal going in to the model in the early layers, the NN should have a good chance of learning that the differences in the modulating signal corresponds to distance in time. 

This technigue was popularized by early Large Language Models to encode the distance between words. There’s been refinement (search “Rotary Encoding” for instance) but the basic idea of sin waves generalizes well.