Do It Yourself

The premise behind most to all insulation is to prevent air movement as much as possible with as little material as possible. For windows, this usually means more panes, which means more trapped layers of isolated air (or other gases). The best bang for your buck for garage door windows is window insulation film. You can buy it in a number of stores. If you have a single pane window in your garage door, you have almost no insulation at all for the window. Double pane would be better, but will still let a fair amount of heat transfer. Given the small window size, you may want to try putting the film on the outside as well as the typical inside layer, giving an additional layer of insulating air.

The other thing about windows is they are poorly insulated objects put in a hole punched through better-insulated objects (the garage door). Sometimes the hole the window is put in isn't properly sealed, and that again allows air flow. Check for any air flow around the windows and caulk them before installing the film. This may require removing the trim and removing any old or decayed caulking before applying the new caulking.

If you in fact have single-pane windows in your garage door, you may want to see about getting double-pane windows instead. This won't be cheap, and may require modification or replacement of the trim for the windows. The windows are likely double-pane, though, and even if the seal is worn out the improvement from a new set of double-pane windows won't be much.