Caulk in a tube is pretty permanent. I wouldn't caulk between the unit and the window frame because it could be a pain to remove in the future. There are putties that work pretty well and are easily removable in the future.
Mortite makes a decent one. They call it "caulking cord" but it's just a thin string of putty you can press into gaps and cracks. It won't stick to dust, so you have to wipe down the surfaces beforehand and let them dry if you used a damp cloth. You can fill larger gaps by pressing more of it in place. It's like working with clay.