I know guys who don't have a set routine, but they intuitively hit all of the points you raised. I find having a routine or a program makes things a LOT easier, both in working at the gym and in tracking my progress.
For those who can afford it, I'd recommend getting a personal trainer. I started with a trainer after about a year of training on my own, and it made a huge difference: my form improved, I saw more progress, and I learned a shitload of useful info about how muscle grows.
But at the end of the day: consistency, progressive overload, diet, rest. That's all you need.