Sure, but not with HTML. Unless you use frames. And you don't want to use frames.
It's easily done server side. Earlier SSI, Server Side Includes, that's available on Apache was much used and is still perfectly fine to use. It isn't a programming language per se, more a collection of server directives that can be embedded in the HTML.
Nowadays it's more common to use PHP, which is a programming language. The advantage is that if you later want to use other PHP features you can just go ahead. If you start with SSI and want to add PHP later you would have to redo your SSI included and change them to PHP ones.
It all depends on what you want to do and what's available to you on the server. If you are at a free host that maybe don't offer any server side features there are preprocessors that run locally on your machine and basically do what SSI does on the server. The difference is that if you update your included menu or footer you need to run the preprocessor again and upload the changed files to the server.