GPT4All is all you need.
Easy to use, and to update. Everything is simply one click away, available for Linux, macOS, and Windows.
If you want more powerful stuff, there is oobabooga's text generation web ui, which is a web based application that you could run on your server.