There are three ways to go. The first is an all-in-one camera such as the one suggested above. This means the lens is built into the camera. The advantage is these are typically smaller than a camera with changeable lenses and many of them have longer zoom lenses (eg more telephoto) than what is available on lenses that are bought separately. For a budget solution this is a good way to go. The main disadvantage is that many of them - especially those with long zoom lens ranges - have smaller sensors than a changeable lens camera, which means the picture quality is not as good (escpecially in low light such as indoor exhibits).
The second option is a DSLR with changeable lens. The advantage is supreme image quality and a crisp (non-electronic) viewfinder and the ability to add lenses later. The disadvantage is larger size, additional cost if you start wanting more lenses, and the fact that DSLR's are being phased out in favor of the new mirrorless systems.
The third option, which appears to be the wave of the future (even though I don't like them), is a mirrorless camera with changeable lens. These are usually smaller than a DSLR but bigger than and all-in-one, so kind of a compromise. However the lenses are similar in size to a DSLR so I find the combination unbalanced (eg the lens is too big for the body). Another disadvantage is electronic viewfinder which I find annoying. However, this is the way all manufacturers are going and DSLR's are on the way out, whether I like it or not (I don't).