Finally had some luck in spotting one after ten minutes of waiting. This one was just sticking it's head out of the foliage beforehand. Slowly, it walked over to the shelter (just out of shot) followed by another. I left to go check out the next door Baboons for just a few minutes and came back to find both had disappeared yet again!
