Most definitely chimpanzees. I say this anecdotally and from what others who have worked with them have told me as thankfully I haven't had much personal experience with these and frankly I dont ever want to either. I don't particularly like them as animals or find them interesting at all.
A close second to the chimp from what I've observed would be male Hamydryas baboons which can behave very aggressively both towards females of their species and also to their keepers. There is one particular captive baboon that I remember which (this might be an anthromorphism on my part) almost seemed to me to be a bit of a psychopath.
Among the New world world primates , none from what I've observed have a particular propensity towards aggression towards people, though every individual will have a different temperament based on its life experience, and all will attack if threatened.
Spider monkeys can be a bit boistrous and play rough (one once tore a tuft of my hair out while playing) but are typically very amiable animals and build strong bonds with keepers.
Woolly monkeys generally the same as spider monkeys but some of the temperament is "howler monkeyish" in terms of being a bit more easy going and affable.
Howler monkeys are usually too lethargic to be aggressive and typically have very easy going and relaxed characters and form bonds easy.
Capuchin monkeys are a bit temperamental , jittery, and highly strung particularly the alpha males but as long as the alpha doesn't take a disliking to you/ feel threatened then so will the group and you are fine.
Muriquis from what I've seen live up to their reputation as being the "hippy" monkey and I haven't even seen any aggressive behaviour in them towards eachother let alone towards people (though some individuals definitely have favourite keepers and show passive aggressive behaviour towards those they dont like).
Saki and squirrel monkeys , in my experience they are kind of a bit more shy, standoffish and wary/tenative with people.
Titi monkeys , similar to saki monkeys in terms of being mistrustful and standoffish and easily alarmed.
Owl monkeys , shy and easily alarmed, not aggressive at all (from what I've observed), good natured.
This is very interesting!! Anything else you could tell us about the primates you worked with?