I've got two contemporary books that discuss it, "The World of Zoos" (1968) and Geoffrey Schomberg's "The Penguin Guide to British Zoos" (1970). By the time of Anthony Smith's "Animals on View" (1977) the collection remained only as an Aquarium (due to the opening of Blackpool Zoo in 1973).
Anyway, the Aquarium had existed since 1875 (the construction of the Tower itself though was only in 1891, and it opened to the public in 1894). The zoo part started when a collection of animals was introduced for the opening of the Tower. By the date of publication of the books mentioned above, the animals were mainly monkeys, small cats and various other small mammals and tropical birds, although there were also (in 1970) lion, tiger, brown bear, Asiatic black bear, dingo and spotted hyaena. The 1968 book says the total collection of zoo and aquarium at that time was 360 species and 3170 specimens (mainly fish I suspect). In contrast to the 1970 book the 1968 one says that there had been no large mammals (bears, tigers etc) there since 1955, so it is possible that in the intervening years more large mammals were acquired.
I would say that it would be impossible for there to have been a giant panda at the Tower zoo. Perhaps it was just one of the bears, or perhaps there was confusion with London Zoo?