I believe the strange presence of the crosses and the stained glass frames are meant to evoke the image of a church or chapel. According to signage at the park one of the original abbey chapels was found close to this location and the ruins of the old abbey church aren't far from here either.