Groups are calling for greater consultation on the future of the trees in Wisbech Market Place.
Wisbech Town Council wants to alter the Market Place to improve events which are held there.
The council is planning to remove the central plinth to ensure the area is no longer split and has also been looking at the eight London Plane trees on the site.
But this has sparked some controversy.
Wisbech Society is calling for the council to “kill off” any proposal to remove the trees.
Society chairman Ray Johnson is calling for a consultation to be held on the plans.
He said: “We want the town council to withdraw any attempt to remove the trees.
“The trees have been in place for 25 years and there is nothing wrong with them and they are a nice addition to the Market Place.
“What they should be doing is holding a public consultation on whether people want to leave the trees in the Market Place.”
Leader of the town council David Oliver had a meeting on Monday with Fenland District Council’s tree officer to discuss the plans.
He said: “We met with the conservation officer and had lengthy discussions about the trees.
“We are going to take it back to our Markets Committee so they can make a decision on how to take it forward.
“Whatever the plans are, we will hold a consultation with the public and traders.”
Cllr Oliver said the current layout of the Market Place is split and does not help visitors to see everything which is happening at events.
He said: “All the events we put on are for the benefit of the town and we try to encourage retailers to be open when these are held.”