If I remember correctly it is a real concern and also part of why developing a bacteria that can break down plastic is very dangerous (not just because it could degrade our tools and infrastructure, but also because it will release literally megatons of CO2 into the atmosphere)

Fyi, in the past there was nothing in nature to break down trees (lignin) and it actually was a problem as they would literally pile up and essentially be the same problem we have today with plastic (ironically it caused global cooling) https://www.thorogood.co.uk/treevolution-how-trees-came-first-and-rot-came-later-in-earths-deep-past/