some more searching I think this is Tobacco Bush. The leaves are narrower than some other photos out there, everything else seems to match though. Julia’s comment I also think this is a Flannel leaf! When searching through Google images I still think tobacco bush (Solanum mauritianum), and the native flannel leaf (Austrotricha floccosa) are remarkably similar looking but the Flannel leafs description of a shrub 2 – 4m tall matches more closely with what I saw.
It was running wild in an area by Middle Harbour River, all the plants looked young and the same height though, as if they’d all sprouted within the last few weeks.
It is an introduced species from South America. Flannel leaf is a native species found in sheltered gullies near streams, common on sandstone.