Bright red berries produced in the fall close along the stems of a deciduous shrub, most often in damp areas in the Eastern US, is Winterberry - Ilex verticillata.
Yes some ilex, ie hollies, are deciduous.
Many cultivars with larger and more abundant berries are now available of this native shrub. They are commonly used in weaths and Christmas plant arrangements. Still, the plant looks best in the woods and swamps.