Brugmansia are native to South America and all parts of the plant are poisonous. I don't think it's dangerous to grow toxic plants. It's just good to know not to eat them.
Because it is native to tropical climates, it is frost tender. In the Los Angeles area this winter we had several bouts of extreme cold. I missed the first night and woke up to burned leaf edges. I covered the plants with sheets the next few nights. It was amazing how quickly they recovered. About a month later we had severe hail that shredded the foliage. Again the plants recovered.
Angel's Trumpets come in white, pink and yellow. Fast growing in sun or shade, it is a wonderful plant.