Looks like WordPress has added Trac support to the Subversion repository that powers the WordPress “Extend” plugin hosting system, so now you’ll be able to track bugs there instead of trying to run your own system.
I’d highly recommend using Trac (or some other actual bug tracking system) as opposed to less formal processes like support forums, blog post comments, etc. as it makes the process of managing your issues much easier.
If you’re concerned about security, though, there is one catch in that I don’t think the extend system supports SSL (https) for subversion or trac access, which means that if you’re managing your plugins from a network you don’t own (work, coffee shop, etc.) your password is up for grabs for whoever wants it, unless you’re tunnelling through another secure channel of course.
For this reason, I’d probably prefer to use my own SVN + Trac server for now, but for a lot of people this is a huge extra feature, if they’ll take the time to learn to use it.