I'd like to be able to delete all albums / categories at the same time through the admin interface.
Deleting all categories and albums with one click is something that will hardly be necessary for the majority of users. Therefor, it won't be added to the core. It would only end up with a lot of support requests from curious users who tried that and wonder how they could undo it. As bmossavari suggested, this is a trivial thing to do in phpMyAdmin. However, there's no support on the feature requests board (as suggested in the sticky thread).
