If you create a set which isn't checked as active when you create a button the modal window doesn't close on save and refresh the button grid. So you have to then click close and manually refresh the grid for the button to display. So initially it doesn't look as if the button was created.
Where as if you perform the same actions on a set that's active, the modal window closes when saved and the grid refreshes displaying the new button.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.