The specification filter groups are sorted by display order and the options in each group are sorted by display order too.
If you want to reorder the filters you need to change the display order of the groups or the options in them (clear your cache after each change in ordering to see the changes).