victorian (& modern) uniforms with whistle seem to generally have the lanyard chain attached to a boss on the cross belt and the whistle holder is on the lower part of the cross belt rather than the waist belt. the pattern seems pretty much the same between regiments and over the last century anyway.
modern:
more ethnic victorian one:
another:
closeup of boss, chain, whistle and holder.
thunderer whistle:

these are the whistles we normally see nowadays, usually in plastic.
these have replaced the older style tubular whistles of yore.