The weather doesn't make a whole lot of difference to the number of people who ride, so long as it's not absolutely horrific. More people rode in Brighton under gloomy skies last year than took part in the gloriously sunny 2014 ride. The weather was about perfect in Canterbury last week and barely 20 people turned up. Sunshine in Southampton yesterday evening and the ride consisted of '20-30 old men.'
The reason for the almost entirely male smaller rides is obvious - women feel intimidated by the inevitable gawpers. Whereas at the bigger rides, there's safety in numbers and anyone overstepping the mark will quickly be dealt with usually.
Smaller rides are increasingly less successful to a point where most are now a waste of time.
Edit in view of post immediately above this one. 20-30 old men and 3 ladies