Defect evolution under irradiation is investigated in a set of single-phase concentrated solid solution alloys (SP-CSAs) containing Ni with Co, Fe and/or Cr. We show that atomic segregation of Ni takes place already at very early stages of radiation damage in the 2–4 element SP-CSAs containing Fe or Cr, well below 1 dpa. We arrive at this conclusion by following the evolution of positron annihilation signals as a function of irradiation dose in single crystal samples, complemented by molecular dynamics simulations in the same model systems for high entropy alloys (HEAs). This manifestation of short-range order calls attention to composition fluctuations at the atomic level in irradiated HEAs. Ion irradiation may induce short-range order in certain alloys due to chemically biased elemental diffusion. The work highlights the necessity of updating the assumption of a totally random arrangement in the irradiated alloys, even though the alloys before irradiation have random arrangements of different chemical elements.