The Lamb shift, an energy shift arising from the presence of the electromagnetic vacuum, has been observed in various quantum systems and established as part of the energy shift independent of the environmental photon number. However, typical studies are based on simplistic bosonic models which may be challenged in practical quantum devices. We demonstrate a hybrid bosonic-fermionic environment for a linear resonator mode and observe that the photon number in the environment can dramatically increase both the dissipation and the effective Lamb shift of the mode. Our observations are quantitatively described by a first-principles model, which we develop here also to guide device design for future quantum-technological applications. The device demonstrated here can be utilized as a fully rf-operated quantum-circuit refrigerator to quickly reset superconducting qubits.