The fuel itself won't stay radioactive as you point out. There is however neutron radiation produced while the reactor is running. This necessitates shielding for components and personnel.

Additionally there's a phenomenon called neutron activation, where a non-radioactive substance absorbs neutrons and becomes a slighter heavier isotope that is radioactive. For something like a research vessel with relatively little operation time and low fluxes, this isn't a major concern. For power generating reactors with high flux over long periods of time, this will make some reactor components radioactive. That means servicing and decommissioning fusion reactors will still require protocols to monitor and control contamination.

https://en.wikipedia.org/wiki/Neutron_activation

Since there's no risk of meltdown, none of this poses a risk to the general public. So fusion is indeed much safer than fission, but you can't just say there's 100% no radiation. That is erroneous.

An x-ray machine doesn't stay radioactive when not in use, but it still produces ionizing radiation when it's on. Fusion power is similar. The radiation produced mostly goes away instantly, but is definitely present during operation.

https://www.iaea.org/newscenter/news/neutrons-blast-fusion-materials-in-new-iaea-project