This issue can occur when the same appkey is used for end-devices.
The remedy is to use a unique key for each end-device.
Or add a confirmed uplink that must be responded to following the join accept. Only the intended device will create the correct session keys using its random devnonce.
Of course multiple end-device could randomly select the same devnonce for the join request and create the same keys.