With ABP the session keys are stored in flash. With OTAA the session keys are renewed by a device reset (at the join procedure). The key stream (for both encryption of the payload as generation of the MIC Message Integrity Code) are derived using the session keys and the frame counter.
Now consider a device reset:
• The framecounter (FCntUp) is reset to 0
• The session keys will stay the same for ABP (stored in Flash)
As the key stream is derived from both values above, it will be identical after each reset. LoRaWAN has a mechanism that it only accepts uplink messages with a higher frame count value (in order to prevent the acceptance of older messages being replayed).
Now as the frame counter is reset to zero, an attacker could replay a (previously recorded) message with a higher frame count which leads to an acceptance. While the device itself holds lower frame counts, any subsequent messages from the device will not be accepted anymore.
Other than that, eavesdropping will be possible because after reset, the key stream will be the same for each individual subsequent message as before the reset. This may lead to recognition of the key stream for each individual message.
Basically, shipping out devices with ABP keys is the same as shipping out devices with OTAA which are already joined to a network. The advantage of shipping out devices with OTAA keys is that the devices will be able to re-join and receive new session keys which prevents the possibility on the above mentioned replay attacks.