Why do duplicate NAT log lines appear, and what do they cost?

Duplicates arrive from a rule that fires on every packet, from two devices watching the same traffic, from a router reporting to two places that share one store, or from records being loaded twice. Each has a different signature, and the fix for one makes no difference to the others.

What does a duplicate actually look like?

A run of lines describing one connection, separated by fractions of a second, with nothing changing between them. The values are identical because they describe the same mapping, and the only thing moving is the clock.

Compare that with two genuine connections from one household, which differ in the source port at minimum, usually in the destination as well, and are spaced by however long it took somebody to click something.

The distinction matters because the two look similar in a row count and completely different in a search. A person scanning results sees repetition and assumes the tool is broken. It is reporting exactly what it received.

Why does a rule end up firing on every packet?

Because logging was attached to a rule that examines all traffic rather than one that only notices connections beginning. A translation gets settled as the connection is set up, and then covers everything that follows, so a rule that logs the ongoing traffic writes the same decision out again and again.

The volume this produces is not a modest increase. A single file download can involve thousands of packets, all belonging to one mapping, and each one produces a line identical to the last. One busy customer can generate more records in an hour than the entire network should produce in a day.

What makes this hard to notice is that everything works. Records arrive, they are correct, and searches return them. The only symptom is volume, and volume looks like growth to anybody who was expecting the network to grow.

The signature is the timestamp. Where repeated lines share a second, or differ by hundredths, the rule is watching traffic rather than connections. Nothing else produces bursts that tight.

Which duplicates come from having more than one device in the path?

The ones that arrive from different sources describing the same event. Networks with a core and an edge, or with a failover pair, often have two devices that both see the traffic and both have been configured to report it, usually years apart by different people.

These read differently from the packet by packet kind. The values match, the timing is close but not identical, and the device name in the record differs. That last field is what tells you which situation you are in, and it is the first thing to look at when duplicates appear without any rule having changed.

Failover pairs deserve their own mention because they produce duplicates only sometimes. Where both members are active, both report. Where one is standby, records appear from one device until a failover, then from the other, and an operator who cleaned up believing one device to be authoritative has quietly discarded the period covered by the other.

A related pattern turns up on networks that translate in two places. Here the lines are not copies at all, because each device records a different mapping for the same traffic, and merging them into one store produces something that reads like repetition while actually describing two separate steps. Treating those as duplicates and removing one destroys half the chain.

What creates duplicates outside the routers entirely?

Two arrangements, both of them administrative rather than technical. The first is a router reporting to two destinations that turn out to write into the same store, which happens after a consolidation nobody documented.

The second is records being loaded twice. Somebody restores a backup to check it, imports a file that was already imported, or runs a recovery after a failure without knowing which period had already been written. This produces perfect duplicates spread across a wide time range rather than in bursts, which makes it the hardest kind to spot.

There is a third that is not really duplication but doubles the count all the same. Some arrangements record both the opening and the closing of a mapping, which is two rows describing one connection. That is deliberate and useful, and it should be understood before anybody starts counting rows and drawing conclusions.

How do you tell repetition from two real connections?

By the source port, and then by the clock. Two connections from one device will not share a source port at the same moment, because the whole purpose of that number is to keep them apart.

Where every value including the port matches and the times are within the same second, it is one connection recorded repeatedly. Where the port differs, they are separate connections regardless of how alike everything else looks. Where the port matches but the times are minutes apart, the port has been recycled and these are two different connections, possibly from two different customers.

That last case is the one that punishes careless cleanup, and it is why any rule for removing repetition has to treat the timestamp as part of the identity rather than as a detail attached to it.

One check settles most of these questions quickly. Pick a single busy customer, look at everything recorded for them across five minutes, and count how many distinct source ports appear against how many rows. On a healthy network those two figures stay close. Where rows outnumber ports many times over, the repetition is in front of you.

What do duplicates cost beyond space?

Space is the least interesting of the four costs, and the only one anybody measures.

What repetition takes from you

The cost	How it shows up	Who notices first
Space consumed	The store fills far sooner than the customer count suggests	Whoever pays for disks
Retention shortened	How far back you can go quietly contracts	Nobody, until a request lands outside it
Searches slowed	Every query works through rows that add nothing	Support staff, who assume it has always been slow
Counts made meaningless	Sessions per subscriber and busy hour figures come out inflated	Whoever plans capacity from those numbers

The fourth row is worth more attention than it usually gets. Decisions about address space and equipment get made from these counts, and repetition inflates them unevenly, since it affects heavy users far more than light ones. The result is a plan built on a distortion that grew with the very customers it was meant to model.

Can they be removed safely afterwards?

Yes, carefully, and only where every field including the moment matches exactly. That rule is conservative on purpose, because a slightly looser one starts merging genuine connections that happen to resemble each other.

The safer sequence is to fix the source first and clean up second. Cleaning while the cause is still running is repetitive work with no end, and it removes the evidence you would use to confirm the fix worked.

Keep a count of what was removed, per day, alongside the records. If anybody later questions why one period is thinner than its neighbours, the answer is a number rather than a recollection, and that difference matters more than it sounds.

What stops them coming back?

A number you check rather than a setting you trust. Records per subscriber per day is a figure that stays roughly stable on a healthy network, and any change in it is either genuine growth or something repeating.

Beyond that, three habits cover the causes. Look at the reporting device field occasionally to confirm that only the devices you expect are sending. Note which destinations each router reports to, in writing, so a future consolidation cannot merge two streams by accident. And treat any bulk load of historical records as an event that needs checking afterwards rather than a routine action.

None of that is difficult. It is simply work that nobody assigns, because duplicates do not cause an outage, do not raise an alarm and do not stop anybody's connection working.