[lug] Dropped packet question

David Frye dafr at dafr.us
Fri Sep 27 11:14:57 MDT 2013 

Looks like your packet reassembly is a really hight number, and I'm not sure what is causing the packets to arrive broken. The number of reassembled packets is about 2.5 times the total number of incoming packets, so that's a lot of extra overhead for the device.

As others have suggested, check the duplex on the interface (mii-tool) and see what that gets you.


On Sep 26, 2013, at 9:25 AM, Chip Atkinson wrote:

> Wow.  I've never used that.  I'll paste in below what it shows.  I see
> things like outgoing packets dropped, reassemblies, fragments, etc.
> What should I be looking for in particular?
> Oddly too, it looks that eth1 and eth0 have the same numbers or very close
> to the same (since it's actively sending and receiving traffic)
> 
> chip1:/var/log # netstat -s eth0
> Ip:
>    18528785 total packets received
>    61576 forwarded
>    0 incoming packets discarded
>    18229593 incoming packets delivered
>    18372091 requests sent out
>    560 outgoing packets dropped
>    47913643 reassemblies required
>    15979146 packets reassembled ok
>    31957472 fragments received ok
>    47912823 fragments created
> Icmp:
>    82067 ICMP messages received
>    126 input ICMP message failed.
>    ICMP input histogram:
>        destination unreachable: 24598
>        timeout in transit: 1167
>        echo requests: 37000
>        echo replies: 19302
>    45902 ICMP messages sent
>    0 ICMP messages failed
>    ICMP output histogram:
>        destination unreachable: 8902
>        echo replies: 37000
> Tcp:
>    90757 active connections openings
>    74760 passive connection openings
>    631 failed connection attempts
>    1725 connection resets received
>    6 connections established
>    1488263 segments received
>    1565368 segments send out
>    20421 segments retransmited
>    9 bad segments received.
>    100442 resets sent
> Udp:
>    16653309 packets received
>    5151 packets to unknown port received.
>    0 packet receive errors
>    16680883 packets sent
> TcpExt:
>    145 invalid SYN cookies received
>    29 resets received for embryonic SYN_RECV sockets
>    ArpFilter: 0
>    28479 TCP sockets finished time wait in fast timer
>    142 packets rejects in established connections because of timestamp
>    25313 delayed acks sent
>    61 delayed acks further delayed because of locked socket
>    Quick ack mode was activated 1483 times
>    54672 packets directly queued to recvmsg prequeue.
>    366613 packets directly received from backlog
>    1968664 packets directly received from prequeue
>    178056 packets header predicted
>    40329 packets header predicted and directly queued to user
>    TCPPureAcks: 379540
>    TCPHPAcks: 451987
>    TCPRenoRecovery: 5
>    TCPSackRecovery: 897
>    TCPSACKReneging: 7
>    TCPFACKReorder: 58
>    TCPSACKReorder: 0
>    TCPRenoReorder: 3
>    TCPTSReorder: 48
>    TCPFullUndo: 91
>    TCPPartialUndo: 499
>    TCPDSACKUndo: 51
>    TCPLossUndo: 1027
>    TCPLoss: 552
>    TCPLostRetransmit: 0
>    TCPRenoFailures: 0
>    TCPSackFailures: 439
>    TCPLossFailures: 43
>    TCPFastRetrans: 1600
>    TCPForwardRetrans: 284
>    TCPSlowStartRetrans: 398
>    TCPTimeouts: 6235
>    TCPRenoRecoveryFail: 0
>    TCPSackRecoveryFail: 155
>    TCPSchedulerFailed: 0
>    TCPRcvCollapsed: 0
>    TCPDSACKOldSent: 1472
>    TCPDSACKOfoSent: 0
>    TCPDSACKRecv: 2102
>    TCPDSACKOfoRecv: 5
>    TCPAbortOnSyn: 0
>    TCPAbortOnData: 3261
>    TCPAbortOnClose: 1081
>    TCPAbortOnMemory: 0
>    TCPAbortOnTimeout: 950
>    TCPAbortOnLinger: 0
>    TCPAbortFailed: 0
>    TCPMemoryPressures: 0
> 
> 
> On Fri, 27 Sep 2013, David Frye wrote:
> 
>> What does 'netstat -s <interface>' show?
>> 
>> If packets are being dropped, it may have additional information there not shown with ifconfig.
>> 
>> 
>> On Sep 26, 2013, at 8:53 AM, Chip Atkinson wrote:
>> 
>>> Thanks.  I'm not seeing errors or dropped packets in ifconfig, which is
>>> kind of weird, isn't  it?  If ping reports dropped packets, wouldn't that
>>> droppage appear in the output of ifconfig?  
>>> 
>>> On Fri, 27 Sep 2013, Dan Ferris wrote:
>>> 
>>>> Start with something easy.  Check ifconfig and see if there are errors 
>>>> on the interface.  If so, then start by checking hardware. You could 
>>>> have a bad cable, bad nic, bad switch port, or a duplex mismatch.
>>>> 
>>>> Dan
>>>> 
>>>> On 9/27/2013 9:31 AM, Davide Del Vento wrote:
>>>>> Since you control the server, don't the logs tell you something about
>>>>> the dropped packets? Since you don't see drops with the netbook, you
>>>>> can rule out the rest of the network: it must be the server box.
>>>>> 
>>>>> It may be dropping packets for a variety of reasons, just to mention a
>>>>> couple of stupid ones: a defective network card or too high CPU load.
>>>>> 
>>>>> Cheers,
>>>>> Davide
>>>>> 
>>>>> On Thu, Sep 26, 2013 at 8:48 AM, Chip Atkinson <chip at pupman.com> wrote:
>>>>>> Greetings all,
>>>>>> 
>>>>>> Due to the recent flooding I had to change data centers from my parents'
>>>>>> basement to mine, which resulted in re-doing my network.
>>>>>> 
>>>>>> Now that I've moved and re-IPed the server, I'm seeing large numbers of
>>>>>> dropped packets, slow ping times, basic network malaise.  I've been
>>>>>> running a series of 100 pings 5 sec apart and then looking at the reported
>>>>>> loss figures.
>>>>>> 
>>>>>> With comcast's help, I believe that we've eliminated them and their
>>>>>> hardware.
>>>>>> 
>>>>>> I put a small linux netbook on the network in place of the server and was
>>>>>> able to ping it from outside (vpn to work and out from there) and the
>>>>>> ping response time and dropped packets were basically gone.  Besides being
>>>>>> newer hardware and OS, the netbook had no services (web, dns, email).
>>>>>> 
>>>>>> I then connected the server and see the dropped packet and slow ping time
>>>>>> issue again.
>>>>>> 
>>>>>> I was using tcpdump and noticed that a large portion of the traffic is DNS
>>>>>> lookups:
>>>>>> 
>>>>>> 08:42:23.411809 IP (tos 0x0, ttl  64, id 42252, offset 0, flags [+],
>>>>>> length: 1500) 173.14.7.2.53 > 108.174.149.7.2305:  13490| 250/0/1
>>>>>> bitstress.com. SOA[|domain]
>>>>>> 08:42:23.411817 IP (tos 0x0, ttl  64, id 42252, offset 1480, flags [+],
>>>>>> length: 1500) 173.14.7.2 > 108.174.149.7: udp
>>>>>> 08:42:23.411822 IP (tos 0x0, ttl  64, id 42252, offset 2960, flags [none],
>>>>>> length: 1150) 173.14.7.2 > 108.174.149.7: udp
>>>>>> 
>>>>>> Googling found this:
>>>>>> http://dnsamplificationattacks.blogspot.com/2013/09/domain-bitstresscom.html
>>>>>> 
>>>>>> My question is whether or not the dns traffic could be responsible for all
>>>>>> the dropped network packets or should I start looking elsewhere for the
>>>>>> problem?
>>>>>> 
>>>>>> I switched network interfaces and took the original server network
>>>>>> interface off the network, thinking that it could be broadcasting a bunch
>>>>>> of noise but still am seeing packet losses, though perhaps not as severe.
>>>>>> 
>>>>>> 
>>>>>> Thanks in advance for any insight and help.
>>>>>> 
>>>>>> Chip
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Web Page:  http://lug.boulder.co.us
>>>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>>>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>>>>> _______________________________________________
>>>>> Web Page:  http://lug.boulder.co.us
>>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>>>> 
>>>> _______________________________________________
>>>> Web Page:  http://lug.boulder.co.us
>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>>>> 
>>> 
>>> _______________________________________________
>>> Web Page:  http://lug.boulder.co.us
>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>> 
>> 
>> --
>> David
>> dafr at dafr.us
>> 
>> _______________________________________________
>> Web Page:  http://lug.boulder.co.us
>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>> 
> 
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety


--
David
dafr at dafr.us

More information about the LUG mailing list