Echo issues even on VoIP-only calls

Discussion in 'General' started by oldrags, Feb 15, 2011.

  1. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    hi there

    I'm experiencing echo problems on an Elastix System. I've been searching on the Internet for answers but I can't seem to find something that can be applied to my case.

    Echo appears and disappears within calls, and it's hard to reproduce. I've never experienced it myself, but my customers complain about it a few times per week.
    I first thought the far ends were causing the problem, but now I get reports that it happens even on internal calls, for example between two SNOM 190 VoIP phones, using the G.911A codec. (I'm in Switzerland). The echo apparently comes and goes within a single call.

    Some info about the system:

    The hardware is an Asus eeepc laptop with an Atom single core processor and 1Gig of ram. I know it's not a resource issue, I've been monitoring that for a while now. I thought maybe it has to do with the clock accuracy... ? (some measurements at the bottom of this post)

    [root@lotertux ~]# asterisk -V
    Asterisk 1.6.2.13

    Versions:
    Kernel
    Linux(i386)-2.6.18-194.3.1.el5

    Elastix
    elastix-2.0.0-58
    elastix-reports-2.0.0-20
    elastix-addons-2.0.0-19
    elastix-asterisk-sounds-1.2.3-1
    elastix-vtigercrm-5.1.0-8
    elastix-firstboot-2.0.0-14
    elastix-system-2.0.0-38
    elastix-fax-2.0.0-18
    elastix-email_admin-2.0.0-23
    elastix-agenda-2.0.0-24
    elastix-a2billing-1.3.0-4
    elastix-pbx-2.0.0-41

    RounCubeMail
    RoundCubeMail-0.3.1-5

    Mail
    postfix-2.3.3-2.1.el5_2
    cyrus-imapd-2.3.7-7.el5_4.3

    IM
    openfire-3.5.1-2

    FreePBX
    freePBX-2.7.0-10

    Asterisk
    asterisk-1.6.2.13-0
    asterisk-perl-0.10-2
    asterisk-addons-1.6.2.1-0

    FAX
    hylafax-4.3.9-0rhel5
    iaxmodem-1.2.0-1.1

    DRIVERS
    dahdi-2.3.0.1-3
    rhino-0.99.3-2.beta2
    wanpipe-util-3.5.14-0

    I've read in some places that this can have to do with timing, so here some measures about that:
    [root@lotertux ~]# dahdi_test -vv -c 30
    Opened pseudo dahdi interface, measuring accuracy...

    8192 samples in 8192.824 system clock sample intervals (100.010%)
    8192 samples in 8191.224 system clock sample intervals (99.991%)
    8192 samples in 8191.808 system clock sample intervals (99.998%)
    8192 samples in 8223.800 system clock sample intervals (100.388%)
    8192 samples in 8191.768 system clock sample intervals (99.997%)
    8192 samples in 8159.760 system clock sample intervals (99.606%)
    8192 samples in 8191.784 system clock sample intervals (99.997%)
    8192 samples in 8191.792 system clock sample intervals (99.997%)
    8192 samples in 8191.792 system clock sample intervals (99.997%)
    8192 samples in 8191.776 system clock sample intervals (99.997%)
    8192 samples in 8191.752 system clock sample intervals (99.997%)
    8192 samples in 8191.784 system clock sample intervals (99.997%)
    8192 samples in 8191.240 system clock sample intervals (99.991%)
    8192 samples in 8191.800 system clock sample intervals (99.998%)
    8192 samples in 8224.424 system clock sample intervals (100.396%)
    8192 samples in 8159.712 system clock sample intervals (99.606%)
    8192 samples in 8191.832 system clock sample intervals (99.998%)
    8192 samples in 8192.832 system clock sample intervals (100.010%)
    8192 samples in 8190.736 system clock sample intervals (99.985%)
    8192 samples in 8183.784 system clock sample intervals (99.900%)
    8192 samples in 8191.776 system clock sample intervals (99.997%)
    8192 samples in 8191.792 system clock sample intervals (99.997%)
    8192 samples in 8191.240 system clock sample intervals (99.991%)
    8192 samples in 8191.920 system clock sample intervals (99.999%)
    8192 samples in 8215.615 system clock sample intervals (100.288%)
    8192 samples in 8191.913 system clock sample intervals (99.999%)
    8192 samples in 8191.704 system clock sample intervals (99.996%)
    8192 samples in 8191.800 system clock sample intervals (99.998%)
    8192 samples in 8184.328 system clock sample intervals (99.906%)
    8192 samples in 8191.808 system clock sample intervals (99.998%)
    --- Results after 30 passes ---
    Best: 99.999 -- Worst: 99.604 -- Average: 99.928031, Difference: 100.000863
    [root@lotertux ~]#

    Let me know if you need any additional information.
    Thanks in advance for your help!
     
  2. jgutierrez

    Joined:
    Feb 28, 2008
    Messages:
    5,737
    Likes Received:
    0
    Try setting this crontab:
    crontab -e
    3 * * * * sync; echo 3 > /proc/sys/vm/drop_caches

    It will clear your cache every hour, that will prevent for echo on internal calls
     
  3. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    also your atom micro cpu maybe enter low voltage and low consume mode and this could cause real time processing to be almost impossible.
    Remember that this low cost low power low mhz systems could not handle moren tha 10 20 calls...
     
  4. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Hello

    The maximum that happens on this system is a 6-call conference, I hope performance isn't a problem.

    I'll first give your idea a shot, and if that doesn't help, I'll go check on the machine's bios about cpu power save stuff.

    I'll post back if it worked.

    Muchas Gracias!
     
  5. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Hello!

    I tried this idea (set it up yesterday):
    crontab -e
    3 * * * * sync; echo 3 > /proc/sys/vm/drop_caches

    and this morning I get a report that the echo is still there. i'm using call monitoring to record the calls, and on there, no echo at all can be heard, call is fluid. ???
    would it make sense to clear the cache more than every hour?

    or should I start looking into those processor issues? I don't quite see how slow performance would induce echo. I have been monitoring system resources in the past, and it never seemed to be a problem. Does that make sense to you?

    what do you think?

    Christoph
     
  6. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    what is your cpu use on that mini system??? try to give real stats.
     
  7. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    this is the result of less /proc/cpuinfo
    processor : 0
    vendor_id : GenuineIntel
    cpu family : 6
    model : 28
    model name : Intel(R) Atom(TM) CPU N270 @ 1.60GHz
    stepping : 2
    cpu MHz : 1596.103
    cache size : 512 KB
    physical id : 0
    siblings : 1
    core id : 0
    cpu cores : 1
    apicid : 0
    fdiv_bug : no
    hlt_bug : no
    f00f_bug : no
    coma_bug : no
    fpu : yes
    fpu_exception : yes
    cpuid level : 10
    wp : yes


    Code:
    This is an extract of sysstat (sar -A) on a typical day (today, 8am-3pm):
                       CPU     %user     %nice   %system   %iowait    %steal     %idle
    08:00:01 AM         0      0.09      0.00      0.05      0.11      0.00     99.75
    08:10:01 AM       all      0.09      0.00      0.05      0.86      0.00     99.01
    08:10:01 AM         0      0.09      0.00      0.05      0.86      0.00     99.01
    08:20:01 AM       all      0.11      0.00      0.03      0.20      0.00     99.66
    08:20:01 AM         0      0.11      0.00      0.03      0.20      0.00     99.66
    08:30:02 AM       all      0.09      0.00      0.06      0.21      0.00     99.65
    08:30:02 AM         0      0.09      0.00      0.06      0.21      0.00     99.65
    08:40:01 AM       all      0.08      0.00      0.03      0.06      0.00     99.83
    08:40:01 AM         0      0.08      0.00      0.03      0.06      0.00     99.83
    08:50:01 AM       all      0.29      0.00      0.14      0.12      0.00     99.45
    08:50:01 AM         0      0.29      0.00      0.14      0.12      0.00     99.45
    09:00:01 AM       all      0.09      0.00      0.05      0.24      0.00     99.62
    09:00:01 AM         0      0.09      0.00      0.05      0.24      0.00     99.62
    09:10:01 AM       all      0.22      0.00      0.09      0.96      0.00     98.74
    09:10:01 AM         0      0.22      0.00      0.09      0.96      0.00     98.74
    09:20:01 AM       all      0.09      0.00      0.03      0.16      0.00     99.71
    09:20:01 AM         0      0.09      0.00      0.03      0.16      0.00     99.71
    09:30:01 AM       all      0.09      0.00      0.05      0.22      0.00     99.64
    09:30:01 AM         0      0.09      0.00      0.05      0.22      0.00     99.64
    09:40:01 AM       all      0.08      0.00      0.03      0.07      0.00     99.81
    09:40:01 AM         0      0.08      0.00      0.03      0.07      0.00     99.81
    09:50:01 AM       all      0.09      0.00      0.03      0.10      0.00     99.78
    09:50:01 AM         0      0.09      0.00      0.03      0.10      0.00     99.78
    10:00:01 AM       all      0.08      0.00      0.05      0.27      0.00     99.60
    10:00:01 AM         0      0.08      0.00      0.05      0.27      0.00     99.60
    10:10:02 AM       all      0.17      0.00      0.06      0.92      0.00     98.85
    10:10:02 AM         0      0.17      0.00      0.06      0.92      0.00     98.85
    10:20:01 AM       all      0.31      0.00      0.12      0.24      0.00     99.33
    10:20:01 AM         0      0.31      0.00      0.12      0.24      0.00     99.33
    10:30:01 AM       all      0.31      0.00      0.16      0.37      0.00     99.16
    10:30:01 AM         0      0.31      0.00      0.16      0.37      0.00     99.16
    10:40:01 AM       all      0.09      0.00      0.02      0.14      0.00     99.74
    10:40:01 AM         0      0.09      0.00      0.02      0.14      0.00     99.74
    10:50:01 AM       all      0.08      0.00      0.03      0.10      0.00     99.79
    10:50:01 AM         0      0.08      0.00      0.03      0.10      0.00     99.79
    11:00:01 AM       all      0.30      0.00      0.17      0.19      0.00     99.35
    11:00:01 AM         0      0.30      0.00      0.17      0.19      0.00     99.35
    11:10:01 AM       all      0.10      0.00      0.03      0.84      0.00     99.03
    11:10:01 AM         0      0.10      0.00      0.03      0.84      0.00     99.03
    11:20:01 AM       all      0.10      0.00      0.03      0.17      0.00     99.70
    11:20:01 AM         0      0.10      0.00      0.03      0.17      0.00     99.70
    11:30:01 AM       all      0.10      0.00      0.06      0.17      0.00     99.68
    11:30:01 AM         0      0.10      0.00      0.06      0.17      0.00     99.68
    11:40:01 AM       all      0.43      0.00      0.37      0.13      0.00     99.07
    11:40:01 AM         0      0.43      0.00      0.37      0.13      0.00     99.07
    11:50:01 AM       all      0.09      0.00      0.03      0.10      0.00     99.78
    11:50:01 AM         0      0.09      0.00      0.03      0.10      0.00     99.78
    12:00:01 PM       all      0.09      0.00      0.05      0.17      0.00     99.69
    12:00:01 PM         0      0.09      0.00      0.05      0.17      0.00     99.69
    12:10:01 PM       all      0.10      0.00      0.04      0.87      0.00     98.99
    12:10:01 PM         0      0.10      0.00      0.04      0.87      0.00     98.99
    12:20:01 PM       all      0.10      0.00      0.03      0.16      0.00     99.70
    12:20:01 PM         0      0.10      0.00      0.03      0.16      0.00     99.70
    12:30:02 PM       all      0.24      0.00      0.10      0.30      0.00     99.36
    12:30:02 PM         0      0.24      0.00      0.10      0.30      0.00     99.36
    12:40:01 PM       all      0.15      0.00      0.11      0.06      0.00     99.68
    12:40:01 PM         0      0.15      0.00      0.11      0.06      0.00     99.68
    12:50:01 PM       all      0.10      0.00      0.03      0.08      0.00     99.80
    12:50:01 PM         0      0.10      0.00      0.03      0.08      0.00     99.80
    01:00:01 PM       all      0.08      0.00      0.05      0.16      0.00     99.70
    01:00:01 PM         0      0.08      0.00      0.05      0.16      0.00     99.70
    01:10:02 PM       all      0.10      0.00      0.04      0.83      0.00     99.03
    01:10:02 PM         0      0.10      0.00      0.04      0.83      0.00     99.03
    01:20:01 PM       all      0.11      0.00      0.03      0.17      0.00     99.69
    01:20:01 PM         0      0.11      0.00      0.03      0.17      0.00     99.69
    01:30:01 PM       all      0.09      0.00      0.05      0.25      0.00     99.60
    01:30:01 PM         0      0.09      0.00      0.05      0.25      0.00     99.60
    01:40:01 PM       all      0.29      0.00      0.26      0.01      0.00     99.44
    01:40:01 PM         0      0.29      0.00      0.26      0.01      0.00     99.44
    01:50:01 PM       all      0.36      0.00      0.18      0.21      0.00     99.25
    01:50:01 PM         0      0.36      0.00      0.18      0.21      0.00     99.25
    02:00:01 PM       all      0.18      0.00      0.11      0.13      0.00     99.58
    02:00:01 PM         0      0.18      0.00      0.11      0.13      0.00     99.58
    02:10:01 PM       all      0.10      0.00      0.04      0.76      0.00     99.10
    02:10:01 PM         0      0.10      0.00      0.04      0.76      0.00     99.10
    02:20:01 PM       all      0.11      0.00      0.02      0.20      0.00     99.67
    02:20:01 PM         0      0.11      0.00      0.02      0.20      0.00     99.67
    02:30:01 PM       all      0.09      0.00      0.06      0.21      0.00     99.65
    02:30:01 PM         0      0.09      0.00      0.06      0.21      0.00     99.65
    02:40:01 PM       all      0.22      0.00      0.11      0.20      0.00     99.48
    02:40:01 PM         0      0.22      0.00      0.11      0.20      0.00     99.48
    02:50:01 PM       all      0.09      0.00      0.02      0.01      0.00     99.88
    02:50:01 PM         0      0.09      0.00      0.02      0.01      0.00     99.88
    03:00:01 PM       all      0.27      0.00      0.13      0.19      0.00     99.42
    03:00:01 PM         0      0.27      0.00      0.13      0.19      0.00     99.42
    Average:          all      0.15      0.00      0.06      0.32      0.00     99.47
    Average:            0      0.15      0.00      0.06      0.32      0.00     99.47

    Another thought:
    can the internal phones be the source of the problem? (Snom 190)
     
  8. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    could you post any "top" application information and cpu usage info when yu have the echo issues ...
    im sure its your tiny cpu when entering low voltage modes as i had this type of problemns eith atoms using nettops as testbed for low budget systems
     
  9. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    Hello

    I'm sorry, I don't know how to go about that.
    The echo is only temporary within calls, and happens maybe once per week. What can I do have TOP or CPU usage logs at sufficiently fine intervals available to me?
     
  10. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    Hello there

    I had other problems than echo in the last 2 days, some calls got cut or were choppy.

    in the performance graphic of elastix (the one on the very first page) cpu usage shows 100% for the last few hours. (0 before that, see attached screenshot) http://forum.elastix.org/old_files/2011_02_23_1521.zip

    now, I've been looking at TOP and other system monitoring tools, and everything seems fine there, so this cpu availability information is a bit contradictory.
    here's the output of top, while Elastix tells me I'm maxing my CPU.... ?

    top - 15:30:30 up 7 days, 37 min, 1 user, load average: 0.14, 0.18, 0.17
    Tasks: 117 total, 2 running, 115 sleeping, 0 stopped, 0 zombie
    Cpu(s): 10.6%us, 1.0%sy, 0.0%ni, 88.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 1026656k total, 520372k used, 506284k free, 7456k buffers
    Swap: 2064376k total, 0k used, 2064376k free, 267460k cached
    Code:
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    22534 asterisk  18   0 39792  19m 5736 S 10.6  2.0   0:08.70 httpd
     2441 asterisk  15   0 72840  43m 9.8m S  0.3  4.4  41:16.79 asterisk
    10379 root      15   0  2416 1028  796 R  0.3  0.1   0:00.06 top
        1 root      15   0  2156  648  556 S  0.0  0.1   0:00.61 init
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
        5 root      10  -5     0    0    0 S  0.0  0.0   0:00.03 events/0
        6 root      10  -5     0    0    0 S  0.0  0.0   0:00.07 khelper
        7 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
       10 root      14  -5     0    0    0 S  0.0  0.0   0:00.24 kblockd/0
       11 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0
       14 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khubd
       16 root      10  -5     0    0    0 S  0.0  0.0   0:00.10 kseriod
       58 root      15   0     0    0    0 S  0.0  0.0   0:00.01 kapmd
      104 root      15   0     0    0    0 S  0.0  0.0   0:00.00 khungtaskd
      105 root      25   0     0    0    0 S  0.0  0.0   0:00.00 pdflush
      106 root      15   0     0    0    0 S  0.0  0.0   0:04.54 pdflush
      107 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kswapd0
      108 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
    and here's a ps output:
    Code:
    # ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
    %CPU   PID USER     COMMAND
     0.4  2441 asterisk /usr/sbin/asterisk -f -U asterisk -G asterisk -vvvg -c
     0.1  8482 root     sshd: root@pts/1
     0.0  8484 root     -bash
     0.0  8374 postfix  pickup -l -t fifo -u
     0.0     7 root     [kthread]
     0.0   754 root     [hd-audio0]
     0.0     6 root     [khelper]
     0.0     5 root     [events/0]
     0.0    58 root     [kapmd]

    any ideas?

    should I try the switch to another machine?
    if so, what's the best way to migrate my whole config?
     
  11. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    as i say try to seek for echo issues and correlation to cpu peaks.....im pretty sure yu will find your answer there as in your latest choppy sound and cpu peak
     
  12. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    I would love to do that, but I don't know how...
    can you give me a hint on how to log what processes were using how much cpu in the past?

    or do you know other options on finding out what's using my ressources right now? how come top shows my system as quasi-idle but elastix thinks it's maxed?
     
  13. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    hm... now I rebooted the system, and the period that was marked 100% in the graph previously is now only marked with about 5 %. I guess that was a display error.
     
  14. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    nope the stats get cleaned every x time...use top and uptime programs or webmin yo seek for historical stats...its a cpu erlated issue...
     
  15. alambro1

    Joined:
    Aug 6, 2009
    Messages:
    5
    Likes Received:
    0
    I am having the exact same issues. Users complain about random echo during calls. I have recorded calls and you cannot hear the echo in the recordings. I had these echo issues with elastix 64-bit installed on a dual-quad core HP server with Sangoma A101D T1, and continue to have them on a commercially branded system (won't mention vendor). Both systems have enough CPU and Ram to handle the 30 phones I have on the network so CPU is not an issue. The problem exists with my Aastra 57i and Polycom phones, as well as softphones installed on PC's. The echo is the worst with the Polycoms, moderate with the Aastras and almost un-noticable with softphones. Again, as previously stated by "oldrags", the echo is not constant, it comes and goes during calls and varies in intensity. However, unlike "oldrags", my echo issue is most prominent on inbound calls over our T1, but can be experienced on outbound and internal calls as well. I have turned down txgain on the T1 and it seems to have helped some.

    Could this be related to some type of network phenomenon? In my network I have 2 cisco 6509e cores with around 50 cisco switches of varying models at the edge, all of them connected to the cores via fiber, one of them (4506e) is full POE. The 4506e powers 24 of the phones on my network, the rest of the phones are powered locally or are softphones. The biggest complaints come from the users connected to the 4506e POE switch.

    I am going to mess around with gain some more this weekend as well as taking some of the phones powered by POE and plug them into the wall with the A/C adapter. I will post my results after a short test period.
     
  16. fmvillares

    Joined:
    Sep 8, 2007
    Messages:
    1,785
    Likes Received:
    0
    Re: Re:Echo issues even on VoIP-only calls

    maybe or maybe asterisk related in your systems, you could need a total tune up in your system but i acnt reproduce that kind of echo issues and i have 5000 extensions systems using 12 core dual opteron servers with minimum cpu usage
    remember echo is sound repeated back to you for many reasons, cpu latency network latency, network, bad asterisk implementation firwmare issues etc etc
     
  17. oldrags

    Joined:
    Dec 3, 2010
    Messages:
    13
    Likes Received:
    0
    Hello

    I still owe you some feedback on this.

    I now moved the whole Elastix installation from the Atom-based laptop to a good old Sempron 2800+ processor based one.
    I used the Elastix backup and restore facility to move my config from one architecture to another. That didn't work quite out-of-the-box, I had to google for help for a number of small details. Anyhow, it took me 1/2 day to get the new system running, which is certainly much shorter than setting the whole works up from scratch.

    While the customer still thinks it's not as good as the ISDN installation he had before, he says the situation has improved A LOT since the move Atom -> Sempron.

    I don't see any difference when looking at CPU performance logs, or at what "top" returns. Possibly fmvillares was right
    "im sure its your tiny cpu when entering low voltage modes as i had this type of problemns eith atoms using nettops as testbed for low budget systems"


    Thanks for all your help, hasta la vista!

    oldRags
     

Share This Page