Help! Internet problems



  • Our phone system is currently down. Our provider, BT, don't seem know what the problem is. So far they've just told us to reboot our firewall and reboot our phone system. Suspiciously, we are also have problems connecting to a few websites, although the vast majority are fine. I've done a tracert on one of them and the results are:

    bt.JPG

    The first IP is our firewall, the second one is our public IP address and the ones after that are BT servers. Does the fact that it gets as far as BT's servers suggest that the problem is at their end rather than ours, or is there anything at our end that could cause these issues? I can ping the IP address via external tools like http://tools.pingdom.com/ping/?target=54.76.183.250&o=1 so I know that the websites in question are ok.

    I know next to nothing about networks, so any advice would be much appreciated. If I can establish that there is a problem with BT's servers, then I can assume that the problem we are having with our phone system is likely to be connected.

    So, is tracert telling me anything useful?



  • OK, BT has now established that there is a problem at their end with "SIP trunks not registering", so I guess I'll just sit and wait.



  • @Carnival-Boy said:

    So, is tracert telling me anything useful?

    Tells a lot.

    It's getting to the edge of BT's network, the 109.159.253.83 IP, then bombing out on the peer. AWS looks to be using NTT for their peer, from what I can tell. I route over AT&T's network in the EU though, so it might be me.

    Either way, it's well outside your control and not anything you can fix from your location.



  • Thanks. That's really reassuring.

    BT engineer is saying he can't communicate with our phone server so is blaming our internet connection, although when I explained the tracert issue he did concede that it might be an issue outside of our organisation. But their only suggestion so far is to reboot everything again. We're getting nowhere. 5 hours of down time during our busiest sales day - this is not looking good.

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!


  • Banned

    @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Or get a phone system separate to BT 🙂



  • @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Most ISDNs are sip with a PRI handoff instead of keeping it SIP at the handoff. Same Technology on the back end anymore.



  • @Breffni-Potter said:

    @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Or get a phone system separate to BT 🙂

    He does. It's on AWS. 🙂

    This highlights another one of those things that SIP brings up. Peering issues, like with Cogent in the US, can cause major issues with third party sites. And fixing them is NOT easy, as most ISPs refuse to believe that they have any issues or will flat out tell you they can't do shit because it's "outside the network".

    Sticking with on-net SIP providers can alleviate some of this, as can multiple connections. I had a few years back an ISP not worth a shit who couldn't keep their own internal routing working.



  • @thecreativeone91 said:

    @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Most ISDNs are sip with a PRI handoff instead of keeping it SIP at the handoff. Same Technology on the back end anymore.

    ???

    I don't know of any telco company doing IP like that. ISDN from BT is the same PRI signaling that they have done for decades. AT&T does standard framing for PRI so all you need to do is plug in your TA and go to town.

    AT&T's U-Verse product is pure IP, but does fall into a different category of service versus a PRI. On the wireless side of things, there was some stuff being done pure IP on WiMax with Clearwire, but that's mostly because they were a data focused company.



  • @PSX_Defector said:

    @thecreativeone91 said:

    @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Most ISDNs are sip with a PRI handoff instead of keeping it SIP at the handoff. Same Technology on the back end anymore.

    ???

    I don't know of any telco company doing IP like that. ISDN from BT is the same PRI signaling that they have done for decades. AT&T does standard framing for PRI so all you need to do is plug in your TA and go to town.

    AT&T's U-Verse product is pure IP, but does fall into a different category of service versus a PRI. On the wireless side of things, there was some stuff being done pure IP on WiMax with Clearwire, but that's mostly because they were a data focused company.

    We have about 10 locations with analog phone systems still. All are on either CentruyLink or AT&T PRIs. they all just go through a converter over the fiber now days to do a PRI handoff instead of SIP.



  • It's all gobbledygook to me 🙂

    We have a Samsung Ubigate box which is a SIP gateway, so that converts the IP interface to an ISDN interface. So it sits between our PBX box and our LAN. BT says they can't get any reply from the Ubigate box so they're sending an engineer to poke around in our server room. But that doesn't explain my traceroute issue.



  • @thecreativeone91 said:

    @PSX_Defector said:

    @thecreativeone91 said:

    @Carnival-Boy said:

    Really tempted to go back to ISDN - I never had a problem in 20 years of service!

    Most ISDNs are sip with a PRI handoff instead of keeping it SIP at the handoff. Same Technology on the back end anymore.

    ???

    I don't know of any telco company doing IP like that. ISDN from BT is the same PRI signaling that they have done for decades. AT&T does standard framing for PRI so all you need to do is plug in your TA and go to town.

    AT&T's U-Verse product is pure IP, but does fall into a different category of service versus a PRI. On the wireless side of things, there was some stuff being done pure IP on WiMax with Clearwire, but that's mostly because they were a data focused company.

    We have about 10 locations with analog phone systems still. All are on either CentruyLink or AT&T PRIs. they all just go through a converter over the fiber now days to do a PRI handoff instead of SIP.

    A little different.

    https://www.digium.com/products/voip-gateways/g100

    It's SIP all the way to you, it's just using a converter to integrate into your old school PBX as a requirement. Nothing is stopping you from going pure SIP on it, just gotta have the right equipment.

    A PRI straight from the central office is never gonna use framing like this.



  • @PSX_Defector said:

    A little different.

    https://www.digium.com/products/voip-gateways/g100

    It's SIP all the way to you, it's just using a converter to integrate into your old school PBX as a requirement. Nothing is stopping you from going pure SIP on it, just gotta have the right equipment.

    A PRI straight from the central office is never gonna use framing like this.

    Yes, but that's the only PRI service they will sell us these days. It's not over our WAN connection or anything (or even the same providers).

    There's something like 6-8 fiber strains going into the converter cabinets.

    P_20150713_101937.jpg



  • Fixed and phone system back on line. It's only taken 27 hours 😡

    As I suspected, internet issues were the cause. The SIP trunks tried to register at hipcom.co.uk and we couldn't access this site. BT initially blamed our firewall and denied any responsibility ("none of our other customers have reported any problems"). It took us quite a while to convince them that it was a problem at their end. We ended up plugging a laptop directly into their router to prove that it wasn't our firewall or anything on our LAN. We even used a different router plugged directly into the leased line to rule out router problems.

    Eventually they escalated it up their organisation and discovered that some engineer somewhere had created a "duplicate route in the exchange" whatever that means. I presume someone was doing some work for another company and mistyped the IP address and used our IP address which caused certain routes to be blocked, including the route needed to register the SIP trunks.

    It's quite a relief but it's been a horrible couple of days and who knows how much damage to our reputation and order book.



  • I'm now looking into any resilience it would be prudent to implement to prevent any future issues. One idea is to order some SIP trunks from another provider and have them connected over a spare ADSL line. So if we have any problems with our main internet connection or our main SIP trunks we can just route the phone system over the ADSL line on alternate SIP trunks and get our phone number diverted to that new route whilst the old route is repaired.

    Or I may not bother and accept that our issues were pretty unusual and very unlikely to happen ever again.



  • @Carnival-Boy said:

    I'm now looking into any resilience it would be prudent to implement to prevent any future issues. One idea is to order some SIP trunks from another provider and have them connected over a spare ADSL line. So if we have any problems with our main internet connection or our main SIP trunks we can just route the phone system over the ADSL line on alternate SIP trunks and get our phone number diverted to that new route whilst the old route is repaired.

    Or I may not bother and accept that our issues were pretty unusual and very unlikely to happen ever again.

    Getting two companies to work together to allow the numbers to roll over would be hard. I dobut they'd do that so likely pretty much just be outbound calls that it would help.



  • We got our phone number rerouted to a new number whilst this was going on, so I assume we'd just do that again. ie get calls to 12345 to route to number 678910. It was done within a few minutes though we were told it could take up to 24 hours.



  • @Carnival-Boy said:

    We got our phone number rerouted to a new number whilst this was going on, so I assume we'd just do that again. ie get calls to 12345 to route to number 678910. It was done within a few minutes though we were told it could take up to 24 hours.

    Ah, so with your setup it doesn't matter what phone number the phone system see. Ours would take too long to configure if we didn't have them roll over. We have a lot of department DID's and such.



  • We do have DIDs but have been trying not to use them, preferring a single number with an auto-attendant (which everyone hates).



  • @thecreativeone91 said:

    @Carnival-Boy said:

    I'm now looking into any resilience it would be prudent to implement to prevent any future issues. One idea is to order some SIP trunks from another provider and have them connected over a spare ADSL line. So if we have any problems with our main internet connection or our main SIP trunks we can just route the phone system over the ADSL line on alternate SIP trunks and get our phone number diverted to that new route whilst the old route is repaired.

    Or I may not bother and accept that our issues were pretty unusual and very unlikely to happen ever again.

    Getting two companies to work together to allow the numbers to roll over would be hard. I dobut they'd do that so likely pretty much just be outbound calls that it would help.

    My local SIP provider will allow me to forward my calls to another number. Assuming he could do the same, just have the main SIP provider forward all calls to his numbers to his backup number during the outage and bob's your uncle.



  • @Dashrender said:

    My local SIP provider will allow me to forward my calls to another number. Assuming he could do the same, just have the main SIP provider forward all calls to his numbers to his backup number during the outage and bob's your uncle.

    That doesn't work when you have a lot of inbound rules as well as multiple Toll Free lines as well.



  • @Dashrender said:

    My local SIP provider will allow me to forward my calls to another number. Assuming he could do the same, just have the main SIP provider forward all calls to his numbers to his backup number during the outage and bob's your uncle.

    Yes. That is exactly what we do.



  • @thecreativeone91 said:

    That doesn't work when you have a lot of inbound rules as well as multiple Toll Free lines as well.

    Sure it does. You just need an emergency IVR message stating there is a problem with the telephone provider and calls are being rerouted to a backup system. Then dump the calls to a few desginated emergency operators.



  • @JaredBusch said:

    @thecreativeone91 said:

    That doesn't work when you have a lot of inbound rules as well as multiple Toll Free lines as well.

    Sure it does. You just need an emergency IVR message stating there is a problem with the telephone provider and calls are being rerouted to a backup system. Then dump the calls to a few desginated emergency operators.

    We have lines with a roll over agreement between to providers so we don't have to do that.



  • @thecreativeone91 said:

    @JaredBusch said:

    @thecreativeone91 said:

    That doesn't work when you have a lot of inbound rules as well as multiple Toll Free lines as well.

    Sure it does. You just need an emergency IVR message stating there is a problem with the telephone provider and calls are being rerouted to a backup system. Then dump the calls to a few desginated emergency operators.

    We have lines with a roll over agreement between to providers so we don't have to do that.

    You're situation is a bit larger than most, so those agreements are definitely a good idea.

    Our provider does allow us to designate a failover forward to phone number, so in the case where the SIP traffic can't register, it would auto failover.


Log in to reply