1. Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

TPU server move completed!

Discussion in 'Comments & Feedback' started by W1zzard, Sep 17, 2012.

  1. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    Fixed

    investigating the cause now ... seems the db and image caching server (same machine) had some kind of failure

    Code:
    Sep 18 05:19:27 db kernel: swapper: page allocation failure. order:1, mode:0x20
    Sep 18 05:19:27 db kernel: Pid: 0, comm: swapper Not tainted 2.6.32-279.5.2.el6.x86_64 #1
    Sep 18 05:19:27 db kernel: Call Trace:
    Sep 18 05:19:27 db kernel: <IRQ>  [<ffffffff8112781f>] ? __alloc_pages_nodemask+0x77f/0x940
    Sep 18 05:19:27 db kernel: [<ffffffff81161fe2>] ? kmem_getpages+0x62/0x170
    Sep 18 05:19:27 db kernel: [<ffffffff81162bfa>] ? fallback_alloc+0x1ba/0x270
    Sep 18 05:19:27 db kernel: [<ffffffff81162979>] ? ____cache_alloc_node+0x99/0x160
    Sep 18 05:19:27 db kernel: [<ffffffff8116375b>] ? kmem_cache_alloc+0x11b/0x190
    Sep 18 05:19:27 db kernel: [<ffffffff8142dee8>] ? sk_prot_alloc+0x48/0x1c0
    Sep 18 05:19:27 db kernel: [<ffffffff8142e1b2>] ? sk_clone+0x22/0x2e0
    Sep 18 05:19:27 db kernel: [<ffffffff8147be06>] ? inet_csk_clone+0x16/0xd0
    Sep 18 05:19:27 db kernel: [<ffffffff81494d63>] ? tcp_create_openreq_child+0x23/0x450
    Sep 18 05:19:27 db kernel: [<ffffffff8149261d>] ? tcp_v4_syn_recv_sock+0x4d/0x310
    Sep 18 05:19:27 db kernel: [<ffffffff81494b06>] ? tcp_check_req+0x226/0x460
    Sep 18 05:19:27 db kernel: [<ffffffff8149203b>] ? tcp_v4_do_rcv+0x35b/0x430
    Sep 18 05:19:27 db kernel: [<ffffffffa030e557>] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
    Sep 18 05:19:27 db kernel: [<ffffffff8149383e>] ? tcp_v4_rcv+0x4fe/0x8d0
    Sep 18 05:19:27 db kernel: [<ffffffff81471480>] ? ip_local_deliver_finish+0x0/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff8147155d>] ? ip_local_deliver_finish+0xdd/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff814717e8>] ? ip_local_deliver+0x98/0xa0
    Sep 18 05:19:27 db kernel: [<ffffffff81470cad>] ? ip_rcv_finish+0x12d/0x440
    Sep 18 05:19:27 db kernel: [<ffffffff81471235>] ? ip_rcv+0x275/0x350
    Sep 18 05:19:27 db kernel: [<ffffffff8143aa3b>] ? __netif_receive_skb+0x49b/0x6f0
    Sep 18 05:19:27 db kernel: [<ffffffff8143ccb8>] ? netif_receive_skb+0x58/0x60
    Sep 18 05:19:27 db kernel: [<ffffffff8143cdc0>] ? napi_skb_finish+0x50/0x70
    Sep 18 05:19:27 db kernel: [<ffffffff8143f2f9>] ? napi_gro_receive+0x39/0x50
    Sep 18 05:19:27 db kernel: [<ffffffffa015b20b>] ? e1000_receive_skb+0x5b/0x90 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffffa015efa0>] ? e1000_clean_rx_irq+0x380/0x580 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffffa015d7c5>] ? e1000_clean+0xb5/0x2c0 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffff81096182>] ? enqueue_hrtimer+0x82/0xd0
    Sep 18 05:19:27 db kernel: [<ffffffff8143f413>] ? net_rx_action+0x103/0x2f0
    Sep 18 05:19:27 db kernel: [<ffffffff81073ec1>] ? __do_softirq+0xc1/0x1e0
    Sep 18 05:19:27 db kernel: [<ffffffff810dba80>] ? handle_IRQ_event+0x60/0x170
    Sep 18 05:19:27 db kernel: [<ffffffff8100c24c>] ? call_softirq+0x1c/0x30
    Sep 18 05:19:27 db kernel: [<ffffffff8100de85>] ? do_softirq+0x65/0xa0
    Sep 18 05:19:27 db kernel: [<ffffffff81073ca5>] ? irq_exit+0x85/0x90
    Sep 18 05:19:27 db kernel: [<ffffffff81505d75>] ? do_IRQ+0x75/0xf0
    Sep 18 05:19:27 db kernel: [<ffffffff8100ba53>] ? ret_from_intr+0x0/0x11
    Sep 18 05:19:27 db kernel: <EOI>  [<ffffffff812f7cf4>] ? acpi_idle_enter_bm+0x29c/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff812f7ced>] ? acpi_idle_enter_bm+0x295/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff814078a7>] ? cpuidle_idle_call+0xa7/0x140
    Sep 18 05:19:27 db kernel: [<ffffffff81009e06>] ? cpu_idle+0xb6/0x110
    Sep 18 05:19:27 db kernel: [<ffffffff814f6f5f>] ? start_secondary+0x22a/0x26d
    
    the log is full of this, like once every few hours, has been going on for a few days

    seems to be that: https://bugzilla.redhat.com/show_bug.cgi?id=770545
     
    Last edited: Sep 25, 2012
    silkstone and Solaris17 say thanks.
    10 Year Member at TPU
  2. HossHuge

    HossHuge

    Joined:
    Jun 26, 2008
    Messages:
    2,298 (0.69/day)
    Thanks Received:
    887
    Location:
    EDM, AB, CAN
    OMG its too fast!! It burned a hole in my monitor....:eek:
     
  3. Solaris17

    Solaris17 Creator Solaris Utility DVD

    Joined:
    Aug 16, 2005
    Messages:
    18,992 (4.36/day)
    Thanks Received:
    5,799
    Location:
    Florida
    ? Maybe if the machine calls for it. is it being flushed?
     
    10 Year Member at TPU More than 25k PPD
  4. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    yes it's 0 on the server, gonna switch that to 1 after a bit to see how it changes things
     
    10 Year Member at TPU
  5. eidairaman1

    eidairaman1 The Exiled Airman

    Joined:
    Jul 2, 2007
    Messages:
    17,917 (4.88/day)
    Thanks Received:
    4,122
    wowsers this place was out of wack for a bit.
     
    10 Year Member at TPU
  6. mediasorcerer New Member

    Joined:
    Sep 15, 2011
    Messages:
    978 (0.46/day)
    Thanks Received:
    225
    Location:
    coast ,melbourne
    Hi Wizzard,

    At first i found the pages to be loading very slowly, however, it seems to load fine now so,
    good stuff! Yep, loading quick as.
     
    Last edited: Sep 25, 2012
  7. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    everything should be working normally now, please keep reporting any problems you find
     
    mediasorcerer says thanks.
    10 Year Member at TPU
  8. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,820 (0.58/day)
    Thanks Received:
    511
    Who is hosting?
     
  9. eidairaman1

    eidairaman1 The Exiled Airman

    Joined:
    Jul 2, 2007
    Messages:
    17,917 (4.88/day)
    Thanks Received:
    4,122
    for those who Use Facebook, if this site goes offline or very slow loading, find Techpowerup on Facebook
     
    10 Year Member at TPU
  10. Mussels

    Mussels Moderprator Staff Member

    Joined:
    Oct 6, 2004
    Messages:
    45,917 (9.83/day)
    Thanks Received:
    13,465
    Location:
    Australalalalalaia.
    i had my first internet connection in over a week (moved house) right as this happened.


    you have any idea how much time i wasted troubleshooting, only to realise it was TPU that was down? -.-
     
    10 Year Member at TPU
  11. eidairaman1

    eidairaman1 The Exiled Airman

    Joined:
    Jul 2, 2007
    Messages:
    17,917 (4.88/day)
    Thanks Received:
    4,122
    4 hrs worth of time
     
    mediasorcerer says thanks.
    10 Year Member at TPU
  12. mediasorcerer New Member

    Joined:
    Sep 15, 2011
    Messages:
    978 (0.46/day)
    Thanks Received:
    225
    Location:
    coast ,melbourne
    Oh how frustrating!

    [ot]~moving house= big stress.
     
  13. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    Before: SoftLayer
    Now: Futurehosting at an Equinix datacenter
     
    10 Year Member at TPU
  14. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,820 (0.58/day)
    Thanks Received:
    511
    Funny the amount of routers it goes through when you only live 100 miles from Chicago

    Tracing route to techpowerup.com [173.249.152.206]
    over a maximum of 30 hops:

    1 <1 ms 1 ms <1 ms 192.168.1.1
    2 13 ms 9 ms 12 ms 10.132.9.1
    3 17 ms 15 ms 15 ms 172.30.22.17
    4 15 ms 12 ms 14 ms 172.30.23.17
    5 17 ms 16 ms 18 ms 12.249.212.21
    6 24 ms 19 ms 15 ms cr1.cgcil.ip.att.net [12.122.99.22]
    7 45 ms 79 ms 43 ms cgcil02jt.ip.att.net [12.122.81.21]
    8 23 ms 15 ms 17 ms ae3.chi10.ip4.tinet.net [173.241.128.29]
    9 18 ms 22 ms 19 ms xe-1-1-0.chi12.ip4.tinet.net [89.149.185.97]
    10 27 ms 31 ms 18 ms steadfast-gw.ip4.tinet.net [216.221.156.178]
    11 17 ms 16 ms 17 ms ip78.208-100-32.static.steadfastdns.net [208.100.32.78]
    12 22 ms 15 ms 18 ms ip4.67-202-87.static.steadfastdns.net [67.202.87.4]
    13 23 ms 19 ms 17 ms 173.249.152.206

    Trace complete.
     
  15. remixedcat

    remixedcat

    Joined:
    May 13, 2010
    Messages:
    4,301 (1.64/day)
    Thanks Received:
    1,495
    hmmm any lookups I do say steadfast networks

    Reverse DNS (PTR record) not available
    ASN number 32748
    ASN name (ISP) STEADFAST - Steadfast Networks
    IP-range/subnet 173.249.152.0/24
    173.249.152.0 - 173.249.152.255


    [​IMG]
     
  16. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    10 Year Member at TPU
  17. D007

    D007

    Joined:
    Mar 7, 2007
    Messages:
    3,788 (1.00/day)
    Thanks Received:
    695
    Location:
    Pompano beach, Florida


    OOh, I like shiny.. :)

    :toast:
     
    10 Year Member at TPU
  18. TheMailMan78

    TheMailMan78 Big Member

    Joined:
    Jun 3, 2007
    Messages:
    22,253 (6.01/day)
    Thanks Received:
    8,997
    Location:
    'Merica. The Great SOUTH!
    Not getting PM emails anymore.
     
    10 Year Member at TPU
  19. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    anyone else got this problem?

    edit: i added a possible fix, not sure if there even was a problem in the first place, but dns on the mail server was kinda slow

    yup, the isp-provided dns server of email server went away .. using google dns now.
     
    D007 says thanks.
    10 Year Member at TPU
  20. D007

    D007

    Joined:
    Mar 7, 2007
    Messages:
    3,788 (1.00/day)
    Thanks Received:
    695
    Location:
    Pompano beach, Florida
    Idk, feel free to pm me and I'll look right at it now..

    PM sent to TMM, just to test..
     
    10 Year Member at TPU
  21. TheMailMan78

    TheMailMan78 Big Member

    Joined:
    Jun 3, 2007
    Messages:
    22,253 (6.01/day)
    Thanks Received:
    8,997
    Location:
    'Merica. The Great SOUTH!
    That worked. Just got a few E-mails for PM's. Thanks boss!
     
    D007 says thanks.
    10 Year Member at TPU
  22. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    Thank you for reporting it
     
    10 Year Member at TPU
  23. Mussels

    Mussels Moderprator Staff Member

    Joined:
    Oct 6, 2004
    Messages:
    45,917 (9.83/day)
    Thanks Received:
    13,465
    Location:
    Australalalalalaia.
    was the outage today related to this move?
     
    10 Year Member at TPU
  24. eidairaman1

    eidairaman1 The Exiled Airman

    Joined:
    Jul 2, 2007
    Messages:
    17,917 (4.88/day)
    Thanks Received:
    4,122
    this is like the second outage that has occurred within a weeks time, The older servers seemed to never have outages...
     
    10 Year Member at TPU
  25. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,718 (3.47/day)
    Thanks Received:
    17,364
    yup .. updated the kernel on both systems, seems there was a bug in the older version that was affecting us.
     
    m1dg3t, Ikaruga and eidairaman1 say thanks.
    10 Year Member at TPU

Currently Active Users Viewing This Thread: 1 (0 members and 1 guest)