1. Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

TPU server move completed!

Discussion in 'Comments & Feedback' started by W1zzard, Sep 17, 2012.

  1. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    Fixed

    investigating the cause now ... seems the db and image caching server (same machine) had some kind of failure

    Code:
    Sep 18 05:19:27 db kernel: swapper: page allocation failure. order:1, mode:0x20
    Sep 18 05:19:27 db kernel: Pid: 0, comm: swapper Not tainted 2.6.32-279.5.2.el6.x86_64 #1
    Sep 18 05:19:27 db kernel: Call Trace:
    Sep 18 05:19:27 db kernel: <IRQ>  [<ffffffff8112781f>] ? __alloc_pages_nodemask+0x77f/0x940
    Sep 18 05:19:27 db kernel: [<ffffffff81161fe2>] ? kmem_getpages+0x62/0x170
    Sep 18 05:19:27 db kernel: [<ffffffff81162bfa>] ? fallback_alloc+0x1ba/0x270
    Sep 18 05:19:27 db kernel: [<ffffffff81162979>] ? ____cache_alloc_node+0x99/0x160
    Sep 18 05:19:27 db kernel: [<ffffffff8116375b>] ? kmem_cache_alloc+0x11b/0x190
    Sep 18 05:19:27 db kernel: [<ffffffff8142dee8>] ? sk_prot_alloc+0x48/0x1c0
    Sep 18 05:19:27 db kernel: [<ffffffff8142e1b2>] ? sk_clone+0x22/0x2e0
    Sep 18 05:19:27 db kernel: [<ffffffff8147be06>] ? inet_csk_clone+0x16/0xd0
    Sep 18 05:19:27 db kernel: [<ffffffff81494d63>] ? tcp_create_openreq_child+0x23/0x450
    Sep 18 05:19:27 db kernel: [<ffffffff8149261d>] ? tcp_v4_syn_recv_sock+0x4d/0x310
    Sep 18 05:19:27 db kernel: [<ffffffff81494b06>] ? tcp_check_req+0x226/0x460
    Sep 18 05:19:27 db kernel: [<ffffffff8149203b>] ? tcp_v4_do_rcv+0x35b/0x430
    Sep 18 05:19:27 db kernel: [<ffffffffa030e557>] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
    Sep 18 05:19:27 db kernel: [<ffffffff8149383e>] ? tcp_v4_rcv+0x4fe/0x8d0
    Sep 18 05:19:27 db kernel: [<ffffffff81471480>] ? ip_local_deliver_finish+0x0/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff8147155d>] ? ip_local_deliver_finish+0xdd/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff814717e8>] ? ip_local_deliver+0x98/0xa0
    Sep 18 05:19:27 db kernel: [<ffffffff81470cad>] ? ip_rcv_finish+0x12d/0x440
    Sep 18 05:19:27 db kernel: [<ffffffff81471235>] ? ip_rcv+0x275/0x350
    Sep 18 05:19:27 db kernel: [<ffffffff8143aa3b>] ? __netif_receive_skb+0x49b/0x6f0
    Sep 18 05:19:27 db kernel: [<ffffffff8143ccb8>] ? netif_receive_skb+0x58/0x60
    Sep 18 05:19:27 db kernel: [<ffffffff8143cdc0>] ? napi_skb_finish+0x50/0x70
    Sep 18 05:19:27 db kernel: [<ffffffff8143f2f9>] ? napi_gro_receive+0x39/0x50
    Sep 18 05:19:27 db kernel: [<ffffffffa015b20b>] ? e1000_receive_skb+0x5b/0x90 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffffa015efa0>] ? e1000_clean_rx_irq+0x380/0x580 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffffa015d7c5>] ? e1000_clean+0xb5/0x2c0 [e1000e]
    Sep 18 05:19:27 db kernel: [<ffffffff81096182>] ? enqueue_hrtimer+0x82/0xd0
    Sep 18 05:19:27 db kernel: [<ffffffff8143f413>] ? net_rx_action+0x103/0x2f0
    Sep 18 05:19:27 db kernel: [<ffffffff81073ec1>] ? __do_softirq+0xc1/0x1e0
    Sep 18 05:19:27 db kernel: [<ffffffff810dba80>] ? handle_IRQ_event+0x60/0x170
    Sep 18 05:19:27 db kernel: [<ffffffff8100c24c>] ? call_softirq+0x1c/0x30
    Sep 18 05:19:27 db kernel: [<ffffffff8100de85>] ? do_softirq+0x65/0xa0
    Sep 18 05:19:27 db kernel: [<ffffffff81073ca5>] ? irq_exit+0x85/0x90
    Sep 18 05:19:27 db kernel: [<ffffffff81505d75>] ? do_IRQ+0x75/0xf0
    Sep 18 05:19:27 db kernel: [<ffffffff8100ba53>] ? ret_from_intr+0x0/0x11
    Sep 18 05:19:27 db kernel: <EOI>  [<ffffffff812f7cf4>] ? acpi_idle_enter_bm+0x29c/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff812f7ced>] ? acpi_idle_enter_bm+0x295/0x2d0
    Sep 18 05:19:27 db kernel: [<ffffffff814078a7>] ? cpuidle_idle_call+0xa7/0x140
    Sep 18 05:19:27 db kernel: [<ffffffff81009e06>] ? cpu_idle+0xb6/0x110
    Sep 18 05:19:27 db kernel: [<ffffffff814f6f5f>] ? start_secondary+0x22a/0x26d
    
    the log is full of this, like once every few hours, has been going on for a few days

    seems to be that: https://bugzilla.redhat.com/show_bug.cgi?id=770545
    Last edited: Sep 25, 2012
    silkstone and Solaris17 say thanks.
  2. HossHuge

    HossHuge

    Joined:
    Jun 26, 2008
    Messages:
    2,020 (0.91/day)
    Thanks Received:
    489
    Location:
    EDM, AB, CAN
    OMG its too fast!! It burned a hole in my monitor....:eek:
  3. Solaris17

    Solaris17 Creator Solaris Utility DVD

    Joined:
    Aug 16, 2005
    Messages:
    17,094 (5.22/day)
    Thanks Received:
    3,515
    Location:
    Florida
    ? Maybe if the machine calls for it. is it being flushed?
  4. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    yes it's 0 on the server, gonna switch that to 1 after a bit to see how it changes things
  5. eidairaman1

    eidairaman1

    Joined:
    Jul 2, 2007
    Messages:
    12,086 (4.67/day)
    Thanks Received:
    1,377
    wowsers this place was out of wack for a bit.
  6. mediasorcerer New Member

    Joined:
    Sep 15, 2011
    Messages:
    979 (0.93/day)
    Thanks Received:
    225
    Location:
    coast ,melbourne
    Hi Wizzard,

    At first i found the pages to be loading very slowly, however, it seems to load fine now so,
    good stuff! Yep, loading quick as.
    Last edited: Sep 25, 2012
  7. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    everything should be working normally now, please keep reporting any problems you find
    mediasorcerer says thanks.
  8. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,543 (0.75/day)
    Thanks Received:
    299
    Who is hosting?
  9. eidairaman1

    eidairaman1

    Joined:
    Jul 2, 2007
    Messages:
    12,086 (4.67/day)
    Thanks Received:
    1,377
    for those who Use Facebook, if this site goes offline or very slow loading, find Techpowerup on Facebook
  10. Mussels

    Mussels Moderprator Staff Member

    Joined:
    Oct 6, 2004
    Messages:
    42,021 (11.71/day)
    Thanks Received:
    9,406
    i had my first internet connection in over a week (moved house) right as this happened.


    you have any idea how much time i wasted troubleshooting, only to realise it was TPU that was down? -.-
  11. eidairaman1

    eidairaman1

    Joined:
    Jul 2, 2007
    Messages:
    12,086 (4.67/day)
    Thanks Received:
    1,377
    4 hrs worth of time
    mediasorcerer says thanks.
  12. mediasorcerer New Member

    Joined:
    Sep 15, 2011
    Messages:
    979 (0.93/day)
    Thanks Received:
    225
    Location:
    coast ,melbourne
    Oh how frustrating!

    [ot]~moving house= big stress.
  13. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    Before: SoftLayer
    Now: Futurehosting at an Equinix datacenter
  14. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,543 (0.75/day)
    Thanks Received:
    299
    Funny the amount of routers it goes through when you only live 100 miles from Chicago

    Tracing route to techpowerup.com [173.249.152.206]
    over a maximum of 30 hops:

    1 <1 ms 1 ms <1 ms 192.168.1.1
    2 13 ms 9 ms 12 ms 10.132.9.1
    3 17 ms 15 ms 15 ms 172.30.22.17
    4 15 ms 12 ms 14 ms 172.30.23.17
    5 17 ms 16 ms 18 ms 12.249.212.21
    6 24 ms 19 ms 15 ms cr1.cgcil.ip.att.net [12.122.99.22]
    7 45 ms 79 ms 43 ms cgcil02jt.ip.att.net [12.122.81.21]
    8 23 ms 15 ms 17 ms ae3.chi10.ip4.tinet.net [173.241.128.29]
    9 18 ms 22 ms 19 ms xe-1-1-0.chi12.ip4.tinet.net [89.149.185.97]
    10 27 ms 31 ms 18 ms steadfast-gw.ip4.tinet.net [216.221.156.178]
    11 17 ms 16 ms 17 ms ip78.208-100-32.static.steadfastdns.net [208.100.32.78]
    12 22 ms 15 ms 18 ms ip4.67-202-87.static.steadfastdns.net [67.202.87.4]
    13 23 ms 19 ms 17 ms 173.249.152.206

    Trace complete.
  15. remixedcat

    remixedcat

    Joined:
    May 13, 2010
    Messages:
    2,641 (1.71/day)
    Thanks Received:
    549
    hmmm any lookups I do say steadfast networks

    Reverse DNS (PTR record) not available
    ASN number 32748
    ASN name (ISP) STEADFAST - Steadfast Networks
    IP-range/subnet 173.249.152.0/24
    173.249.152.0 - 173.249.152.255


    [​IMG]
  16. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
  17. D007

    D007

    Joined:
    Mar 7, 2007
    Messages:
    3,060 (1.13/day)
    Thanks Received:
    374
    Location:
    Pompano beach, Florida


    OOh, I like shiny.. :)

    :toast:
  18. TheMailMan78

    TheMailMan78 Big Member

    Joined:
    Jun 3, 2007
    Messages:
    20,904 (7.98/day)
    Thanks Received:
    7,490
    Not getting PM emails anymore.
  19. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    anyone else got this problem?

    edit: i added a possible fix, not sure if there even was a problem in the first place, but dns on the mail server was kinda slow

    yup, the isp-provided dns server of email server went away .. using google dns now.
    D007 says thanks.
  20. D007

    D007

    Joined:
    Mar 7, 2007
    Messages:
    3,060 (1.13/day)
    Thanks Received:
    374
    Location:
    Pompano beach, Florida
    Idk, feel free to pm me and I'll look right at it now..

    PM sent to TMM, just to test..
  21. TheMailMan78

    TheMailMan78 Big Member

    Joined:
    Jun 3, 2007
    Messages:
    20,904 (7.98/day)
    Thanks Received:
    7,490
    That worked. Just got a few E-mails for PM's. Thanks boss!
    D007 says thanks.
  22. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    Thank you for reporting it
  23. Mussels

    Mussels Moderprator Staff Member

    Joined:
    Oct 6, 2004
    Messages:
    42,021 (11.71/day)
    Thanks Received:
    9,406
    was the outage today related to this move?
  24. eidairaman1

    eidairaman1

    Joined:
    Jul 2, 2007
    Messages:
    12,086 (4.67/day)
    Thanks Received:
    1,377
    this is like the second outage that has occurred within a weeks time, The older servers seemed to never have outages...
  25. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,647 (3.92/day)
    Thanks Received:
    11,389
    yup .. updated the kernel on both systems, seems there was a bug in the older version that was affecting us.
    m1dg3t, Ikaruga and eidairaman1 say thanks.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guest)

Share This Page