• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Server 2012

Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#1
I wonder if we need a server section....


Anyway. I've got a series of error messages on one server (of two) in a location. This is a Terminal Services server, located in the same facility as its DC. There are error messages popping multiple times a second on one of my servers, and can't seem to tie it down specifically. These particular errors occurred 24 times in the same second, between the three of them, repeated.

Items in Blue italics are changed to spare the not-so-innocent, and also indicate what they represent, ie "SERVERNAME" is the actual server name, etc.

What appears significant to me is the $ after the server name in the first error message, but I don't know what it indicates.... By that I mean that it looks like it could be a share process, since the system generated shares can put a $ in the share name, but I'm not sure....Any M$ server geniuses have any ideas? Thanks for looking and tasking those big brains!!


Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 1/10/2017 3:06:19 PM
Event ID: 4771
Task Category: Kerberos Authentication Service
Level: Information
Keywords: Audit Failure
User: N/A
Computer: FULL.Server.Name
Description:
Kerberos pre-authentication failed.

Account Information:
Security ID: DOMAIN\SERVERNAME$
Account Name: SERVERNAME$

Service Information:
Service Name: krbtgt/DOMAIN.NAME

Network Information:
Client Address: ::1
Client Port: 0

Additional Information:
Ticket Options: 0x40810010
Failure Code: 0x18
Pre-Authentication Type: 2

Certificate Information:
Certificate Issuer Name:
Certificate Serial Number:
Certificate Thumbprint:

Certificate information is only provided if a certificate was used for pre-authentication.

Pre-authentication types, ticket options and failure codes are defined in RFC 4120.

If the ticket was malformed or damaged during transit and could not be decrypted, then many fields in this event might not be present.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4771</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>14339</Task>
<Opcode>0</Opcode>
<Keywords>0x8010000000000000</Keywords>
<TimeCreated SystemTime="2017-01-10T20:06:19.664595600Z" />
<EventRecordID>13461329</EventRecordID>
<Correlation />
<Execution ProcessID="1004" ThreadID="16664" />
<Channel>Security</Channel>
<Computer>FULL.Server.Name</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserName">SERVERNAME$</Data>
<Data Name="TargetSid">S-1-5-21-1979150871-126566477-2868468453-1104</Data>
<Data Name="ServiceName">krbtgt/DOMAIN.NAME</Data>
<Data Name="TicketOptions">0x40810010</Data>
<Data Name="Status">0x18</Data>
<Data Name="PreAuthType">2</Data>
<Data Name="IpAddress">::1</Data>
<Data Name="IpPort">0</Data>
<Data Name="CertIssuerName">
</Data>
<Data Name="CertSerialNumber">
</Data>
<Data Name="CertThumbprint">
</Data>
</EventData>
</Event>

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 1/10/2017 3:06:19 PM
Event ID: 4776
Task Category: Credential Validation
Level: Information
Keywords: Audit Failure
User: N/A
Computer: FULL.Computer.Name
Description:
The computer attempted to validate the credentials for an account.

Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon Account: SERVERNAME
Source Workstation: SERVERNAME
Error Code: 0xC0000064
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4776</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>14336</Task>
<Opcode>0</Opcode>
<Keywords>0x8010000000000000</Keywords>
<TimeCreated SystemTime="2017-01-10T20:06:19.664595600Z" />
<EventRecordID>13461330</EventRecordID>
<Correlation />
<Execution ProcessID="1004" ThreadID="16664" />
<Channel>Security</Channel>
<Computer>FULL.Server.Name</Computer>
<Security />
</System>
<EventData>
<Data Name="PackageName">MICROSOFT_AUTHENTICATION_PACKAGE_V1_0</Data>
<Data Name="TargetUserName">SERVERNAME</Data>
<Data Name="Workstation">SERVERNAME</Data>
<Data Name="Status">0xc0000064</Data>
</EventData>
</Event>

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 1/10/2017 3:06:19 PM
Event ID: 4625
Task Category: Logon
Level: Information
Keywords: Audit Failure
User: N/A
Computer: FULL.Server.Name
Description:
An account failed to log on.

Subject:
Security ID: NULL SID
Account Name: -
Account Domain: -
Logon ID: 0x0

Logon Type: 3

Account For Which Logon Failed:
Security ID: NULL SID
Account Name: SERVERNAME
Account Domain: DOMAIN

Failure Information:
Failure Reason: Unknown user name or bad password.
Status: 0xC000006D
Sub Status: 0xC0000064

Process Information:
Caller Process ID: 0x0
Caller Process Name: -

Network Information:
Workstation Name: SERVERNAME
Source Network Address: ::1
Source Port: 54483

Detailed Authentication Information:
Logon Process: NtLmSsp
Authentication Package: NTLM
Transited Services: -
Package Name (NTLM only): -
Key Length: 0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4625</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>12544</Task>
<Opcode>0</Opcode>
<Keywords>0x8010000000000000</Keywords>
<TimeCreated SystemTime="2017-01-10T20:06:19.664595600Z" />
<EventRecordID>13461331</EventRecordID>
<Correlation />
<Execution ProcessID="1004" ThreadID="16664" />
<Channel>Security</Channel>
<Computer>FULL.Server.Name</Computer>
<Security />
</System>
<EventData>
<Data Name="SubjectUserSid">S-1-0-0</Data>
<Data Name="SubjectUserName">-</Data>
<Data Name="SubjectDomainName">-</Data>
<Data Name="SubjectLogonId">0x0</Data>
<Data Name="TargetUserSid">S-1-0-0</Data>
<Data Name="TargetUserName">SERVERNAME</Data>
<Data Name="TargetDomainName">DOMAIN</Data>
<Data Name="Status">0xc000006d</Data>
<Data Name="FailureReason">%%2313</Data>
<Data Name="SubStatus">0xc0000064</Data>
<Data Name="LogonType">3</Data>
<Data Name="LogonProcessName">NtLmSsp </Data>
<Data Name="AuthenticationPackageName">NTLM</Data>
<Data Name="WorkstationName">SGFS1</Data>
<Data Name="TransmittedServices">-</Data>
<Data Name="LmPackageName">-</Data>
<Data Name="KeyLength">0</Data>
<Data Name="ProcessId">0x0</Data>
<Data Name="ProcessName">-</Data>
<Data Name="IpAddress">::1</Data>
<Data Name="IpPort">54483</Data>
</EventData>
</Event>
 
Last edited:

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
21,056 (6.22/day)
Likes
10,181
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) PowerColor PCS+ 390 8 GiB DVI + HDMI
Storage Crucial MX300 275 GB, Seagate 6 TB 7200 RPM
Display(s) Samsung SyncMaster T240 24" LCD (1920x1200 HDMI) + Samsung SyncMaster 906BW 19" LCD (1440x900 DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek Onboard, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
#2
https://en.wikipedia.org/wiki/Kerberos_(protocol)

Maybe UDP 88 is getting blocked? Pretty sure the second two are chained form the Kerberos protocol issue.


More info on Windows Server specifics:
https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4771

Are you absolutely sure it isn't a user or program trying to login and failing? Pre-authentication is when the password handshake occurs so something as benign as not typing in the password correctly can trigger these events to get logged.

0x18 result code is very likely a bad password.


Since you make it sound like these three events happen repeatedly in rapid succession, my guess is some program is trying to log in to a user account and failing 8 times in a row.
 
Last edited:
Joined
Dec 7, 2005
Messages
947 (0.21/day)
Likes
115
System Name GRAYSCALE\Butterfly
Processor Intel Core i7 8700k @ 5.2Ghz\Intel 4690k
Motherboard ASUS Maximus X Hero \Asus Z97 Maximus Hero VI
Cooling Custom Water\Stock
Memory 2x8GB G.Skill RGB DDR4-3200 \2x8GB Crucial Ballistix DDR3-1600
Video Card(s) NVidia Titan Xp w/ EK Block \ MSI Reference GTX 780
Storage 512GB Samsung 960 PRO (M.2)\128GB OCZ Vertex 4 + 500GB WD Black
Display(s) Asus PG278Q ROG Swift\Acer x213h 21.3'' 1920x1080 LCD
Case Thermaltake P3 Core\NZXT S340
Audio Device(s) Integrated w/ AKG K702 65th Anny's\Integrated
Power Supply Corsair HXi 1000 \Corsair HX850
Mouse Logitech G502 Proteus Spectrum\2014 Razer Naga
Keyboard Ducky One TKL RGB
Software Windows 10 Pro (x64)\Windows 10 Pro (x64)
#3
You haven't deleted any AD accounts recently have you? Ones with odd names?

Is the DC an RODC or writable DC?

This is an authentication error between this particular server and the DC. Quick and easy method to correct it should be re-joining it to the domain.
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
21,056 (6.22/day)
Likes
10,181
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) PowerColor PCS+ 390 8 GiB DVI + HDMI
Storage Crucial MX300 275 GB, Seagate 6 TB 7200 RPM
Display(s) Samsung SyncMaster T240 24" LCD (1920x1200 HDMI) + Samsung SyncMaster 906BW 19" LCD (1440x900 DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek Onboard, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
#4
The dollar sign indicates the user is a computer and not an individual.


Edit: This is literally the key:
Account Information:
Security ID: DOMAIN\SERVERNAME$
Account Name: SERVERNAME$
That computer is trying to log into the domain but it's failing to because the credentials are wrong.
 

Solaris17

Creator Solaris Utility DVD
Joined
Aug 16, 2005
Messages
19,376 (4.27/day)
Likes
6,288
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EKWB L360 R2.0
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Gigabyte 1080TI
Storage 2x SG 3TB HDDs (RAID 0) | 1x ADATA 128 SSD (Cache) | 1x Drevo 256 SSD | 1x 1TB Samsung 850 EVO (OS)
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Realtek ALC 1220 on Audio-Technica ATH-AG1
Power Supply Seasonic 1050W Snow
Mouse Roccat Tyon White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
#5
Looks like these 3 errors are related to each other this seems to be leading back to an authentication issue. Like wastedslayer said have you modified any user accounts?
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#6
Thanks so much for the replies everyone. :toast:

The dollar sign indicates the user is a computer and not an individual.


Edit: This is literally the key:


That computer is trying to log into the domain but it's failing to because the credentials are wrong.
That computer is the computer logging the error :oops:

And, the password for that server's admin account hasn't been changed since day 1....
I don't know how long we've been getting the error, since the early security logged was scrolled out due to a series of mass brute force attacks.... Would love to change the IP, but since the company is national, we'd have SO many clients (many of whom are complete computer morons) who would have to be hand walked thru the connections for several weeks to get them up and running again... but, like I said, different problem.

https://en.wikipedia.org/wiki/Kerberos_(protocol)

Maybe UDP 88 is getting blocked? Pretty sure the second two are chained form the Kerberos protocol issue.

.....


Since you make it sound like these three events happen repeatedly in rapid succession, my guess is some program is trying to log in to a user account and failing 8 times in a row.
I agree, that's what it appears to be... I just can't figure out what. And I'm not sure how to proceed with sorting it... There's a different error code for failed user logins for RDP, which is its primary purpose.

You haven't deleted any AD accounts recently have you? Ones with odd names?

Is the DC an RODC or writable DC?

This is an authentication error between this particular server and the DC. Quick and easy method to correct it should be re-joining it to the domain.
This client actually has hackers trying to attach to their server, but I'm running wireshark and snagging all of them as they show up (Ukraine, France, Poland, etc). This never actually leaves the server: it's totally an internal error.

Maybe verifying that it's connected properly to the domain is an avenue I need to look at.. I'll dig in when I get to the office.

thanks all :)
 
Last edited:

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
21,056 (6.22/day)
Likes
10,181
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) PowerColor PCS+ 390 8 GiB DVI + HDMI
Storage Crucial MX300 275 GB, Seagate 6 TB 7200 RPM
Display(s) Samsung SyncMaster T240 24" LCD (1920x1200 HDMI) + Samsung SyncMaster 906BW 19" LCD (1440x900 DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek Onboard, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
#7
That computer is the computer logging the error :oops:


Got no ideas on that one. I'd probably nuke that server and start over.
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#8
I believe I tied it down. We had some interesting activity overnight from Moscow and Oregon lol, so I was prowling thru the logs and users to make sure something hadn't been compromised, and noticed that the krbtgt user itself was disabled, and I'm betting that's where my issue lies. Since we didn't set up an RODC, I don't think this account is needed at all, so I'm going to leave it deactivated. I'm not positive that's the best solution, since I'd still like to find out what's trying to call that user for login, but I'm assuming it's in some way related to (mis-)setup between the App server and the DC. thanks all for the inputs, and feel free if I'm making a grave mistake to point it out :) thanks again!
 
Joined
Dec 7, 2005
Messages
947 (0.21/day)
Likes
115
System Name GRAYSCALE\Butterfly
Processor Intel Core i7 8700k @ 5.2Ghz\Intel 4690k
Motherboard ASUS Maximus X Hero \Asus Z97 Maximus Hero VI
Cooling Custom Water\Stock
Memory 2x8GB G.Skill RGB DDR4-3200 \2x8GB Crucial Ballistix DDR3-1600
Video Card(s) NVidia Titan Xp w/ EK Block \ MSI Reference GTX 780
Storage 512GB Samsung 960 PRO (M.2)\128GB OCZ Vertex 4 + 500GB WD Black
Display(s) Asus PG278Q ROG Swift\Acer x213h 21.3'' 1920x1080 LCD
Case Thermaltake P3 Core\NZXT S340
Audio Device(s) Integrated w/ AKG K702 65th Anny's\Integrated
Power Supply Corsair HXi 1000 \Corsair HX850
Mouse Logitech G502 Proteus Spectrum\2014 Razer Naga
Keyboard Ducky One TKL RGB
Software Windows 10 Pro (x64)\Windows 10 Pro (x64)
#9
I believe I tied it down. We had some interesting activity overnight from Moscow and Oregon lol, so I was prowling thru the logs and users to make sure something hadn't been compromised, and noticed that the krbtgt user itself was disabled, and I'm betting that's where my issue lies. Since we didn't set up an RODC, I don't think this account is needed at all, so I'm going to leave it deactivated. I'm not positive that's the best solution, since I'd still like to find out what's trying to call that user for login, but I'm assuming it's in some way related to (mis-)setup between the App server and the DC. thanks all for the inputs, and feel free if I'm making a grave mistake to point it out :) thanks again!
Just a heads up, the krbtgt account should be disabled. Even without an RODC.

See: https://technet.microsoft.com/en-us/library/dn745899.aspx#Sec_KRBTGT

Your conclusion is correct though. It's an authentication error between this server and your DC. I've seen similar errors before on servers when a couple other key generation accounts (for an RODC) were accidentally deleted. The gist of it was that the computer thinks it's authenticated, but the DC says no. As mentioned, a quick re-join will resolve the issue.
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#10
Just a heads up, the krbtgt account should be disabled. Even without an RODC.

See: https://technet.microsoft.com/en-us/library/dn745899.aspx#Sec_KRBTGT

Your conclusion is correct though. It's an authentication error between this server and your DC. I've seen similar errors before on servers when a couple other key generation accounts (for an RODC) were accidentally deleted. The gist of it was that the computer thinks it's authenticated, but the DC says no. As mentioned, a quick re-join will resolve the issue.
I haven't had time to jump back in here, but I was digging around some more earlier today, and came across an... anomaly....

The best that I can tell (and I hate AD and DC, so my knowledge on this is less than it should be), both servers in this location are convinced that they are the domain controller.

I found the issue when I went to look at re-adding the app server to the domain as recommended above. However, I immediately received an error that I would need to demote this domain controller before moving it from one domain to another... o_O....

I was a little unsure how to verify what I was seeing, but in trying to go to the next screen, I do not have the ability to change the domain name at all. So, I ran DCDIAG, and this server believes itself to be the "Home Server" for the domain.

Thinking "Ok, this server must have grabbed control when the DC went down... which none of us recall happening.. ever..." So, I checked the actual Domain controller. Which also thinks it is the "Home server" for the same domain.... In jumping back to the "impostor", I discovered that all the users are now listed in the app server. They are also changing up who controls users. Sometimes, the impostor registers the change first. Sometimes, the Real DC does it.... and sometimes, they match....

True.JPG impostor.JPG


Now I'm googling "Domain Controller Hostile Takeover"......:banghead:
 
Joined
Dec 7, 2005
Messages
947 (0.21/day)
Likes
115
System Name GRAYSCALE\Butterfly
Processor Intel Core i7 8700k @ 5.2Ghz\Intel 4690k
Motherboard ASUS Maximus X Hero \Asus Z97 Maximus Hero VI
Cooling Custom Water\Stock
Memory 2x8GB G.Skill RGB DDR4-3200 \2x8GB Crucial Ballistix DDR3-1600
Video Card(s) NVidia Titan Xp w/ EK Block \ MSI Reference GTX 780
Storage 512GB Samsung 960 PRO (M.2)\128GB OCZ Vertex 4 + 500GB WD Black
Display(s) Asus PG278Q ROG Swift\Acer x213h 21.3'' 1920x1080 LCD
Case Thermaltake P3 Core\NZXT S340
Audio Device(s) Integrated w/ AKG K702 65th Anny's\Integrated
Power Supply Corsair HXi 1000 \Corsair HX850
Mouse Logitech G502 Proteus Spectrum\2014 Razer Naga
Keyboard Ducky One TKL RGB
Software Windows 10 Pro (x64)\Windows 10 Pro (x64)
#11
Why are you using a DC as an App server? This sounds like it could be a more wide spread issue then just these two particular machines. It's likely that domain authentication, both user and computer, is skewed across the board.

Do you have multiple domains? How many DCs in total are there?

The way AD works there isn't really a "Primary DC" per se, rather if updates are made on one DC they are synced with other DC's. We wont get into discussions about FSMO roles, because that's not part of the problem... yet.
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#12
Have you tried restarting the App Server since your other Domain Controller went down? - Classic IT answer, but worth a try.

@Wastedslayer - aha, FSMO roles. I was thinking PDC Emulator when first reading although, sounds like user sign-in elsewhere is unaffected.
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#13
Why are you using a DC as an App server? This sounds like it could be a more wide spread issue then just these two particular machines. It's likely that domain authentication, both user and computer, is skewed across the board.

Do you have multiple domains? How many DCs in total are there?

The way AD works there isn't really a "Primary DC" per se, rather if updates are made on one DC they are synced with other DC's. We wont get into discussions about FSMO roles, because that's not part of the problem... yet.
Hahaha not intentionally using the app server as a DC :) It managed to self-promote itself, best we can tell. I'm still trying to figure out "how", and what will happen if I demote it.

This location only has 2 servers that are involved in this mess. Originally, one server was set as the Domain controller with DHCP (which we moved to the router right before "live"), and the second was setup as an RDP/app server, and since M$ does its best to force it, it's the fallback DC. Somewhere along the line, within, I'd guess, the last 4 months or so, it looks like the App server decided it needed to be the DC. I have no other explanation for what I'm seeing.... I don't know of any better way to check to verify that a server is acting as a DC other than the DCDiag I ran, which told me that on both servers, they believe they are the "Home" server...
So, 2 servers in play, one supposedly the DC, one an App/RDP server, one domain, but both appear to be in the role of DC. Same building, physical network, within one number of each other IP-wise.... thoughts ? :)



Have you tried restarting the App Server since your other Domain Controller went down? - Classic IT answer, but worth a try.

....
I hear ya...This is a multinational, and with users in several time zones, plus the management working at complete random times of the day and night (2:00 AM on a saturday morning their time?? really??!! *sigh*), we try not to reboot except for major app updates, or severe problems... And since this doesn't really seem to be breaking anything... I didn't want to push it...

extra note, best we can tell, the "Real" DC really hasn't gone down in around a year. We've got them both on monster UPSs, and while the company hasn't gone forward with the generator plan, that's really only because the systems haven't been down again since last years major power outage to push them to do it... If the servers had gone down again, we'd have a generator up.

So, I'm not sure what event would have prompted the backup to grab control, and I didn't know that you could force a promotion to DC if a DC already existed in the forest.

I think it's a conspiracy... my google searches for "Domain Controller Hostile Takeover" don't return any results... I think I'm being "filtered"....
 
Last edited:
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#14
Hahaha not intentionally using the app server as a DC :) It managed to self-promote itself, best we can tell. I'm still trying to figure out "how", and what will happen if I demote it.

This location only has 2 servers that are involved in this mess. Originally, one server was set as the Domain controller with DHCP (which we moved to the router right before "live"), and the second was setup as an RDP/app server, and since M$ does its best to force it, it's the fallback DC. Somewhere along the line, within, I'd guess, the last 4 months or so, it looks like the App server decided it needed to be the DC. I have no other explanation for what I'm seeing.... I don't know of any better way to check to verify that a server is acting as a DC other than the DCDiag I ran, which told me that on both servers, they believe they are the "Home" server...
So, 2 servers in play, one supposedly the DC, one an App/RDP server, one domain, but both appear to be in the role of DC. Same building, physical network, within one number of each other IP-wise.... thoughts ? :)
The only way a server could be promoted to a DC is by installing AD DS and promoting it through Server Manager. By saying "Decided it needed to be a DC" do you mean you saw a yellow triangle in Server Manager asking for you to promote?

Are you able to login to the RDS Server with the Local Admin account? - Check DNS Settings on the server to see that it can communicate with your other DC. Although there's no Primary DC (like what @Wastedslayer said) the first created DC will have the FSMO roles. And if your server can't communicate with the DC that has them, it will fail sign-in.
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#15
The only way a server could be promoted to a DC is by installing AD DS and promoting it through Server Manager. By saying "Decided it needed to be a DC" do you mean you saw a yellow triangle in Server Manager asking for you to promote?

Are you able to login to the RDS Server with the Local Admin account? - Check DNS Settings on the server to see that it can communicate with your other DC. Although there's no Primary DC (like what @Wastedslayer said) the first created DC will have the FSMO roles. And if your server can't communicate with the DC that has them, it will fail sign-in.
Sorry, see additional notes in response to your earlier post. I'll try to create a local admin only on both servers, and see what lets me login....

This is probably a complete misunderstanding on my part, but when we initially set these servers in place, it looked like M$ wouldn't really allow us to setup an RDP 2012 std server without creating a Domain, and having a "fallback" DC, especially since we really didn't want to make the App server a DC. Now, since then, we've been able to wiggle around it by loading the terminal services/remote desktop service without all the nice bells and whistles, managing the users thru command prompts when we need to remote to them. But for this one, our 1st 2012, we went with the "recommended" build, to minimize ghetto hacking the server.

the main point of that tl;dr, is that we believed that the second server was required to act as "fallback" DC, in case the "actual" DC failed for whatever reason, thereby allowing users to continue to login for normal operations, until recovery of the original DC. That's the only way we know of that the App server could have gotten promoted: somehow, it decided the Real DC had failed, and promoted itself...
 

Solaris17

Creator Solaris Utility DVD
Joined
Aug 16, 2005
Messages
19,376 (4.27/day)
Likes
6,288
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EKWB L360 R2.0
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Gigabyte 1080TI
Storage 2x SG 3TB HDDs (RAID 0) | 1x ADATA 128 SSD (Cache) | 1x Drevo 256 SSD | 1x 1TB Samsung 850 EVO (OS)
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Realtek ALC 1220 on Audio-Technica ATH-AG1
Power Supply Seasonic 1050W Snow
Mouse Roccat Tyon White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
#16
This is a mess. You need to work the problem backwards. You need to run a dcdiag and DNS tests to make sure communication and credentials are functioning normally. Its ok (and recommended) to have more than 1 DC in a forest. You should NOT demote that machine unless you are CERTAIN it is not running critical FSMO roles.

After you make sure the DCs can communicate and sync you can work on why the unit is giving you those errors. It seems at this point their are multiple issues, for all we know the domain isnt compromised and instead just configured incorrectly.
 
Joined
Dec 7, 2005
Messages
947 (0.21/day)
Likes
115
System Name GRAYSCALE\Butterfly
Processor Intel Core i7 8700k @ 5.2Ghz\Intel 4690k
Motherboard ASUS Maximus X Hero \Asus Z97 Maximus Hero VI
Cooling Custom Water\Stock
Memory 2x8GB G.Skill RGB DDR4-3200 \2x8GB Crucial Ballistix DDR3-1600
Video Card(s) NVidia Titan Xp w/ EK Block \ MSI Reference GTX 780
Storage 512GB Samsung 960 PRO (M.2)\128GB OCZ Vertex 4 + 500GB WD Black
Display(s) Asus PG278Q ROG Swift\Acer x213h 21.3'' 1920x1080 LCD
Case Thermaltake P3 Core\NZXT S340
Audio Device(s) Integrated w/ AKG K702 65th Anny's\Integrated
Power Supply Corsair HXi 1000 \Corsair HX850
Mouse Logitech G502 Proteus Spectrum\2014 Razer Naga
Keyboard Ducky One TKL RGB
Software Windows 10 Pro (x64)\Windows 10 Pro (x64)
#17
IF you want to try seizing the FSMO roles to see if it will help, here is a quick guide (this should be done on the DC you want to keep):

https://blogs.technet.microsoft.com...ation-master-roles-in-windows-server-2012-r2/

Knowing that this is a DC now I'm really convinced that something is wrong with the AD account for this particular server. It's possible that the promotion to a DC didn't work correctly leaving you with a sort of broken DC.

A reboot is always worth a shot; has that been tried Ahhzz? If it has, then the next step would be to demote this DC and rejoin it to the domain. Once rejoined you can run DCPromo again. Microsoft best practice is to have at least 2 DCs for redundancy.
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#18
IF you want to try seizing the FSMO roles to see if it will help, here is a quick guide (this should be done on the DC you want to keep):

https://blogs.technet.microsoft.com...ation-master-roles-in-windows-server-2012-r2/

Knowing that this is a DC now I'm really convinced that something is wrong with the AD account for this particular server. It's possible that the promotion to a DC didn't work correctly leaving you with a sort of broken DC.

A reboot is always worth a shot; has that been tried Ahhzz? If it has, then the next step would be to demote this DC and rejoin it to the domain. Once rejoined you can run DCPromo again. Microsoft best practice is to have at least 2 DCs for redundancy.
Also don't forget to reset the account in AD Users and Computers to avoid any SID issues.
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#19
Sorry, see additional notes in response to your earlier post. I'll try to create a local admin only on both servers, and see what lets me login....

This is probably a complete misunderstanding on my part, but when we initially set these servers in place, it looked like M$ wouldn't really allow us to setup an RDP 2012 std server without creating a Domain, and having a "fallback" DC, especially since we really didn't want to make the App server a DC. Now, since then, we've been able to wiggle around it by loading the terminal services/remote desktop service without all the nice bells and whistles, managing the users thru command prompts when we need to remote to them. But for this one, our 1st 2012, we went with the "recommended" build, to minimize ghetto hacking the server.

the main point of that tl;dr, is that we believed that the second server was required to act as "fallback" DC, in case the "actual" DC failed for whatever reason, thereby allowing users to continue to login for normal operations, until recovery of the original DC. That's the only way we know of that the App server could have gotten promoted: somehow, it decided the Real DC had failed, and promoted itself...
No worries, sorry.

You say its your 1st 2012 server. Whats the other DC running?
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#20
No worries, sorry.

You say its your 1st 2012 server. Whats the other DC running?
After this one, we managed to sort out how to avoid setting up a DC, so none of our other clients with 2012 are running a DC.

Or, if that's not the question you meant to ask, both servers at this location are running 2012 R2. Only one is supposed to be a DC :)
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#21
After this one, we managed to sort out how to avoid setting up a DC, so none of our other clients with 2012 are running a DC.

Or, if that's not the question you meant to ask, both servers at this location are running 2012 R2. Only one is supposed to be a DC :)
The second one :p

To check what FSMO roles our on what server, run 'netdom fsmo query' using Command Prompt. It will tell you where all 5 roles are located.
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#22
IF you want to try seizing the FSMO roles to see if it will help, here is a quick guide (this should be done on the DC you want to keep):

https://blogs.technet.microsoft.com...ation-master-roles-in-windows-server-2012-r2/

Knowing that this is a DC now I'm really convinced that something is wrong with the AD account for this particular server. It's possible that the promotion to a DC didn't work correctly leaving you with a sort of broken DC.

A reboot is always worth a shot; has that been tried Ahhzz? If it has, then the next step would be to demote this DC and rejoin it to the domain. Once rejoined you can run DCPromo again. Microsoft best practice is to have at least 2 DCs for redundancy.
We'll probably run a reboot this weekend, and warn users there will be a short down time.

As for keeping them both as DCs, I really don't want to do that. Didn't want a DC in the first place since all we're using it for is managing users and passwords. We'd have been content (and intended at first blush) to only have a single App/RDP server. It's only that when we started the prep work for replacing their old server, it looked like the only "legitimate" way to run RDP was to setup two servers in AD, one as DC and the other as RDP/App.

If a reboot doesn't resolve our issue, I'll probably plan another weekend to demote the app server, rejoin the domain, and leave it that way.

I agree with your statement that the promotion to DC didn't work correctly, mainly because we didn't do it lol.

Can someone answer specifically, can you correctly run 2 domain controllers, of the same domain, in the same subnet/network? IE, ServerA and ServerB both DC for the domain Server.domain.controls.com? I thought this was a bad thing, and caused things like what I'm seeing with them randomly swapping out "management" of the user profiles?
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#23
We'll probably run a reboot this weekend, and warn users there will be a short down time.

As for keeping them both as DCs, I really don't want to do that. Didn't want a DC in the first place since all we're using it for is managing users and passwords. We'd have been content (and intended at first blush) to only have a single App/RDP server. It's only that when we started the prep work for replacing their old server, it looked like the only "legitimate" way to run RDP was to setup two servers in AD, one as DC and the other as RDP/App.

If a reboot doesn't resolve our issue, I'll probably plan another weekend to demote the app server, rejoin the domain, and leave it that way.

I agree with your statement that the promotion to DC didn't work correctly, mainly because we didn't do it lol.

Can someone answer specifically, can you correctly run 2 domain controllers, of the same domain, in the same subnet/network? IE, ServerA and ServerB both DC for the domain Server.domain.controls.com? I thought this was a bad thing, and caused things like what I'm seeing with them randomly swapping out "management" of the user profiles?
You can have as many DCs as you want. It doesn't matter if they're on the same subnet. However, you should only have domain controllers working as domain controllers, cos if they ever go tits up, you can just demote them and start again with minimal admin work in setting up different roles and features. If your DC wasn't running RDS, we would of just told you to demote it and start again. Probably saving you some hassle and time.

I hope you don't mind me asking, do you have any MS Certs for Server?
 
Joined
Feb 27, 2008
Messages
4,279 (1.18/day)
Likes
3,635
System Name Ironic
Processor Intel 2500k 4.4Ghz
Motherboard ASROCK|Z68 PROFESSIONAL Gen 3
Cooling Corsair H60
Memory 32GB GSkill Ripjaw X 1866
Video Card(s) Sapphire R9 290 Vapor-X 4Gb
Storage Western Digital Caviar Black 2TB SATA 3 (6G/s)
Display(s) 22" Dell Wide/ 22" Acer wide/24" Asus
Case Antec Lanboy Air Black & Blue
Audio Device(s) SB Audigy 7.1
Power Supply Corsair Enthusiast TX750
Mouse Logitech G9x, custom frame
Keyboard Roccat Ryos MK
Software Win 7 Ult 64 bit (with a side of XP64)
#24
The second one :p

To check what FSMO roles our on what server, run 'netdom fsmo query' using Command Prompt. It will tell you where all 5 roles are located.
Can't seem to find the "Big Thanks!" button.... even if you did mix the command :p . :respect:

"netdom query fsmo" tells me that the desired DC (DC1 ), does seem to be in charge from both computers, so that does answer a large question for me. The response from both servers tells me DC1 is the Schema master and the Domain naming master, so that eliminates one of my concerns. I do appear to only have one DC in my forest, and it's the one I wanted, or at least the one we setup that way heheh.

Which rolls back a lot of statements here, since I'm no longer chewing on how I got 2 DCs in one forest. *sigh* ok guys, I really appreciate all the responses, but it looks like I've been dragging you all around an elephant, insisting it's a tree, and asking how it got there. At this point, I know it's not a tree, but I still gotta figure out what the elephant is doing. I'll dig in some more, and post back with results... And I think the first thing I want to do is schedule that reboot this weekend....

*cheers* to all....:toast:
 
Joined
Jun 14, 2016
Messages
40 (0.07/day)
Likes
26
Location
Cornwall, UK
Processor Intel Core i7 6700K
Motherboard Gigabyte Z170 Gaming K3
Cooling Coolermaster Hyper TX3 Evo
Memory Corsair Vengeance LPX (Red) 2x8GB 2400MHz
Video Card(s) MSI GeForce GTX 1070 Founders Edition
Storage 1TB WD Blue 7200rpm
Display(s) 2x Acer K222HQL 1080p
Case Corsair Spec-01
Power Supply EVGA SuperNova G2 650W
Mouse Asus Cerberus Mouse
Keyboard Asus Cerberus Keyboard
Software Windows 10 Pro 64-bit
#25
Can't seem to find the "Big Thanks!" button.... even if you did mix the command :p . :respect:

"netdom query fsmo" tells me that the desired DC (DC1 ), does seem to be in charge from both computers, so that does answer a large question for me. The response from both servers tells me DC1 is the Schema master and the Domain naming master, so that eliminates one of my concerns. I do appear to only have one DC in my forest, and it's the one I wanted, or at least the one we setup that way heheh.

Which rolls back a lot of statements here, since I'm no longer chewing on how I got 2 DCs in one forest. *sigh* ok guys, I really appreciate all the responses, but it looks like I've been dragging you all around an elephant, insisting it's a tree, and asking how it got there. At this point, I know it's not a tree, but I still gotta figure out what the elephant is doing. I'll dig in some more, and post back with results... And I think the first thing I want to do is schedule that reboot this weekend....

*cheers* to all....:toast:
Sorry :D - Trying to do stuff from memory :p