Random Server 2003 Crashes

Just logged in to a box we have running Windows Server 2003 SP2 and was greeted with that lovely 'Why did the system restart' message.

The machine is less than a year old and we ordered 3 of them. 1 is running RH Linux and the other 2 Windows Server 2003 Standard.

The first windows machine had lots of intermittant crashes and was sent back and a replacement was shipped. The replacement seems to be successful and all is well with that machine. The second never had any bugchecks ... until today!

Below is the debuglog.txt generated. Was wondering if anyone can help source out the issues. these machines are running W2003 Standard Edition and are mainly used as low load Web (IIS 6, PHP & ASP.NET 2&3), Database (MySQL 5.1.22) and Email (Mercury & hMailserver) servers (combined).

Further, MySQL seems to be crashing a lot as well. Event log has this message:

The application, c:\mysql\bin\mysqld.exe, generated an application error The error occurred on 10/20/2008 @ 08:30:12.762 The exception generated was c0000005 at address 006290CB (mysqld!hash_first)


HARDWARE
Intel Dual Core E2160 1.8GHz
2GB RAM
Biostar P4M900-M7 SE mobo
Dual Seagate 250GB SATA drives RAID1

debuglog.txt:

Opened log file 'c:debuglog.txt'

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\WINDOWS\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available

Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: C:\WINDOWS;C:\WINDOWS\system32;C:\WINDOWS\system32\drivers
Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (2 procs) Free x86 compatible
Product: Server, suite: TerminalServer SingleUserTS
Built by: 3790.srv03_sp2_gdr.070304-2240
Kernel base = 0x80800000 PsLoadedModuleList = 0x808a6ea8
Debug session time: Tue Oct 21 11:02:07.590 2008 (GMT-4)
System Uptime: 7 days 0:51:51.531
Loading Kernel Symbols
.....................................................................................................
Loading User Symbols
PEB is paged out (Peb.Ldr = 7ffd400c). Type ".hh dbgerr001" for details
Loading unloaded module list
........
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 8E, {c0000005, 80934392, b9fa0c84, 0}

PEB is paged out (Peb.Ldr = 7ffd400c). Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffd400c). Type ".hh dbgerr001" for details
Probably caused by : ntkrpamp.exe ( nt!ObpCloseHandleTableEntry+16 )

Followup: MachineOwner
---------

0: kd> !analyze -v;r;kv;lmtn;.logclose;q
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

KERNEL_MODE_EXCEPTION_NOT_HANDLED (8e)
This is a very common bugcheck. Usually the exception address pinpoints
the driver/function that caused the problem. Always note this address
as well as the link date of the driver/image that contains this address.
Some common problems are exception code 0x80000003. This means a hard
coded breakpoint or assertion was hit, but this system was booted
/NODEBUG. This is not supposed to happen as developers should never have
hardcoded breakpoints in retail code, but ...
If this happens, make sure a debugger gets connected, and the
system is booted /DEBUG. This will let us see why this breakpoint is
happening.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: 80934392, The address that the exception occurred at
Arg3: b9fa0c84, Trap Frame
Arg4: 00000000

Debugging Details:
------------------

PEB is paged out (Peb.Ldr = 7ffd400c). Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffd400c). Type ".hh dbgerr001" for details

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx ". The memory could not be "%s ".

FAULTING_IP:
nt!ObpCloseHandleTableEntry+16
80934392 8b80a8000000 mov eax,dword ptr [eax+0A8h]

TRAP_FRAME: b9fa0c84 -- (.trap 0xffffffffb9fa0c84)
.trap 0xffffffffb9fa0c84
ErrCode = 00000000
eax=4d013101 ebx=e1255f30 ecx=00000000 edx=88837d61 esi=88837d60 edi=e24b7940
eip=80934392 esp=b9fa0cf8 ebp=b9fa0d04 iopl=0 nv up ei ng nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010286
nt!ObpCloseHandleTableEntry+0x16:
80934392 8b80a8000000 mov eax,dword ptr [eax+0A8h] ds:0023:4d0131a9=????????
.trap
Resetting default scope

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0x8E

PROCESS_NAME: mysqld.exe

CURRENT_IRQL: 0

LAST_CONTROL_TRANSFER: from 8082d800 to 80827c63

STACK_TEXT:
b9fa0850 8082d800 0000008e c0000005 80934392 nt!KeBugCheckEx+0x1b
b9fa0c14 8088a262 b9fa0c30 00000000 b9fa0c84 nt!KiDispatchException+0x3a2
b9fa0c7c 8088a216 b9fa0d04 80934392 badb0d00 nt!CommonDispatchException+0x4a
b9fa0c98 809396fe 8857e2f0 ee657645 89b93c00 nt!KiExceptionExit+0x186
b9fa0d04 80934546 e1255f30 e24b7940 00038ca0 nt!ObpFreeObject+0x192
b9fa0d48 80934663 00038ca0 00000001 b9fa0d64 nt!ObpCloseHandle+0x82
b9fa0d58 8088978c 00038ca0 067ef064 7c8285ec nt!NtClose+0x1b
b9fa0d58 7c8285ec 00038ca0 067ef064 7c8285ec nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
067ef064 00000000 00000000 00000000 00000000 0x7c8285ec


STACK_COMMAND: kb

FOLLOWUP_IP:
nt!ObpCloseHandleTableEntry+16
80934392 8b80a8000000 mov eax,dword ptr [eax+0A8h]

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: nt!ObpCloseHandleTableEntry+16

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

IMAGE_NAME: ntkrpamp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 45ec0a19

FAILURE_BUCKET_ID: 0x8E_nt!ObpCloseHandleTableEntry+16

BUCKET_ID: 0x8E_nt!ObpCloseHandleTableEntry+16

Followup: MachineOwner
---------

eax=ffdff13c ebx=80934392 ecx=00000001 edx=8088ed62 esi=ffdff120 edi=00000000
eip=80827c63 esp=b9fa0838 ebp=b9fa0850 iopl=0 nv up ei ng nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00000286
nt!KeBugCheckEx+0x1b:
80827c63 5d pop ebp
ChildEBP RetAddr Args to Child
b9fa0850 8082d800 0000008e c0000005 80934392 nt!KeBugCheckEx+0x1b (FPO: [Non-Fpo])
b9fa0c14 8088a262 b9fa0c30 00000000 b9fa0c84 nt!KiDispatchException+0x3a2 (FPO: [Non-Fpo])
b9fa0c7c 8088a216 b9fa0d04 80934392 badb0d00 nt!CommonDispatchException+0x4a (FPO: [0,20,0])
b9fa0c98 809396fe 8857e2f0 ee657645 89b93c00 nt!KiExceptionExit+0x186
b9fa0d04 80934546 e1255f30 e24b7940 00038ca0 nt!ObpFreeObject+0x192 (FPO: [Non-Fpo])
b9fa0d48 80934663 00038ca0 00000001 b9fa0d64 nt!ObpCloseHandle+0x82 (FPO: [Non-Fpo])
b9fa0d58 8088978c 00038ca0 067ef064 7c8285ec nt!NtClose+0x1b (FPO: [Non-Fpo])
b9fa0d58 7c8285ec 00038ca0 067ef064 7c8285ec nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ b9fa0d64)
WARNING: Frame IP not in any known module. Following frames may be wrong.
067ef064 00000000 00000000 00000000 00000000 0x7c8285ec
start end module name
80800000 80a56000 nt ntkrpamp.exe Mon Mar 05 07:16:25 2007 (45EC0A19)
80a56000 80a82000 hal halmacpi.dll Sat Feb 17 00:48:26 2007 (45D6972A)
8c0fc000 8c126000 Fastfat Fastfat.SYS Sat Feb 17 01:27:55 2007 (45D6A06B)
b0609000 b0634000 RDPWD RDPWD.SYS Sat Feb 17 00:44:38 2007 (45D69646)
b078c000 b0797000 TDTCP TDTCP.SYS Sat Feb 17 00:44:32 2007 (45D69640)
b088c000 b08e9000 srv srv.sys Sat Feb 17 01:27:20 2007 (45D6A048)
b0961000 b09b1000 HTTP HTTP.sys Sat Feb 17 01:28:12 2007 (45D6A07C)
b9a0d000 b9a29100 dump_viamraid dump_viamraid.sys Tue Jul 17 01:35:21 2007 (469C5519)
b9a2a000 b9a3f000 Cdfs Cdfs.SYS Sat Feb 17 01:27:08 2007 (45D6A03C)
b9a3f000 b9a50000 Fips Fips.SYS Sat Feb 17 01:26:33 2007 (45D6A019)
b9a50000 b9ac6000 mrxsmb mrxsmb.sys Sat Feb 17 01:28:15 2007 (45D6A07F)
b9ac6000 b9af6000 rdbss rdbss.sys Sat Feb 17 01:27:37 2007 (45D6A059)
b9af6000 b9b20000 afd afd.sys Fri Jun 20 10:20:58 2008 (485BBCCA)
b9b20000 b9b51000 netbt netbt.sys Sat Feb 17 01:28:57 2007 (45D6A0A9)
b9c71000 b9c7b000 Dxapi Dxapi.sys Tue Mar 25 03:06:01 2003 (3E7FFFD9)
b9fd2000 ba062000 tcpip tcpip.sys Fri Jun 20 10:20:25 2008 (485BBCA9)
ba06c000 ba085000 ipsec ipsec.sys Sat Feb 17 01:29:28 2007 (45D6A0C8)
ba184000 ba191000 Npfs Npfs.SYS Sat Feb 17 00:50:36 2007 (45D697AC)
ba194000 ba19f000 Msfs Msfs.SYS Sat Feb 17 00:50:33 2007 (45D697A9)
ba1a4000 ba1b1000 wanarp wanarp.sys Sat Feb 17 00:59:17 2007 (45D699B5)
ba1e4000 ba1f1000 netbios netbios.sys Sat Feb 17 00:58:29 2007 (45D69985)
ba27c000 ba284000 rasacd rasacd.sys Tue Mar 25 03:11:50 2003 (3E800136)
ba31f000 ba32d000 msgpc msgpc.sys Sat Feb 17 00:58:37 2007 (45D6998D)
ba35f000 ba369000 dump_diskdump dump_diskdump.sys Sat Feb 17 01:07:44 2007 (45D69BB0)
ba605000 ba618000 i8042prt i8042prt.sys Sat Feb 17 01:30:40 2007 (45D6A110)
ba618000 ba62d000 usbhub usbhub.sys Sat Feb 17 01:13:05 2007 (45D69CF1)
ba6e4000 ba743000 update update.sys Mon May 28 08:15:16 2007 (465AC7D4)
ba743000 ba77a000 rdpdr rdpdr.sys Sat Feb 17 00:51:00 2007 (45D697C4)
ba77a000 ba78c000 raspptp raspptp.sys Sat Feb 17 01:29:20 2007 (45D6A0C0)
ba78c000 ba7a5000 ndiswan ndiswan.sys Sat Feb 17 01:29:22 2007 (45D6A0C2)
ba7a5000 ba7b9000 rasl2tp rasl2tp.sys Sat Feb 17 01:29:02 2007 (45D6A0AE)
ba7cc000 ba7e1000 serial serial.sys Sat Feb 17 01:06:46 2007 (45D69B76)
ba7e1000 ba80b000 USBPORT USBPORT.SYS Sat Feb 17 01:12:59 2007 (45D69CEB)
ba80b000 ba832000 ks ks.sys Sat Feb 17 01:30:40 2007 (45D6A110)
ba832000 ba846000 redbook redbook.sys Sat Feb 17 01:07:26 2007 (45D69B9E)
ba846000 ba85b000 cdrom cdrom.sys Sat Feb 17 01:07:48 2007 (45D69BB4)
ba85b000 ba877000 VIDEOPRT VIDEOPRT.SYS Sat Feb 17 01:10:30 2007 (45D69C56)
ba877000 ba92c000 S3gIGPm S3gIGPm.sys Wed Jul 11 01:08:47 2007 (469465DF)
ba964000 ba96c000 RDPCDD RDPCDD.sys Tue Mar 25 03:03:05 2003 (3E7FFF29)
ba974000 ba97c000 mnmdd mnmdd.SYS Tue Mar 25 03:07:53 2003 (3E800049)
ba984000 ba98a300 HIDPARSE HIDPARSE.SYS Sat Feb 17 01:12:35 2007 (45D69CD3)
ba98c000 ba993000 Beep Beep.SYS Tue Mar 25 03:03:04 2003 (3E7FFF28)
ba994000 ba99b000 Null Null.SYS Tue Mar 25 03:03:05 2003 (3E7FFF29)
ba99c000 ba9a4000 Fs_Rec Fs_Rec.SYS Tue Mar 25 03:08:36 2003 (3E800074)
bab65000 bab6c000 dxgthk dxgthk.sys Tue Mar 25 03:05:52 2003 (3E7FFFD0)
bf800000 bf9d0000 win32k win32k.sys Wed Mar 19 07:31:40 2008 (47E0F99C)
bf9d0000 bf9e7000 dxg dxg.sys Sat Feb 17 01:14:39 2007 (45D69D4F)
bf9e7000 bfac6000 S3gIGP S3gIGP.dll Wed Jul 11 01:12:41 2007 (469466C9)
bfac6000 bfd66000 s3ginv s3ginv.dll Wed Jul 11 01:11:01 2007 (46946665)
bff60000 bff7e000 RDPDD RDPDD.dll Sat Feb 17 09:01:19 2007 (45D70AAF)
bffa0000 bffea000 ATMFD ATMFD.DLL Sat Feb 17 08:59:31 2007 (45D70A43)
f7208000 f722e000 KSecDD KSecDD.sys Sat Feb 17 00:46:32 2007 (45D696B8)
f722e000 f7253000 fltmgr fltmgr.sys Sat Feb 17 00:51:08 2007 (45D697CC)
f7253000 f7266000 CLASSPNP CLASSPNP.SYS Sat Feb 17 01:28:16 2007 (45D6A080)
f7266000 f7285000 SCSIPORT SCSIPORT.SYS Sat Feb 17 01:28:41 2007 (45D6A099)
f7285000 f72a1100 viamraid viamraid.sys Tue Jul 17 01:35:21 2007 (469C5519)
f72a2000 f72bf000 atapi atapi.sys Sat Feb 17 01:07:34 2007 (45D69BA6)
f72bf000 f72e9000 volsnap volsnap.sys Sat Feb 17 01:08:23 2007 (45D69BD7)
f72e9000 f7315000 dmio dmio.sys Sat Feb 17 01:10:44 2007 (45D69C64)
f7315000 f733c000 ftdisk ftdisk.sys Sat Feb 17 01:08:05 2007 (45D69BC5)
f733c000 f7352000 pci pci.sys Sat Feb 17 00:59:03 2007 (45D699A7)
f7352000 f7386000 ACPI ACPI.sys Sat Feb 17 00:58:47 2007 (45D69997)
f7487000 f7490000 WMILIB WMILIB.SYS Tue Mar 25 03:13:00 2003 (3E80017C)
f7497000 f74a6000 isapnp isapnp.sys Sat Feb 17 00:58:57 2007 (45D699A1)
f74a7000 f74b4000 PCIIDEX PCIIDEX.SYS Sat Feb 17 01:07:32 2007 (45D69BA4)
f74b7000 f74c7000 MountMgr MountMgr.sys Sat Feb 17 01:05:35 2007 (45D69B2F)
f74c7000 f74d2000 PartMgr PartMgr.sys Sat Feb 17 01:29:25 2007 (45D6A0C5)
f74d7000 f74e7000 disk disk.sys Sat Feb 17 01:07:51 2007 (45D69BB7)
f74e7000 f74f3000 Dfs Dfs.sys Sat Feb 17 00:51:17 2007 (45D697D5)
f74f7000 f7501000 crcdisk crcdisk.sys Sat Feb 17 01:09:50 2007 (45D69C2E)
f7517000 f7522000 TDI TDI.SYS Sat Feb 17 01:01:19 2007 (45D69A2F)
f7527000 f7531000 flpydisk flpydisk.sys Tue Mar 25 03:04:32 2003 (3E7FFF80)
f7537000 f7545000 NDProxy NDProxy.SYS Sat Feb 17 00:59:21 2007 (45D699B9)
f7577000 f7580000 watchdog watchdog.sys Sat Feb 17 01:11:45 2007 (45D69CA1)
f7587000 f7592000 kbdclass kbdclass.sys Sat Feb 17 01:05:39 2007 (45D69B33)
f75c7000 f75d1000 mouclass mouclass.sys Tue Mar 25 03:03:09 2003 (3E7FFF2D)
f75d7000 f75e3000 vga vga.sys Sat Feb 17 01:10:30 2007 (45D69C56)
f75f7000 f7601600 fetnd5bv fetnd5bv.sys Tue Feb 27 03:16:14 2007 (45E3E8CE)
f7607000 f7610000 raspti raspti.sys Sat Feb 17 00:59:23 2007 (45D699BB)
f7617000 f7626000 raspppoe raspppoe.sys Sat Feb 17 00:59:23 2007 (45D699BB)
f7627000 f7632000 ptilink ptilink.sys Sat Feb 17 01:06:38 2007 (45D69B6E)
f7637000 f7646000 termdd termdd.sys Sat Feb 17 00:44:32 2007 (45D69640)
f7657000 f7661000 serenum serenum.sys Sat Feb 17 01:06:44 2007 (45D69B74)
f7667000 f7676000 intelppm intelppm.sys Sat Feb 17 00:48:30 2007 (45D6972E)
f7687000 f7690000 ndistapi ndistapi.sys Sat Feb 17 00:59:19 2007 (45D699B7)
f76a7000 f76b0000 mssmbios mssmbios.sys Sat Feb 17 00:59:12 2007 (45D699B0)
f76b7000 f76c2000 fdc fdc.sys Sat Feb 17 01:07:16 2007 (45D69B94)
f76c8000 f7707000 NDIS NDIS.sys Sat Feb 17 01:28:49 2007 (45D6A0A1)
f7707000 f770f000 kdcom kdcom.dll Tue Mar 25 03:08:00 2003 (3E800050)
f770f000 f7717000 BOOTVID BOOTVID.dll Tue Mar 25 03:07:58 2003 (3E80004E)
f7717000 f771e000 viaide viaide.sys Tue Mar 25 03:04:49 2003 (3E7FFF91)
f771f000 f7726000 dmload dmload.sys Tue Mar 25 03:08:08 2003 (3E800058)
f7727000 f772f000 videX32 videX32.sys Fri Sep 21 05:49:09 2007 (46F39395)
f772f000 f7737000 wd wd.sys Tue Mar 25 03:09:12 2003 (3E800098)
f7737000 f773dd00 viaagp1 viaagp1.sys Wed Jul 02 06:08:01 2003 (3F02AF01)
f7797000 f779db80 usbehci usbehci.sys Sat Feb 17 01:12:56 2007 (45D69CE8)
f779f000 f77a4180 usbuhci usbuhci.sys Sat Feb 17 01:13:02 2007 (45D69CEE)
f77a7000 f77af000 audstub audstub.sys Tue Mar 25 03:09:12 2003 (3E800098)
f7878000 f7897000 Mup Mup.sys Sat Feb 17 01:27:41 2007 (45D6A05D)
f79d1000 f79d2280 swenum swenum.sys Sat Feb 17 01:05:56 2007 (45D69B44)
f79e1000 f79e2580 USBD USBD.SYS Tue Mar 25 03:10:39 2003 (3E8000EF)
f7b4a000 f7bdf000 Ntfs Ntfs.sys Sat Feb 17 01:27:23 2007 (45D6A04B)

Unloaded modules:
ba2df000 ba2ed000 HIDCLASS.SYS
Timestamp: unavailable (00000000)
Checksum: 00000000
ba2ff000 ba308000 hidusb.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
ba284000 ba28c000 usbccgp.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
ba46c000 ba475000 kbdhid.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
f7567000 f7575000 imapi.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
f7567000 f7570000 kbdhid.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
f7807000 f780f000 Sfloppy.SYS
Timestamp: unavailable (00000000)
Checksum: 00000000
ba7b9000 ba7cc000 i8042prt.sys
Timestamp: unavailable (00000000)
Checksum: 00000000
Closing open log file c:debuglog.txt


Thanks in advance!
ynot2k

 

Hi ynot2k

This is not very informative. So lets do a few things before delving into the dumps.

Instead of fiddlyfarting around there are several steps to do but 1 at a time simple and is not as complex as it appears.

1st to save your a** lets get some protections in place.

1. Do a full backup with you own backup procedure.

2. D/L RSIT http://images.malwareremoval.com/random/RSIT.exe
Run it post both logs (1 log is maximized the other is minimized) here.

3. ERUNT Add a redundent Reg backup, get and install ERUNT let it add itself to startup and do a backup on install check all boxes.

ERUNT http://www.larshederer.homepage.t-online.de/erunt/
Yes! Even if you use system restore and other backups Registry and Images.

4. Windows 2003 Server has no System Restore but it can be added. This is optional but I would do it.

To add System Restore to Server 2003 follow instructions here

http://www.windowsreference.com/wind...n-server-2003/

Once installed do a Restore point!

========================================

Now that we are fairly well backed up we need to Cleanup some

D/L install and run ATF-Cleaner clear all except passwords in all browsers you have. Run repeatedly until no more found.

http://www.majorgeeks.com/ATF_Cleaner_d4949.html

D/L and install CCleaner: Clean temps and registry. Run both (temp and registry) repeatedly until no more found.
http://www.ccleaner.com/download/builds get Slim version

========================================

Now lets eliminate the possibility of malware

D/L Xclean_Micro http://www.xblock.com/download/xclean_micro.exe
No install, run it delete all it finds decline to reboot on each item found, until the program finishes then reboot.

Xclean will run minimized and will pop up a window if it finds anything. If it finds nothing it will exit.

Please make a note of what it found if any as this might indicate a deeper Malware diag.

Get the below install and update then post log

http://www.malwarebytes.org/mbam.php

If it found much the reboot to Safe Mode and run it again then paste s new log for this run when back to normal mode!

Your ball!

Mike

 

HOLY MACARONI - wasn't expecting such a thorough reply.

will do as suggested and post back - most likely tomorrow.

in the interim i deduced from that bugcheck that it was crashing on mysqld - so i downgraded mysql to the more stable 5.0.67 rather than the rc 5.1.22. so far so good. but i will still follow your steps.

THANKS MIKE!

best, ynot2k

 

Yes good idea! I was going to suggest reinstalling after I saw the RSIT log. But dropping back a level ma be even better.

Have you updated it recently like right before this issue?

Mike

 

I am so sorry but due issues with the BBS Admin I will not be back to help you!

Good luck,
Mike

 

But others of use will be here to pick this up.