WinDbg: 执行 SOS 扩展命令 !clrstack时报错 Access violation exception (0xC0000005)
今天在调试分析一个dump文件时,当我执行了".loadby sos clrjit"指令后,准备分析托管代码的问题,于是我又输入了"!clrstack"扩展指令想看下托管栈,结果Windbg给我如下输出:
0:000> .loadby sos clrjit
0:000> !clrstack
c0000005 Exception in C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\sos.clrstack debugger extension.
PC: 0c53d0f3 VA: 00000000 R/W: 0 Parameter: 00000000
这还是第一次遇到,当我在执行"!pe"指令时,又有正确输出了
0:000> !pe
Exception object: 1eda1120
Exception type: System.ExecutionEngineException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131506
然后在执行"!clrstack",结果也是能正确输出:
0:000> !clrstack
OS Thread Id: 0x6f90 (0)
Child SP IP Call Site
00efc708 50e51a49 [PrestubMethodFrame: 00efc708] XXXXXX..cctor()
00efc8ec 50e51a49 [GCFrame: 00efc8ec]
00efce54 50e51a49 [PrestubMethodFrame: 00efce54] XXXXXX..ctor()
00efcec4 1d8908f4 XXXXXX(Int64)
00efd208 50e1f066 [DebuggerU2MCatchHandlerFrame: 00efd208]
00efcfc8 50e1f066 [HelperMethodFrame_PROTECTOBJ: 00efcfc8] System.RuntimeMethodHandle.InvokeMethod(System.Object, System.Object[], System.Signature, Boolean)
00efd2a4 78dfc799 System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(System.Object, System.Object[], System.Object[])
00efd2c8 78dfc2aa System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
00efd2fc 1d7b2cf0 *** WARNING: Unable to verify checksum for Adapter.dll
.?A0xc8a1231f.CallFunctionByReflectionInternal(DiyHomePlugin.Plugin, System.String, System.Object ByRef, System.Object[])
00efd364 1d8907eb .XXXX.AppStart(CAdapter*, Int64)
00efd380 1d890731 DomainBoundILStubClass.IL_STUB_ReversePInvoke(Int32, Int64)
后来又多试了几次,发现当加载了sos.dll扩展后,第一个扩展指令(任何指令)都会报错,然后第二条指令不会。看来要研究研究一下,开启Windbg调试自己:
(9ec.4b0): Access violation - code c0000005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. sos!GetCMDOption+0x63: 00007ff9`a35ac7e3 488b01 mov rax,qword ptr [rcx] ds:00000000`00000000=???????????????? 0:004> ub sos!GetCMDOption+0x3f: 00007ff9`a35ac7bf 4889542420 mov qword ptr [rsp+20h],rdx 00007ff9`a35ac7c4 488bf1 mov rsi,rcx 00007ff9`a35ac7c7 4d85c0 test r8,r8 00007ff9`a35ac7ca 743f je sos!GetCMDOption+0x8b (00007ff9`a35ac80b) 00007ff9`a35ac7cc 488d5a18 lea rbx,[rdx+18h] 00007ff9`a35ac7d0 392dd2e20500 cmp dword ptr [sos!ControlC (00007ff9`a360aaa8)],ebp 00007ff9`a35ac7d6 0f85fe000000 jne sos!GetCMDOption+0x15a (00007ff9`a35ac8da) 00007ff9`a35ac7dc 488b0dfd6d0500 mov rcx,qword ptr [sos!g_ExtControl (00007ff9`a36035e0)] 0:004> k # Child-SP RetAddr Call Site 00 0000000a`05aad2c0 00007ff9`a35973ac sos!GetCMDOption+0x63 01 0000000a`05aad3b0 00007ff9`a86f353b sos!ClrStack+0x21c 02 0000000a`05aad570 00007ff9`a86f3718 dbgeng!ExtensionInfo::CallA+0x233 03 0000000a`05aad630 00007ff9`a86f37f8 dbgeng!ExtensionInfo::Call+0x16c 04 0000000a`05aad830 00007ff9`a86f2689 dbgeng!ExtensionInfo::CallAny+0x78 05 0000000a`05aad870 00007ff9`a872a89b dbgeng!ParseBangCmd+0x4a9 06 0000000a`05aadd30 00007ff9`a872b6ab dbgeng!ProcessCommands+0xa8f 07 0000000a`05aade00 00007ff9`a8685fe8 dbgeng!ProcessCommandsAndCatch+0x8f 08 0000000a`05aade70 00007ff9`a868628f dbgeng!Execute+0x24c 09 0000000a`05aae340 00007ff7`0d0c5c72 dbgeng!DebugClient::ExecuteWide+0x83 0a 0000000a`05aae3a0 00007ff7`0d0c60d5 windbg!ProcessCommand+0x2b2 0b 0000000a`05aae7c0 00007ff7`0d0c7c17 windbg!ProcessEngineCommands+0x185 0c 0000000a`05aaf800 00007ff9`d6a316ad windbg!EngineLoop+0x3e3 0d 0000000a`05aaf840 00007ff9`d7244629 KERNEL32!BaseThreadInitThunk+0xd 0e 0000000a`05aaf870 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
让我们为sos!g_ExtControl变量上的写访问设置一个断点,以查找其归零位置。
0:001> ba w8 sos!g_ExtControl 0:001> g 0:004> k # Child-SP RetAddr Call Site 00 0000008d`40d2d3f0 00007ff9`a7b1dc6e dbgeng!DebugClient::QueryInterface+0xb 01 0000008d`40d2d420 00007ff9`a7b471d0 sos!ExtQuery+0x2e 02 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x40 ... 0:004> k # Child-SP RetAddr Call Site 00 0000008d`40d2c080 00007ff9`a7b1dc6e dbgeng!DebugClient::QueryInterface+0xb 01 0000008d`40d2c0b0 00007ff9`a7b1d973 sos!ExtQuery+0x2e 02 0000008d`40d2c0e0 00007ff9`a86f2d25 sos!DebugExtensionInitialize+0x83 03 0000008d`40d2c120 00007ff9`a86f4182 dbgeng!ExtensionInfo::Load+0x48d 04 0000008d`40d2c3f0 00007ff9`a87312f6 dbgeng!ExtensionInfo::CheckAdd+0x6e 05 0000008d`40d2c430 00007ff9`a8731ca8 dbgeng!LoadSOSAndCheckVer+0x36 06 0000008d`40d2c690 00007ff9`a86f00e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828 07 0000008d`40d2cec0 00007ff9`a7b61004 dbgeng!ExtIoctl+0xc6d 08 0000008d`40d2d410 00007ff9`a7b4722c sos!LoadClrDebugDll+0x24 09 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x9c ... 0:004> k # Child-SP RetAddr Call Site 00 0000008d`40d2c0b0 00007ff9`a7b1d98d sos!ExtRelease+0x28 01 0000008d`40d2c0e0 00007ff9`a86f2d25 sos!DebugExtensionInitialize+0x9d 02 0000008d`40d2c120 00007ff9`a86f4182 dbgeng!ExtensionInfo::Load+0x48d 03 0000008d`40d2c3f0 00007ff9`a87312f6 dbgeng!ExtensionInfo::CheckAdd+0x6e 04 0000008d`40d2c430 00007ff9`a8731ca8 dbgeng!LoadSOSAndCheckVer+0x36 05 0000008d`40d2c690 00007ff9`a86f00e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828 06 0000008d`40d2cec0 00007ff9`a7b61004 dbgeng!ExtIoctl+0xc6d 07 0000008d`40d2d410 00007ff9`a7b4722c sos!LoadClrDebugDll+0x24 08 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x9c ... 0:004> g (fe4.f70): Access violation - code c0000005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. sos!GetCMDOption+0x63: 00007ff9`a7b5c7e3 488b01 mov rax,qword ptr [rcx] ds:00000000`00000000=???????????????? 0:004> k # Child-SP RetAddr Call Site 00 0000008d`40d2d360 00007ff9`a7b473ac sos!GetCMDOption+0x63 01 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x21c ... sos!ExtQuery: 00007ff9`6a16dc40 48895c2408 mov qword ptr [rsp+8],rbx 00007ff9`6a16dc45 4889742410 mov qword ptr [rsp+10h],rsi 00007ff9`6a16dc4a 57 push rdi 00007ff9`6a16dc4b 4883ec20 sub rsp,20h 00007ff9`6a16dc4f 33f6 xor esi,esi 00007ff9`6a16dc51 4c8d0588590900 lea r8,[sos!g_ExtControl (00007ff9`6a2035e0)] 00007ff9`6a16dc58 488d1571760600 lea rdx,[sos!GUID_d4366723_44df_4bed_8c7e_4c05424f4588 (00007ff9`6a1d52d0)] 00007ff9`6a16dc5f 4889357a590900 mov qword ptr [sos!g_ExtControl (00007ff9`6a2035e0)],rsi 00007ff9`6a16dc66 488b01 mov rax,qword ptr [rcx] 00007ff9`6a16dc69 488bf9 mov rdi,rcx 00007ff9`6a16dc6c ff10 call qword ptr [rax] ds:00007ff9`60e018d0={dbgeng!DebugClient::QueryInterface (00007ff9`60fd0240)} sos!ExtRelease: 00007ff9`6a16ddc0 4053 push rbx 00007ff9`6a16ddc2 4883ec20 sub rsp,20h 00007ff9`6a16ddc6 488b0d13580900 mov rcx,qword ptr [sos!g_ExtControl (00007ff9`6a2035e0)] 00007ff9`6a16ddcd 33db xor ebx,ebx 00007ff9`6a16ddcf 48891d02580900 mov qword ptr [sos!g_ExtClient (00007ff9`6a2035d8)],rbx 00007ff9`6a16ddd6 4885c9 test rcx,rcx 00007ff9`6a16ddd9 740d je sos!ExtRelease+0x28 (00007ff9`6a16dde8) 00007ff9`6a16dddb 488b01 mov rax,qword ptr [rcx] 00007ff9`6a16ddde ff5010 call qword ptr [rax+10h] ds:00007ff9`60e01358={dbgeng!DebugClient::Release (00007ff9`60fd0200)}
我们可以看到,IDebugControl2接口获得了两次调用,然后仍在使用被释放的sos!ClrStack函数。另外,我们可以看到sos.dll是第二次加载的。为了找出原因,让我们调试dbgeng!LoadSOSAndCheckVerr函数。在dbgeng!ExtensionInfo::Add方法我们可以看到它正在调用dbgeng!ExtensionInfo::FindByName方法中按名称查找扩展名dbgeng!ExtensionInfo::s_Chain扩展信息链表
0:003> k # Child-SP RetAddr Call Site 00 0000003a`a963c970 00007ff9`61044168 dbgeng!ExtensionInfo::Add 01 0000003a`a963c9d0 00007ff9`610812f6 dbgeng!ExtensionInfo::CheckAdd+0x54 02 0000003a`a963ca10 00007ff9`61081ca8 dbgeng!LoadSOSAndCheckVer+0x36 03 0000003a`a963cc70 00007ff9`610400e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828 04 0000003a`a963d4a0 00007ff9`6a601004 dbgeng!ExtIoctl+0xc6d 05 0000003a`a963d9f0 00007ff9`6a5e722c sos!LoadClrDebugDll+0x24 06 0000003a`a963da30 00007ff9`6104353b sos!ClrStack+0x9c ... 0:003> r rax=0000000000000036 rbx=0000003aa9505490 rcx=0000003aa963ca40 rdx=0000000000000000 rsi=0000000000000037 rdi=0000003aa963ca40 rip=00007ff961043dc5 rsp=0000003aa963c970 rbp=0000000000000000 r8=0000000000000000 r9=0000003aa963c9f0 r10=0000000000000000 r11=0000003aa963c9e0 r12=0000000000000000 r13=0000000000000000 r14=0000003aa963c9f0 r15=0000003aa963cd01 iopl=0 nv up ei pl nz na pe nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 dbgeng!ExtensionInfo::Add+0x95: 00007ff9`61043dc5 e876feffff call dbgeng!ExtensionInfo::FindByName (00007ff9`61043c40) 0:003> du @rcx 0000003a`a963ca40 "C:\Windows\Microsoft.NET\Framewo" 0000003a`a963ca80 "rk64\v4.0.30319\SOS.dll" 0:003> u dbgeng!ExtensionInfo::FindByName dbgeng!ExtensionInfo::FindByName: 00007ff9`61043c40 488bc4 mov rax,rsp 00007ff9`61043c43 48895808 mov qword ptr [rax+8],rbx 00007ff9`61043c47 48896810 mov qword ptr [rax+10h],rbp 00007ff9`61043c4b 48897018 mov qword ptr [rax+18h],rsi 00007ff9`61043c4f 48897820 mov qword ptr [rax+20h],rdi 00007ff9`61043c53 4156 push r14 00007ff9`61043c55 4883ec20 sub rsp,20h 00007ff9`61043c59 4883cfff or rdi,0FFFFFFFFFFFFFFFFh 00007ff9`61043c5d 488bf2 mov rsi,rdx 00007ff9`61043c60 488be9 mov rbp,rcx 00007ff9`61043c63 4533f6 xor r14d,r14d 00007ff9`61043c66 48ffc7 inc rdi 00007ff9`61043c69 6644393479 cmp word ptr [rcx+rdi*2],r14w 00007ff9`61043c6e 75f6 jne dbgeng!ExtensionInfo::FindByName+0x26 (00007ff9`61043c66) 00007ff9`61043c70 488b1d91041c00 mov rbx,qword ptr [C:\Windows\Microsoft.NET\Framewo" 0000003a`a94fb360 "rk64\v4.0.30319\sos" 0000003a`a94f8830 "dbghelp" 0000003a`a94f8560 "ext" 0000003a`a94f8290 "exts" 0000003a`a94f7fc0 "uext" 0000003a`a94f7cf0 "ntsdexts"
如果没有,它会叫dbgeng!ExtensionInfo::Link方法将新的扩展dll添加到链接列表中。
0:003> r rax=0000000000000000 rbx=0000003aa5f1e6b0 rcx=0000003aa5f1e6b0 rdx=0000003aa5f1e6d0 rsi=0000000000000037 rdi=0000003aa963ca40 rip=00007ff961043ed0 rsp=0000003aa963c970 rbp=0000000000000000 r8=0000000000000000 r9=0000000000000000 r10=0000003aa95abd30 r11=0000003aa5f1e6d0 r12=0000000000000000 r13=0000000000000000 r14=0000003aa963c9f0 r15=0000003aa963cd01 iopl=0 nv up ei pl nz na pe nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 dbgeng!ExtensionInfo::Add+0x1a0: 00007ff9`61043ed0 e8e7110000 call dbgeng!ExtensionInfo::Link (00007ff9`610450bc) 0:003> du @rdx 0000003a`a5f1e6d0 "C:\Windows\Microsoft.NET\Framewo" 0000003a`a5f1e710 "rk64\v4.0.30319\SOS.dll" 0:003> !list -x "du poi(@$extret+8)" poi(dbgeng!ExtensionInfo::s_Chain) 0000003a`a5f1e6d0 "C:\Windows\Microsoft.NET\Framewo" 0000003a`a5f1e710 "rk64\v4.0.30319\SOS.dll" 0000003a`a94fb320 "C:\Windows\Microsoft.NET\Framewo" 0000003a`a94fb360 "rk64\v4.0.30319\sos" 0000003a`a94f8830 "dbghelp" 0000003a`a94f8560 "ext" 0000003a`a94f8290 "exts" 0000003a`a94f7fc0 "uext" 0000003a`a94f7cf0 "ntsdexts"
最后是dbgeng!ExtensionInfo::CheckAdd方法调用dbgeng!ExtensionInfo::Load加载sos.dll:
0:003> u dbgeng!ExtensionInfo::CheckAdd+0x69: 00007ff9`6104417d e816e7ffff call dbgeng!ExtensionInfo::Load (00007ff9`61042898)
这是DbgEng.dll中的一个错误。为coreclr添加了一个解决方案。不确定它是否已经进入了完整的框架。可以通过下面的命令之一来解决:
.loadby sos.dll clr
.loadby sos.dll clrjit
.cordll -ve -u -l