What is the right tool to detect VMT or heap corruption in Delphi ?
- by Roland Bengtsson
I'm a member in a team that use Delphi 2007 for a larger application and we suspect heap corruption because sometimes there are strange bugs that have no other explanation.
I believe that the Rangechecking option for the compiler is only for arrays. I want a tool that give an exception or log when there is a write on a memory address that is not allocated by the application.
Regards
EDIT: The error is of type:
Error: Access violation at address 00404E78 in module 'BoatLogisticsAMCAttracsServer.exe'. Read of address FFFFFFDD
EDIT2: Thanks for all suggestions. Unfortunately I think that the solution is deeper than that. We use a patched version of Bold for Delphi as we own the source. Probably there are some errors introduced in the Bold framwork. Yes we have a log with callstacks that are handled by JCL and also trace messages. So a callstack with the exception can lock like this:
20091210 16:02:29 (2356) [EXCEPTION] Raised EBold: Failed to derive ServerSession.mayDropSession: Boolean
OCL expression: not active and not idle and timeout and (ApplicationKernel.allinstances->first.CurrentSession <> self)
Error: Access violation at address 00404E78 in module 'BoatLogisticsAMCAttracsServer.exe'. Read of address FFFFFFDD. At Location BoldSystem.TBoldMember.CalculateDerivedMemberWithExpression (BoldSystem.pas:4016)
Inner Exception Raised EBold: Failed to derive ServerSession.mayDropSession: Boolean
OCL expression: not active and not idle and timeout and (ApplicationKernel.allinstances->first.CurrentSession <> self)
Error: Access violation at address 00404E78 in module 'BoatLogisticsAMCAttracsServer.exe'. Read of address FFFFFFDD. At Location BoldSystem.TBoldMember.CalculateDerivedMemberWithExpression (BoldSystem.pas:4016)
Inner Exception Call Stack:
[00] System.TObject.InheritsFrom (sys\system.pas:9237)
Call Stack:
[00] BoldSystem.TBoldMember.CalculateDerivedMemberWithExpression (BoldSystem.pas:4016)
[01] BoldSystem.TBoldMember.DeriveMember (BoldSystem.pas:3846)
[02] BoldSystem.TBoldMemberDeriver.DoDeriveAndSubscribe (BoldSystem.pas:7491)
[03] BoldDeriver.TBoldAbstractDeriver.DeriveAndSubscribe (BoldDeriver.pas:180)
[04] BoldDeriver.TBoldAbstractDeriver.SetDeriverState (BoldDeriver.pas:262)
[05] BoldDeriver.TBoldAbstractDeriver.Derive (BoldDeriver.pas:117)
[06] BoldDeriver.TBoldAbstractDeriver.EnsureCurrent (BoldDeriver.pas:196)
[07] BoldSystem.TBoldMember.EnsureContentsCurrent (BoldSystem.pas:4245)
[08] BoldSystem.TBoldAttribute.EnsureNotNull (BoldSystem.pas:4813)
[09] BoldAttributes.TBABoolean.GetAsBoolean (BoldAttributes.pas:3069)
[10] BusinessClasses.TLogonSession._GetMayDropSession (code\BusinessClasses.pas:31854)
[11] DMAttracsTimers.TAttracsTimerDataModule.RemoveDanglingLogonSessions (code\DMAttracsTimers.pas:237)
[12] DMAttracsTimers.TAttracsTimerDataModule.UpdateServerTimeOnTimerTrig (code\DMAttracsTimers.pas:482)
[13] DMAttracsTimers.TAttracsTimerDataModule.TimerKernelWork (code\DMAttracsTimers.pas:551)
[14] DMAttracsTimers.TAttracsTimerDataModule.AttracsTimerTimer (code\DMAttracsTimers.pas:600)
[15] ExtCtrls.TTimer.Timer (ExtCtrls.pas:2281)
[16] Classes.StdWndProc (common\Classes.pas:11583)
The inner exception part is the callstack at the moment an exception is reraised.
EDIT3: The theory right now is that the Virtual Memory Table (VMT) is somehow broken. When this happen there is no indication of it. Only when a method is called an exception is raised (ALWAYS on address FFFFFFDD, -35 decimal) but then it is too late. You don't know the real cause for the error. Any hint of how to catch a bug like this is really appreciated!!! We have tried with SafeMM, but the problem is that the memory consumption is too high even when the 3 GB flag is used. So now I try to give a bounty to the SO community :)
EDIT4: One hint is that according the log there is often (or even always) another exception before this. It can be for example optimistic locking in the database. We have tried to raise exceptions by force but in test environment it just works fine.
EDIT5: Story continues... I did a search on the logs for the last 30 days now. The result:
"Read of address FFFFFFDB" 0
"Read of address FFFFFFDC" 24
"Read of address FFFFFFDD" 270
"Read of address FFFFFFDE" 22
"Read of address FFFFFFDF" 7
"Read of address FFFFFFE0" 20
"Read of address FFFFFFE1" 0
So the current theory is that an enum (there is a lots in Bold) overwrite a pointer. I got 5 hits with different address above. It could mean that the enum holds 5 values where the second one is most used. If there is an exception a rollback should occur for the database and Boldobjects should be destroyed. Maybe there is a chance that not everything is destroyed and a enum still can write to an address location. If this is true maybe it is possible to search the code by a regexpr for an enum with 5 values ?
EDIT6: To summarize, no there is no solution to the problem yet. I realize that I may mislead you a bit with the callstack. Yes there are a timer in that but there are other callstacks without a timer. Sorry for that. But there are 2 common factors.
An exception with Read of address FFFFFFxx.
Top of callstack is System.TObject.InheritsFrom (sys\system.pas:9237)
This convince me that VilleK best describe the problem.
I'm also convinced that the problem is somewhere in the Bold framework.
But the BIG question is, how can problems like this be solved ?
It is not enough to have an Assert like VilleK suggest as the damage has already happened and the callstack is gone at that moment. So to describe my view of what may cause the error:
Somewhere a pointer is assigned a bad value 1, but it can be also 0, 2, 3 etc.
An object is assigned to that pointer.
There is method call in the objects baseclass. This cause method TObject.InheritsForm to be called and an exception appear on address FFFFFFDD.
Those 3 events can be together in the code but they may also be used much later. I think this is true for the last method call.
EDIT7: We work closely with the the author of Bold Jan Norden and he recently found a bug in the OCL-evaluator in Bold framework. When this was fixed these kinds of exceptions decreased a lot but they still occasionally come. But it is a big relief that this is almost solved.
