As usual, I stumble upon a nasty issue while I was writing another post : create a native breakpoint on a function which I don’t have the source code. In Visual Studio 2010. In my case it was on User32.dll!SendMessageW, but you can experience the same difficulty for any other function if you are using symbol files without the path to the source code (as, by instance, public symbols of Microsoft…).
Of course, with WinDbg, nothing (well, almost) could be simpler: just get the name of your function with format <module>!<function name> (if you’re not sure about the name issue a “x <module>!*<part of function name>*” and search into the results), and then type “bu <module>!<function name>“, and you’re done. You can see here that WinDbg was designed to work without source code. It is definitely not the case of Visual Studio. So, how can you achieve this (simple) goal: break on a function without the source code?
Microsoft has done a nice job when it designed Component Object Model, also known as COM, even if the system’s complexity was painful for some developers. To fix that, they first created Vb6 (well…no comment) to soften the burden of consuming COM objects, while C++ was still a little bit rough to use (but anything concerning COM was possible !). After a few years, they finally created the .NET framework. As a fair part of the Windows architecture was based on COM and COM+, they had to make the most popular language (C# and VB.NET) of the platform compatible with it. I’m currently working on a system where the .NET code use a lot of COM to communicate with other parts of the system, and I can tell you that even most of the features of COM can be used, sometimes you have to dig deeper to make .NET component compatible with other languages. I saw a few custom marshaler described on the net, but no one done to marshal an array of “anything” (called VARIANT in COM), so I thought it could be interesting to share it with others.
To cope with the various needs of any kind of software, every operating system offer a generalist memory management system. As it works out-of-the-box most of the time, developers usually don’t dig into it to find how it behaves under the hood. But you may be surprised to know that you can not access all the memory of your process, even when some parts seem to be free. Beyond available memory and fragmentation of user space, there is another barrier that can prevent you to get the remaining free bits : the heap service and the minimum size of a virtual allocation call.
Imagine a software who, after a few hours of uptime, starts to throw out-of-memory exceptions. First thought is that, indeed, you don’t have enough memory left. Let’s say that 1Go is free, and that you just ask for a small block of 30k. If you’re familiar with memory issues, second thought may be: “memory fragmentation! Check the largest contiguous block!”. And if I say that a block of 60k is available ? Well, if you are like me before this issue happens at my work, you’re stuck. But let’s go back to the beginning.
Maybe you already had this kind of details while using the ‘!clrstack‘ command from the SOS debugging extension. And by the rules of the Murphy’s Law, it always happens specifically on the local variable that you really need to see in order to slash the bug you’re working on.
This <no data> stuff can be seen especially in release compilation because the Jitter will use all the optimization tricks it can, among which you can find inlining and massive use of registry for temporary variables, so reading local variables is almost impossible (remember C++ hum ?). Hopefully the CLR offer a simple way to disable any kind of optimization (beware, doing this have a major impact on performances!).