Remote Debugging
- by Pemdas
Obviously, the easiest way to solve a bug is to be able to reproduce it in house. However, sometimes that is not practical. For starters, users are often not very good at providing you with useful information.
Costumer Service: "what seems to be the issue?"
User: "It crashed!"
To further compound that, sometimes the bug only occurs under certain environmentally conditions that can not be adequately replicated in house. With that in mind, it is important to build some sort of diagnostic framework into your product. What type of solutions have you seen or used in your experience?
Logging seems to be the predominate method, which makes sense. We have a fairly sophisticated logging frame work in place with different levels of verbosity and the ability to filter on specific modules (actually we can filter down to the granularity of a single file). Error logs are placed strategically to manufacture a pretty good representation of a stack trace when an error occurs. We don't have the luxury of 10 million terabytes of disk space since I work on embedded platforms, so we have two ways of getting them off the system: a serial port and a syslog server.
However, an issue we run into sometimes is actually getting the user to turn the logs on. Our current framework often requires some user interaction.
