Debugging code is a lot like the scientific method. Most importantly, both involve careful measurements and analysis of the resulting data. Be wary of the developer who is quick to announce the cause of an error. When an error occurs, a developer's first thought should be, "What data to I need to collect to better understand and analyze this error?" A less experienced developer is one whose first thought is, "Maybe the cause of the problem is (the database query|session timeout issues|invalid user input|etc.)?" This latter approach leads the developer down a series of dark alleys, many of which are dead ends.
Another important component of debugging and science are control groups and experimental groups. For the results of a scientific experiment to be sound, the researcher must have a control group that is not experimented upon, and an experimental group that differs from the control group by only one variable, namely by the "thing" that is being tested. In debugging, constructing the control group involves reproducing the error. If the QA department reports that an error occurs when entering a value longer than 50 characters into the Billing Address textbox on CheckOut.aspx, the first step in debugging is to visit CheckOut.aspx and enter an input longer than 50 characters into the Billing Address textbox to ensure that the reported error occurs. Likewise, the same steps should be repeated after the bug is fixed to help ensure that the fix has indeed solved the problem at hand.
Additionally, when debugging the practitioner must be careful not to taint the observations with his own preconceived notions or bias. The history of science is littered with examples of scientists who knowingly or unwittingly let their personal beliefs or the current societal mores color the data and influence their analysis. Johannes Kepler, the famous German mathematician and astrologer who penned the three laws of planetary motion, is a great example. Kepler started his astronomical career by using astronomical measurements to describe the motion of planets in our solar system. Although Kepler subscribed to Corpernicus's heliocentric model - that the sun was the center of the solar system rather than the Earth - he held to a rather convoluted theory that the planets orbited the sun based on different 3-dimensional polyhedra according to the Platonic solids. Kepler's problem was, in part, that he was approaching the problem backwards. Rather than taking the data at his disposal and asking, "What arrangement of the sun and planets would most sensibly explain these astronomical observations," he asked, "Can I construct a model that fits with the observations yet still maintains a semblance to the work done by the ancient Greeks and upholds my personal theological beliefs?" Fortunately, Kepler eventually abandoned his initial forays and came upon the a much simpler and, more importantly, a correct theory that proposed elliptical orbital paths with the sun as a focus.
A developer can fall victim to biasing observational data just as easily as Kepler did. A developer might be accustomed to bugs originating from a particular module, or from the code of a particular coworker. While such information is useful when deciding what data to collect, it is important to not let these preconceived notions influence the analysis of the data. While it may be mathematically probable that a particular error originated from a certain buggy module, don't ignore data that would suggest otherwise.
A corollary to Kepler's story is Occam's razor: that the simplest hypothesis is usually the correct one. If you find that you need to concoct an elaborate and complex scenario to explain the cause of the error, chances are you're barking up the wrong tree. More briefly, don't think too hard.
All this being said, debugging and science involves more than the mechanical process of gathering data and methodically interpreting it. There is an art form to the practice, and a certain wiliness and level of insightfulness that comes with much practice. But don't forget that the overwhelming bulk of any successful debugging session or any successful scientific experiment is keen measurement and straightforward, analytical interpretation of the data.