It's amazing how much time speculative development can swallow up during the course of a project. Speculative development happens when a developer implements features above and beyond the requirements based on the assumption that he'll need such functionality at a later point in time. I think it's a trap every developer has fallen into at one point or another. We are tasked with something simple - like exposing the latest news through an RSS feed - and turn it into something complex - like building an entire RSS generation framework that can syndicate any data source.
In The Productive Programmer Neal Ford explains why software projects - as opposed to other engineering projects - are so prone to speculative development:
Usually, in the physical world, things tend toward simplicity unless you add energy to disrupt it. Software is the opposite: because it is so easy to create, it tends toward complexity (in other words, it takes the same physical effort to create both complex and simple software). It can take great effort to pull software back toward simplicity.
In other words, it's expensive in time and cost to increase the complexity of a bridge or car or house, which is why you don't see needless complexity added to such systems. But for software, it may only take a few more hours to add some extra functionality that, while not required now, may be useful down the road. Another factor that leads to a high occurrence of speculative work in software projects is the typical mindset of a programmer. We like order and rigidity and building black boxes that can be plugged in elsewhere and used again. We enjoy challenges and solving problems. Spitting out the latest news in a simple RSS feed is easy, boring even. It's much more mentally stimulating to craft an RSS generation framework. (This predilection of self-directed framework building also helps explain why developers working on existing projects oftentimes advocate pitching the existing code base and rewriting the application from scratch rather than build on top of the work that's already been done.)
The main problem with speculative development is that it adds potentially unneeded code to the project, bloating its size, increasing the surface area of the project, and contributing to the code that must be tested. It's also potentially wasted time. Of course, speculative development isn't a problem if the speculative work is actually needed and used at some future point in time. But because there's no way to definitively say that the speculative work will eventually be needed (or that it will be needed but in some alternate form), it's best to build only what is needed in the here and now.
There's an acronym to keep in mind to help temper speculative development - YAGNI, which stands for "You Ain't Gonna Need It." Ron Jeffries, one of the founders of Extreme Programming methodology, summarizes the idea behind YAGNI succinctly:
Often you will be building some class and you’ll hear yourself saying "We’re going to need...". Resist that impulse, every time. Always implement things when you actually need them, never when you just foresee that you need them. ... You’re thinking about what the class might be, rather than what it must be. You were on a mission when you started building that class. Keep on that mission rather than let yourself be distracted for even a moment. Your time is precious. You might not need it after all. If that happens, the time you spend implementing the method will be wasted; the time everyone else spends reading it will be wasted; the space it takes up will be wasted. ... The best way to implement code quickly is to implement less of it. The best way to have fewer bugs is to implement less code.
Wise words from a wise man.