Scott on Writing

Feedback

# re: Where's My Humility? 1/21/2004 1:21 PM Maxim V. Karpov

How to convert Your Interview Into MP3 file.
http://ipattern.com/simpleblog/PermLink.aspx?entryid=26

I enjoy it

# re: Where's My Humility? 1/22/2004 5:28 AM Maxim V. Karpov

Scott,
I enjoy the interview. I started programming in 6th grade as well. My dad got me started, so we have something in common. Anyway, I just wanted to tell ya thanks for sharing.

Maxim

# re: Where's My Humility? 1/22/2004 5:25 PM Mike Singer

From the interview:
> So one important lesson I’ve learned
> is simply “do.”
> Another important lesson I’ve learned
> is that you don’t always know the
> answer to a problem, but don’t let
> that stop you. Answers aren’t
> intrinsically known, they’re
> discovered. These are two philosophies...

While this approach is okay for beginning programmers, this, I suspect, will lead to a catastrophe in an enterprise-wide application.

Compare “An Extensive Examination of the DataGrid Web Control Part 7” quotation with the quotation from Dino Esposito’s “ADO.NET Best Practices”:

Scott Mitchell:
Be sure to take note that both the connection and the DataSet ddlDataSet are defined globally to the page, meaning that any function in this ASP.NET Web page can
access these objects. The reason the connection object is defined globally is because we have two functions now -
GetCategories() and BindData() - that grab database information. Rather than creating, opening, and closing separate connection objects for each of the two functions,
we work with just the one, globally-defined one.
______________________

Dino Esposito:
Two golden rules characterize any code working with connections. First: open the connection as late as possible. Second: close the connection as early as possible. You want your application to work with connections for the shortest time possible.
______________________


Regards,
Mike, Ukraine.

# re: Where's My Humility? 1/22/2004 7:53 PM Scott Mitchell

Steve, you are right - learning by doing is not the best approach for someone writing mission-critical code! Someone in that position should not need to "learn" how to do what it is they need to do - they should know it already, agreed?

But the central point still holds. You could never become one who's writing enterprise-level code if you never sat down and started from somewhere. (Realize that the majority of my books are aimed at the beginning to intermediate developer, so I'm usually in the mindset of teaching someone something that's relatively new to them, and not focusing on "best practices.")

Thanks for the comments...

# re: Where's My Humility? 1/23/2004 8:52 AM Maxim V. Karpov

Exactly, there two types of sample code:

1. demo the technology - goal is to make clear point of how technology (syntax) works

2. best use of it - complex samples and points not just to the syntax.

I would rephrase Scott:
"The best way to learn is through life learning process!"

Maxim

# re: Where's My Humility? 1/23/2004 9:25 PM Mike Singer

I didn't mean mission-critical code.
I was talking about just enterprise-level code.
Sorry for my vagueness.

To illustrate the idea I’d cite the great article
“The Visual Modeling of Software Architecture for the Enterprise”
by Grady Booch
If you want to build a high rise office building, it would be infinitely stupid for you to start with a pile of lumber, some nails, and a few basic tools. Since you are probably using other people's money, they will demand to have input into the size, shape, and style of the building. Often, they will change their minds, even after you've started building. You will want do to an extensive amount of planning, because the cost of failure is high. You will be just a part of a much larger group responsible for developing and deploying the building, and so the team will need all sorts of blueprints and models to communicate with one another. As long as you get the right people and the right tools and actively manage the process of transforming an architectural concept into reality, you will likely end up with a building that will satisfy its tenants. If you want to keep building buildings, then you will want to be certain to balance the desires of your tenants with the realities of building technology, and you will want to treat the rest of your team professionally, never placing them in any risk or driving them so hard that they burn out.

It's a curious thing, but a lot of software development organizations start out wanting to build high rises, but approach the problem as if they were knocking out a dog house.

Sometimes, you get lucky. If you have the right people at just the right moment in time and if the planets align properly, then you might, just might, get your team to push out a software product that dazzles its users. Typically, however, you can't get all the right people (the right ones are often already over-committed), it's never the right moment (yesterday would have been better) and the planets never seem to align (instead, they keep moving, entirely out of your control). Given the increasing demand to develop software in Internet time, development teams often fall back on the only thing they really know how to do well, namely, pound out lines of code. Heroic programming efforts are legend in this industry, and thus often its seems that working harder is the proper reaction to any crisis in development. Oft times, however, these are not necessarily the right lines of code, and furthermore, some projects are of such a magnitude that even adding more hours to the work day is just not enough to get the job done. If you really want to build the software equivalent of a house or of a high rise, the problem is always more than just a matter of writing lots of software-in fact, the trick is in creating the right software and in figuring out how to write less software. This makes quality software development an issue of architecture and process and tools. Even so, many projects start out looking like dog houses, but then grow to the magnitude of a high rise, simply because they are a victim of their own success. There comes a time when, if there was no consideration given to architecture, process, or tools, that the dog house now grown into a high rise collapses of its own sheer weight. The collapse of a dog house may annoy your dog; the failure of a high rise will materially impact its tenants.
+++++++++++++++++++++++++++++++++++


And I’m afraid I can’t agree, that software development must be sorta pleasant adventure “just do it” without boring process of learning theory before it, along it and after it.
Thanks for the discussion,
Mike.

# re: Where's My Humility? 1/23/2004 10:56 PM Scott Mitchell

>And I'm afraid I can't agree,
>that software development must
>be sorta pleasant adventure "just
>do it" without boring process of
>learning theory before it, along
>it and after it.

Mike, I think you're taking my words a bit out of context. In the interview I said: "I’ve always believed that the best way to learn is by doing, and that’s the philosophy I’ve had when learning about computers and programming myself." I stand by that. I've learned the theory - I have both an undergraduate and Masters degree in computer science - but knowledge alone is not enough. I have had classes with and did projects with other grad students who were lifelong students, who had no work experience, and who had trouble even writing the simplest program. Sure, they may know all the theory, but you only learn how to apply that knowledge by doing.

Another quote of mine from the interview: "So one important lesson I’ve learned is simply “do.” Another important lesson I’ve learned is that you don’t always know the answer to a problem, but don’t let that stop you. Answers aren’t intrinsically known, they’re discovered. These are two philosophies I’ve tried to carry from my personal experiences into the books and articles I’ve authored, especially those writings intended for beginning programmers."

Here, I am not saying, "Learn to build enterprise-level code by just doing, but I am saying, when teaching a beginning programmer, don't beat her over the head with theory - show her just how easy and fun programming and technology and computers can be! I followed the above quote with: "Poor authors can easily make a beginner feel overwhelmed; the sign of a good author is his ability to distract the reader from his innate concerns of the technology’s difficulty and complexity. This is done by showing the reader how to do something, anything."

I agree wholeheartedly that when building enterprise-level apps, you can't just "do" and hope everything works out for the best. I've worked with clients/companies where that's happened, and I've worked with clients/companies where there's been the exact opposite, where they planned like crazy (perhaps a bit too much). Your dog house/building analogy is a good one.

To summarize, I think a good motto for learning *anything*, be it computers or unicycle riding, is to "do". Yes, theory is important, and that needs to be learned to advance to the next level, but I don't believe that theory should come before practice. This is a core belief I try to carry through to the books I write and the classes I teach.

Thanks for your comments, I'm enjoying this discourse...

# re: Where's My Humility? 1/25/2004 1:27 PM Mike Singer

> Yes, theory is important, and that needs
> to be learned to advance to the next
> level, but I don't believe that theory
> should come before practice. This is a
> core belief I try to carry through to
> the books I write and the classes I teach.

Scott, this looks like a bit controversial statement. Perhaps this could be true for the very beginners, for those who are buying books of “…for dummies” series, and for those who are paying money for getting into coding as quickly as possible, say, to catch a job.

But this core belief could not help much for those guys who are shifting to ASP.NET world having a solid background in computer or math related fields. Yes, you have a choice to, but I doubt, a guy as brilliant as you has a moral right to restrict himself to the level of playing with very beginners.

The question “theory or practice should come before?” I’d render to the question “Which level of theory before practice is optimal to prospective audience.

I’m sure it’s a matter of time Microsoft will standardize a set of patterns sort of “Theoretical and practical aspects of New Technologies Mastering” :-)

I’m not trying to convince you in anything, just humbly trying to provide some feedback, your web-site was very useful to me (FAQs, articles, examples).

Many Thanks,
Mike.

# re: Where's My Humility? 1/25/2004 4:56 PM Scott Mitchell

> But this core belief could not help much
> for those guys who are shifting to ASP.NET
> world having a solid background in computer
> or math related fields.

Agreed, but how many people are moving from a solid background in computer science and/or math to Web programming? I don't know about you, but many of the people I've worked with in the past, both as a coworker or as a consultant or trainer, have not had this rigid background. A good percentage received an undergraduate degree in computer science many years ago, but a good percentage too have no such background - they may have majored in business, or some other science, or not gone to college at all.

My fiancee works at a company that does a lot of ASP.NET development - I'd guestimate that about half of those employees actively working on Web development have a comp sci background. In the classes I teach, many folks have non-computer backgrounds.

At my undergraduate university (University of Missouri - Rolla [http://www.umr.edu]), my first class was an "Introduction to Data Structures" class, where we learned about arrays, linked lists, hash tables, trees, heaps, and so on. Each other week during the semester we had to write a program that implemented what we learned - practice and theory, married together. Similarly, moving through the school's cirriculum, each class required assignments that put the theory into practice.

Now, at one extreme you could teach three years of theory before you sit someone down at a computer and say, "OK, let's write our first program." But I would expect only those with a strong mathematical background and a keen interest in theory would stick through those first three years. When you are learning something, you want to do. To touch, to feel, to experience. For example, try to teach someone how to play, say, basketball, but don't let them shoot or dribble. How boring will that be? Do you think someone would want to sit there for endless hours as you explained the rules, the regulations, the objectives, offensive and defensive techniques, strategies, and so forth?

There's a reason people say "practice makes perfect" and not "theory makes perfect." In theory, practice and theory are the same, in practice, theory and practice are different.

# re: Where's My Humility? 1/26/2004 4:14 PM Mike Singer

> For example, try to teach someone how
> to play, say, basketball, but don't let
> them shoot or dribble. How boring will
> that be? Do you think someone would
> want to sit there for endless hours as
> you explained the rules, the regulations,
> the objectives, offensive and defensive
> techniques, strategies, and so forth?

Perhaps you are right, it depends heavily on how a person takes the process of software development – like playing basketball at a college or like playing chess at a master competition. Learning to play chess by just playing without wasting “endless hours” learning chess theory couldn’t lead to a great result, I think:-)

To illustrate this let’s look at “An Extensive Examination of the DataGrid Web Control: Part 14”. You wrote:
***********************************************
That is, if there are 15 FAQ categories, there will be 15 separate database queries. While each database query will only return the needed records, it would be much more efficient to do one large database query at the beginning, getting all of the FAQs, and then only binding the correct ones for each FAQ category row. (The reason this approach is more efficient than the former approach is because the majority of the "cost" of a database query is in setting up and tearing down the connection. We can amortize this building up/tearing down cost by doing just one large query at the beginning.)
***********************************************

In fact, you were stating that one large DataSet and 15 DataView’s over it were much more efficient than 15 optimized (SequentialAccess) DataReaders due to “the majority of the "cost" of a database query is in setting up and tearing down the connection”.

Why not spend a bit of time to dig into the Theory before making the non-obvious statement that resulted to the article’s code?

From “Connection Pooling for the .NET Framework Data Provider for SQL Server” MSDN article:
- Pooling connections can significantly enhance the performance and scalability of your application…
- Maintenance of inactive or empty pools involves minimal system overhead.

From “Performance Comparison: Data Access Techniques” MSDN article DataReader approach nearly doubles throughput for the similar to Part 14 conditions.

And why not mention Multiple Result Sets DataReaders?

I don’t know if beginners deserved “best practices approach”, but I’m sure they didn’t deserve to learn “bad practices approach”:-)

Regards,
Mike Singer :-)

# re: Where's My Humility? 1/26/2004 4:49 PM Scott Mitchell

Mike, maybe we'll have to agree to disagree here. I'm not saying, "No theory," I'm just saying if you don't mix in a good amount of doing you're going to bore the hell out of whomever you're teaching. That I promise you.

>you were stating that one large DataSet
>and 15 DataView’s over it were much more
>efficient than 15 optimized (SequentialAccess)
>DataReaders due to “the majority of the
>"cost" of a database query is in setting up
>and tearing down the connection”.
> Why not spend a bit of time to dig into
> the Theory before making the non-obvious
> statement that resulted to the article’s
>code?

I didn't do any performance testing, but I feel confident in saying that using ONE call to the DB is more efficient than one for each row. Now, where is the break-even point? I don't know, like I said, I didn't do any testing. Maybe doing 15 separate queries is better than one, but what if there are 30 rows in the DataGrid, or 50 or 100? In any event, I invite you to do the perf. testing - I think my way will be more efficient than yours, even for just 15 records.

>Pooling connections can significantly enhance
>the performance and scalability of your
>application

You could define the time it takes to get data as:

T = StartupTime + DataSize * MB/Sec + CloseTime

Where StartupTime and CloseTime are the times needed to create the connection and populate the ADO.NET object (DataSet or DataReader), DataSize is the amount of data being retrieved, and MB/Sec is how many MB of data the RDBMS can get per second. Now, unless the startup time plus closing time for a DataReader is 15 times more efficient than a DataSet, then doing one big query will outperform 15 individual ones.

Futhermore, databases are designed to efficiently retrieve **blocks** of data. So, get as much data as you can at once, rather than having to make multiple connections and requests. That is, getting 10 MB of data at once will be more efficient than fetching 1 MB of data 10 times.

>DataReader approach nearly doubles throughput

Ok, rather than using a DataSet, use a DataReader and dump it into an array. Whatever. The point is, you still want to do just one call to the DB.

When explaining something, you have to guage your audience and understand what it is they want and need to accomplish what it is they're after. Check out my article series on Data Structures on MSDN. I could have written 1,500 pages on data structures, just like Cormen, Liesersen, and Rivest did in Introduction to Algorithms. I could have spent pages on the mathematical proofs and asymptotic analyses, but I didn't. Similarly, I could spend dozens of pages comparing and contrasting performance in using a single DataSet vs multiple calls with DataReaders, and using a DataSet vs. using a single DataReader dumped into an array, and so on. But readers are not wanting that - they're wanting to know how in the hell do I do a master/detail DataGrid.

If you disagree, I invite you to try your hand at writing. Your approach will appeal to some - the high end crowd that has ample experience and is looking for best practices, not solutions to their problems. The lion's share of the market, though, will likely find your writing obtuse and frustrating.

# re: Where's My Humility? 1/27/2004 8:29 AM Mike Singer

> I didn't do any testing. Maybe doing
> 15 separate queries is better than one,
> but what if there are 30 rows in the
> DataGrid, or 50 or 100? In any event,
> I invite you to do the perf. testing
> - I think my way will be more efficient
> than yours, even for just 15 records.

Scott, in my previous post I mentioned MSDN article “Performance Comparison: Data Access Techniques.” Please, have a look at Figure 3. “GetCustomers (Customers=100): throughput and latency” and at Figure 5. GetCustomers (Customers=1000): throughput and latency.

Let’s examine response time needed to fetch 1000 rows (user load 40) using two approaches – first - a DataSet with 1000 rows and the second - 10 DataReaders with 100 rows each.

DataSet - Figure 5 - 1000 ms (user load 40)

10 DataReaders with 100 rows - Figure 3 - 10*80ms=800ms

DataReaders have won here. Perhaps with lesser amount of rows at underloaded server they will lose, but who cares about the response time in this case?

> Ok, rather than using a DataSet, use a
> DataReader and dump it into an array.

I will, because I need to emulate VS6 FlexGrid merging cells not only vertically, but horizontally too. The way to complete it I see now is to create DataGrid programmatically from the array.

> Check out my article series on
> Data Structures on MSDN.

I’ll check them carefully. I need it because I’ll transfer my optimization block from VB6 to ASP.NET.

> If you disagree, I invite you to try
> your hand at writing.

He-he with my Ukrainian English I’ll write a lot :-)

Regards,
Mike.

Title:
Name:
Url:
Enter the code you see:
Comments
Remember Me?