Scott on Writing

Musings on technical writing...

Running the Same Query Against Multiple Databases

One of my clients has a data-driven Software as a Service (SaaS) web application that is used by a dozen different customers. In particular, this client of mine has a web application that's used by various clinics and hospitals to track, schedule, and bill health care-related activities. As discussed in the Multi-Tenant Data Architecture guide from Microsoft, there are three approaches for modeling data that comes from many distinct customers:

  • Separate Databases - each customer has a separate database on the database server.
  • Shared Database, Separate Schema - a single databsae is used, but each customer has a unique set of database tables.
  • Shared Database, Shared Schema - a single set of database tables are used, with a CustomerId column or some other technique used to identify what rows belong to what customers.

Each approach has its pros and cons, and is good reading if you plan on building a SaaS web app.

For security and privacy reasons (important concerns for health care providers), we used the Separate Databases approach - each customer's data is hosted as a separate database on the server. I would not recommend this approach for the vast majority of multi-tenant applications. The most pronounced drawback of this model centers around the pain points associated with managing and administration of these databases. For example, when rolling out a new version or a bug fix, the database changes must be applied to all customer databases. If there is a business logic-type bug discovered, where there may be logically incorrect data in the database that needs to be addressed, you now have to look through all databases to see where the problem occurs and apply the fix across all databases.

One tool that assists with such debugging challenges is the undocumented Microsoft SQL Server stored procedure sp_MsForEachDb. In short, this stored procedure allows you to run a command on every database on the server. You can use it like so:

EXEC sp_MsForEachDb @command1='command to execute on all databases'

Within the @command1 input you can use a question mark (?) to denote that the name of the database the command is currently executing on be injected.

sp_MsForEachDb is most commonly used for administrative tasks such as updating statistics on all databases, but can also be used for debugging or bug hunting in a multi-tenant SaaS architecture that uses separate databases. Here's how: consider that you unearth a bug in the application code that allows for some illogical value in the database. Perhaps there's some business rule that dictates that a patient cannot have their insurance information entered if the patient's primary address is not provided, yet the application did not correctly enforce this rule and as a result there may be some patients with insurance information on file but without a primary address. Your task is to find out what patients in each database fit this description. You could run the following query on every single database:

SELECT p.PatientId, p.PatientName
FROM Patients p
   INNER JOIN PatientInsuranceInformation pi ON p.PatientId = pi.PatientId
WHERE p.Address1 IS NULL AND pi.Active = 1

The issue is that this quickly becomes a pain if you have many different databases. Here's where sp_MsForEachDb comes in - you can use it to run the above on each database with the following statement:

EXEC sp_MsForEachDb @command1 = 'SELECT ''?'' as DatabaseName, p.PatientId, p.PatientName
    FROM ?.dbo.Patients p
       INNER JOIN ?.dbo.PatientInsuranceInformation pi ON p.PatientId = pi.PatientId
    WHERE p.Address1 IS NULL AND pi.Active = 1'

This will output a three column resultset that displays those patients that have insurance information but no primary address along with what database the patient information resides in. The ''?'' part in the SELECT list displays the database name. The question mark character is also used in the FROM and INNER JOIN clauses to indicate the database from which to query against (i.e., you can refer to a table as database.owner.tableName).

The only downside of sp_MsForEachDb is that it runs on all databases, meaning it will run on databases like master, which don't have the application tables. As a result, you'll get SQL errors for those databases which clutter the output a bit. But for the databases specific to the application you will quickly see the results of your query and know, in an instant, what patients in what databases have this particular problem.

posted on Monday, November 17, 2008 8:00 AM

Feedback

# re: Running the Same Query Against Multiple Databases 11/18/2008 12:43 PM Mladen

if you're using SSMS you might want to check this add-in of mine that has a Run on Multiple Databases feature amongst others:
http://weblogs.sqlteam.com/mladenp/archive/2008/10/06/SSMS-Tools-PACK-1.1---now-with-SQL-Server-Management.aspx

Might be helpfull...

# re: Running the Same Query Against Multiple Databases 11/26/2008 6:16 AM Atif Aziz

I use the good old FOR (http://technet.microsoft.com/en-us/library/bb490909.aspx) on the Command Prompt together with SQLCMD to run queries or scripts against multiple databases. Here's a simple example where FOR loops through databases DB1, DB2 and DB3 and runs "SELECT DB_NAME()" against each each database (assuming default SQL Server instance on local machine and trusted connection):

for %i in (DB1 DB2 DB3) do sqlcmd -E -S . -Q "SELECT DB_NAME()"

That's just a very simple example, but FOR and SQLCMD have a whole host of options that can be combined in interesting ways to ease the pain in patching and updating multiple databases in staging or production. It especially helps in preparing a batch that the customer can review and execute on their systems.

# Ruminations on Multi-Tenant Data Architectures 8/19/2009 4:09 PM Scott on Writing

# Ruminations on Multi-Tenant Data Architectures 8/19/2009 4:34 PM Community Blogs

One of the key advantages of web applications is that they can be deployed as a hosted service and accessed

# Ruminations on Multi-Tenant Data Architectures 8/19/2009 4:46 PM BusinessRx Reading List

One of the key advantages of web applications is that they can be deployed as a hosted service and accessed

# A Tool For Querying Multiple Databases 9/8/2009 3:16 PM Scott on Writing

# A Tool For Querying Multiple Databases 9/8/2009 3:37 PM ASPInsiders

I recently blogged about different multi-tenant data architectures , comparing and constrasting the Separate

# A Tool For Querying Multiple Databases 9/8/2009 3:45 PM BusinessRx Reading List

I recently blogged about different multi-tenant data architectures , comparing and constrasting the Separate

Title:  
Name:  
Url:
Protected by Clearscreen.SharpHIPEnter the code you see:
Comments   

My Links

Ads Via DevMavens

Archives

Post Categories

 

I am a Microsoft MVP for ASP.NET.
I am an ASPInsider.
<February 2010>
SMTWTFS
31123456
78910111213
14151617181920
21222324252627
28123456
78910111213

Comment Stats

DayTotal% of Total
Sunday 2046.8%
Monday 42514.1%
Tuesday 51617.1%
Wednesday 55318.4%
Thursday 58019.2%
Friday 54718.2%
Saturday 1886.2%
Total 3013100.0%

Hour1Total% of Total
12:00 AM 772.6%
1:00 AM 812.7%
2:00 AM 682.3%
3:00 AM 822.7%
4:00 AM 692.3%
5:00 AM 1264.2%
6:00 AM 1183.9%
7:00 AM 1806.0%
8:00 AM 1926.4%
9:00 AM 1585.2%
10:00 AM 1876.2%
11:00 AM 1936.4%
12:00 PM 2016.7%
1:00 PM 1846.1%
2:00 PM 1695.6%
3:00 PM 1354.5%
4:00 PM 1153.8%
5:00 PM 1063.5%
6:00 PM 1013.4%
7:00 PM 1073.6%
8:00 PM 923.1%
9:00 PM 882.9%
10:00 PM 893.0%
11:00 PM 953.2%
Total 3013100.0%

Comments by Blog Entry Date/Time

Day Entry MadeAvg.Total
Sunday 4.94158
Monday 4.80384
Tuesday 4.08477
Wednesday 7.47680
Thursday 6.25675
Friday 5.02462
Saturday 4.78177
Total 5.413013

Hour1 Entry MadeAvg.Total
12:00 AM 5.2937
1:00 AM 1.002
5:00 AM 0.000
7:00 AM 3.8550
8:00 AM 3.72134
9:00 AM 6.02295
10:00 AM 5.63276
11:00 AM 4.20193
12:00 PM 6.14350
1:00 PM 3.17133
2:00 PM 5.00230
3:00 PM 7.62320
4:00 PM 4.00108
5:00 PM 6.04169
6:00 PM 4.64116
7:00 PM 8.95188
8:00 PM 8.63164
9:00 PM 5.00115
10:00 PM 6.31101
11:00 PM 4.5732
Total 5.413013

Learn More About Comment Stats
1 - All times GMT -8...


Blog Stats

Favorite Web Sites

My Books

My MSDN Articles